Overfitting

“Something I want to try is to move somewhere to be alone and to work on things. All I would need is my laptop.”

My friend Sasa told me he was thinking about moving somewhere, to be alone and to work on personal projects. That all he needed was his laptop and the internet. This way he would be free from distractions and be able to fully focus on projects.

I said to try renting a place for a weekend and try that out before committing to fully moving. That way you get the experience and you’ll have a better idea of if being alone is the missing factor in working on side projects. In other words, you’ll have a better idea of if not being alone is the reason you’re not working on projects without such a drastic change. Because it’s possible that he’s wrong and being alone isn’t the sole reason, and to move seemed like overfitting for something he thinks he needs to do.

A while ago I was interested in doing machine learning stuff. Machine learning could be used in so many ways to do so many things and that’s what fascinated me. But instead of learning and practicing it, my first instinct was to research “what kind of computer should I build to do machine learning?”

At the time I only had a 15" MacBook Pro and I knew that it didn’t have the hardware to run really big models fast. I was obsessed with building a computer specifically built for machine learning, yet I didn’t even get to a point where my laptop’s hardware was the issue. This is overfitting.

I spent a lot of time researching Nvidia graphics cards because CUDA was the big thing. I looked at the differences in ram for GPUs and how it would affect the machine learning workflows I thought I would run. For example, the difference between a GTX 1080 and the GTX 980 was an astonishing 5GB of ram and that would allow me to run bigger models. I learned all these things about building computers that would work great for machine learning that I never did any machine learning.

So the wanting to build a dedicated computer was overfitting. Instead, I should have just done machine learning stuff on my laptop. And yes it would have been slower doing computations on a CPU but at least I would have done something. Isn’t that the important thing anyway?

p.s. From Wikipedia: Overfitting is “the production of an analysis that corresponds too closely or exactly to a particular set of data, and may, therefore, fail to fit additional data or predict future observations reliably”.