The Unbearable Lightness of Overfitting

Sincerity. It looks like discipline. It smells like care. It definitely boils down to sweating the details. Sweating them to the point of exhaustion. This is how obsessions are born.

As an applied scientist/engineer, I often find myself hacking existing science and math to make it work within the constraints imposed by the real world. In designing and building recommender systems, content understanding systems, language models, image recognition systems, I’ve come across and created a few such recipes for hacking the scientific complexity. These recipes in abstraction are simply stories of problem-solving.

During the pandemic, in one spontaneous moment, I had decided to share some of the technical details I’ve obsessed over in my journey in artificial intelligence. In this blog, I will share some of these stories, along with autobiographical anecdotes and some slightly ignored best practices.

All in all, these are just stories of scientific sincerity that have faced extreme resistance from the devilish details hiding in plain sight.

I look forward to sharing my journey with you on a weekly basis, and I hope you join me in exploring the world of artificial intelligence through the lens of scientific sincerity.