I get a lot of messages on X asking me what projects you should do to get the experience necessary for a job in machine learning. My answer to this question is always the same: the best project is the one you come up with.
This means identifying a problem that can be solved with machine learning, understanding how the technology is used to solve the problem, planning and implementing the solution, and making it available for others to use. Solving your own problem shows your ability to design machine learning systems end-to-end.
Huge emphasis on making your project available to others—a lot of people leave this out. Making a product widely available and iterating on feedback is the most important piece of a software system. It’ll be tough to relate a project you’ve built to the skills needed for a job while in an interview if it isn’t available.
Even after responding to messages on X with the above, I frequently get a response similar to I want you to determine the project for me and I can implement it. I won’t do that, but I understand this sentiment. Here are my thoughts:
Working on machine learning in industry doesn’t require just the ability to solve machine learning problems but also the ability to identify them. If I identified the problem for you, you wouldn’t be able to show your ability to do this for a company.
Take a leap and don’t be afraid to learn as you go. A lot of new machine learners get stuck in the learning phase. At some point, you have to implement. Just like other technical fields, the most important pieces of a machine learning education take place on the job (or in this case while working on something machine learning-related). Don’t be afraid to be wrong: identify a problem and try to solve it. If that doesn’t work out, pivot. If you don’t know something, give yourself time to learn it.
The second point was something I really struggled with when I began machine learning research and later when I moved to industry. I was intimidated by the things I didn’t know. It took me a while, but I realized I was looking at learning machine learning incorrectly. It was almost as if, in my mind, advancements in machine learning and artificial intelligence were linear. I felt like I was constantly behind and I needed to catch up. In reality, AI was developing rapidly in many different directions and that mindset meant I would never would feel confident enough in my machine learning knowledge.
The reality of machine learning is that no one understands everything because it’s a large, rapidly developing field. There’s a reason research is done in teams and often even in collaboration between teams. The key to becoming confident for me was focusing on the piece of machine learning I was interested in and being open to not knowing everything about the others.
I still keep up with areas I don’t focus on, but I don’t worry about doing that myself. Instead, I let other people help me out. This is why part of the machine learning road map I created is a list of people that can help you learn more about machine learning. I also frequently pose questions on X about things I don’t understand to experts in those areas and I almost always get a response. This is why the tech community is so strong on X.
It took me a while to work all of this out. Strangely enough, the moment that helped me work it out was when I was reading a book with my daughters. In the book1, Chili tells her daughter Bluey about when Bluey was a baby. As a new mom, Chili was super competitive with other moms in the baby group because she was concerned about Bluey hitting her developmental milestones. Throughout the story she struggled with this. At the end of the story, one of the moms chats with Chili and shows her a picture of her nine children. Chili hadn’t known the other mom had more than one kid. The other mom told Chili she had learned a thing or two after having that many children and she could see that Chili was doing great.
Chili told Bluey that from that point on she “decided to run [her] own race.” I love this because it applies to everything in life—machine learning or anything else you’re working on. I like the imagery because other people might be running a similar race and sometimes those people might even run with you and help you out, but at the end of the day, the race you’re running is your own and should be taken at your pace.
I hope the message impacted my daughters as much as it did me, but even if it didn’t, I hope it can be helpful to you.
A huge welcome to everyone who has joined Society’s Backend recently! I appreciate all of your support. It means the world to me that you enjoy reading my writing. I’m in the middle of a move (which, it turns out, is a time consuming task with 5 young kids), but once my office is set up in the new house I plan to share even more information in audio and video form as well. Follow me on YouTube, X, and subscribe to Society’s Backend if you haven’t.
Thanks to
, , and for letting me guest post for them with their audience. If you haven’t checked out the articles I’ve done in collaboration with them, here they are:A special thanks to my paid supporters! Your support allows me to help almost 1000 people better understand machine learning. I’m excited to bring more benefits to supporters like more learning resources and access to my notes (I’ll also teach you how I use them). If you want to support Society’s Backend, you can for just $1/mo for your first year:
I’ve also added two more ways to support Society’s Backend and get access to paid benefits recently! You can:
Pay once for a lifetime access to avoid a subscription.
Bluey: 5-Minute Stories, Penguin Random House LLC, New York, New York, 2022, pp. 127–158.
Really like your balanced tone in many respects. Is there still a shortage of machine learning scientists? It feels like there aren't enough Newsletter about how to transition from a software engineer into one.
My toddler loves Bluey, but it's just as good for parents and for machine learning engineers too!
Great post, with some wonderful advice.
PS The Bluey 'Sleepytime' episode is a masterpiece!
https://www.youtube.com/watch?v=Au-yaca8eMA