March 13, 2020
Data science has been one of the fastest growing fields over the past five years. Since starting Prolego I’ve reviewed more than 1,000 data scientists resumes, interviewed hundreds of candidates, and hired more than a dozen for my company or our clients. I’ve watched data science careers thrive … and others falter. As we enter this economic downcycle managers will begin identifying their most valuable people—the ones they will fight to retain in the event of layoffs.
What follows is advice for making yourself invaluable to any employer.
You may have the false impression that a chronic shortage of data scientists will insulate you from layoffs. This isn’t true.
There is a shortage of talent in every technical skill. We don’t have enough talented java programmers, Webflow designers, Cobol programmers, and data scientists. There are fewer people who can do machine learning, but there are also fewer jobs which require it. My clients have no problem hiring data scientists when they have the right recruiting strategy—there are a lot of very smart people attracted to this field.
Here’s the reality: Basic machine learning skills which allow you to rank high in Kaggle contests are not sufficiently differentiated—enough other people have them. Here are 3 approaches for becoming an exclusive contributor on your team.
I’ve met hundreds of data scientists who can clean data, perform EDA, train and evaluate models in Jupyter notebooks. Many data scientists come from a research or statistics background and are quite competent using basic Python libraries in notebooks. Far fewer can take code written in notebooks and extract it into standalone Python classes and methods. Very few can build these libraries with production-quality software.
(Obviously if you’re only comfortable with R, SAS, or MATLAB you are at a competitive disadvantage—stop making excuses and learn Python).
One of the biggest challenges our clients have is putting machine learning models into production. Data scientists who can write production-quality (or close to it) software make a huge impact on any project.
Python code like ...
is fine for rapid development in exploratory notebooks. But a professional software engineer only sees a horrifying example of magic numbers, ambiguous variables and code which is difficult to read. A data scientist who can put these basic functions into libraries with PEP8 standards will improve velocity for the whole team.
In case you’re wondering, good Python skills are a must-have for every engineer at my company.
Some projects benefit from leveraging cutting edge research or creating novel models (usually neural networks). Few data scientists have the research skills to:
But many companies ultimately discover they have intractable problems with high potential business impact. If you are skilled at solving them, you can carve out a very valuable role for yourself.
Some data scientists—usually those with a PhD in an experimental discipline—are drawn to this type of work. These engineers have spent years developing mathematical models and building the software and data to test them. We usually see these skills in people with advanced degrees in economics, physics, and astronomy.
Keep in mind that your employer won’t have the appetite for the years of toil which often accompany academic or lab research projects. We break these types of projects into 3-month milestones and systematically identify and remove risk.
Machine learning projects are complex and touch nearly every part of the organization. Model releases have to be coordinated with IT. Legal needs to be consulted about usage rights. Users need to understand false positives and negatives. The list is endless.
Too often the data scientist is tasked with these activities. Many don’t enjoy this type of work—and usually are not very good at it.
But some data scientists love being the connector who keeps the projects running on track. They are good at simplifying data science work and explaining the business impact for executives. Does this sound like you? If so, consider doubling-down and talking to your leadership about carving out a unique role in your company. We call this role an “AI Product Manager”.
You have a unique competitive advantage because most traditional project/product managers do not have your knowledge. They don’t know how to interpret what data scientists do or give them specific feedback to maximize their efficiency.
I don’t know how long the economic fallout from the Covid-19 virus will last. I don’t know if layoffs will result from this downturn as they have in the previous two.
But I do know that every large company is looking for ways to get more value out of their data science team. Start making yourself an invaluable team member and you’ll be prepared to thrive regardless of the market.
February 22, 2021
Entrusting your business decisions to AI does indeed create new risks, and you need new policies and procedures to mitigate them. However, sensational news headlines lead companies to take ineffective approaches that result in unnecessary delays and frustration. After helping some of the world’s largest companies develop model governance policies, I’ve learned a better way to manage the complexities of AI. In this issue of FeedForward, you’ll explore some of my top recommendations.
February 1, 2021
We recently released volume 1 of Adventures in AI, the world’s first AI comic book. It was a smashing success, and many AI Leaders asked us how and why we created it. In this post we share our process and thoughts on how you can use an engaging and compelling story to communicate the value of AI to your organization.