Interview scorecard
Machine Learning Engineer
Score each skill from 1 to 5. A 1 is below the bar for this role, a 3 meets it, and a 5 is above it. The examples show what each sounds like. Use 2 and 4 for answers that fall in between.
Model development
Builds, evaluates, and improves models that solve a defined business problem.
Questions
- Tell me about a model you built. How did you choose the approach?
- Describe a time a model performed worse than expected. What did you do?
- How did you decide your model was good enough to ship?
1: Below the bar
Names a popular algorithm without explaining why. Cannot say how they measured quality or what went wrong.
3: Meets the bar
Explains the approach and a sensible metric. Diagnoses a problem with some guesswork and tries a few fixes.
5: Above the bar
Ties the model choice to the business goal, compares simple baselines, finds root causes of failures, and defines a clear ship threshold.
ML system design
Designs how a model is trained, deployed, monitored, and updated as one working production system.
Questions
- Walk me through a system you designed to serve a model in production. What were the main parts?
- Tell me about a time a deployed model's quality dropped. How did you find out?
- How did you decide between batch and real-time predictions on a past project? What did you give up?
1: Below the bar
Describes only training the model. Has no plan for serving, monitoring, or updating it, and can't explain design choices.
3: Meets the bar
Describes the main parts from data to serving. Names some trade-offs and a basic monitoring plan, but misses failure cases.
5: Above the bar
Ties design to business needs, compares options unprompted, and plans for monitoring, retraining, rollback, and cost before problems show up.
Data handling
Checks, cleans, and safeguards training data so models learn from reliable, representative examples.
Questions
- Tell me about a time you found a problem in a dataset. How did you spot it?
- Describe how you split data for training and testing. What did you guard against?
- Tell me about a time data changed after launch. How did you respond?
1: Below the bar
Trusts data as given and splits it randomly without checking. Can't say how bad data would show up.
3: Meets the bar
Checks for missing values, duplicates, and leakage, and documents fixes. Reacts to data changes once they cause visible problems.
5: Above the bar
Audits data before modeling, designs splits to prevent leakage, and sets up checks that catch drift early.
Experimentation
Designs and runs controlled tests to learn whether a change actually improves results.
Questions
- Tell me about an experiment you ran to test a change. How did you set it up?
- Describe a time results looked better but you doubted them. What did you do?
- Tell me about an experiment that failed. What did you decide afterward?
1: Below the bar
Changes several things at once and compares by gut feel. Reports one good number without checking if it is real.
3: Meets the bar
Compares against a baseline and changes one thing at a time. Checks results for noise when prompted and records outcomes.
5: Above the bar
Defines success before starting, controls for noise unprompted, and explains what the result does and doesn't prove. Turns failures into the next test.
Code quality
Writes ML code that teammates can read, test, reuse, and safely change over time.
Questions
- Tell me about ML code you wrote that others had to maintain. What made it easy or hard?
- Tell me about a time you tested or reviewed code in a model pipeline. What did you check?
- Tell me about a time you cleaned up messy research code. What did you change first, and why?
1: Below the bar
Treats working notebooks as finished. Has no tests or reviews to describe and blames others for hard-to-read code.
3: Meets the bar
Writes readable, organized code and some tests. Reviews others' work and can describe one cleanup, though it was reactive.
5: Above the bar
Designs code for reuse and testing from the start. Tests data and model steps, gives useful review feedback, and explains trade-offs.
Overall recommendation
Decide on your own, before you compare notes with the rest of the panel.