Interview scorecard templates

Machine Learning Engineer interview scorecard template

Score machine learning candidates on getting models into production, not only on building them. This scorecard covers model development, ML system design, data handling, experimentation and code quality.

Customize this scorecard

Free to use. Nothing you type on the scorecard is saved.

Interview scorecard

Machine Learning Engineer

Score each skill from 1 to 5. A 1 is below the bar for this role, a 3 meets it, and a 5 is above it. The examples show what each sounds like. Use 2 and 4 for answers that fall in between.

Model development

Builds, evaluates, and improves models that solve a defined business problem.

Score for Model development

Questions

  • Tell me about a model you built. How did you choose the approach?
  • Describe a time a model performed worse than expected. What did you do?
  • How did you decide your model was good enough to ship?

1: Below the bar

Names a popular algorithm without explaining why. Cannot say how they measured quality or what went wrong.

3: Meets the bar

Explains the approach and a sensible metric. Diagnoses a problem with some guesswork and tries a few fixes.

5: Above the bar

Ties the model choice to the business goal, compares simple baselines, finds root causes of failures, and defines a clear ship threshold.

ML system design

Designs how a model is trained, deployed, monitored, and updated as one working production system.

Score for ML system design

Questions

  • Walk me through a system you designed to serve a model in production. What were the main parts?
  • Tell me about a time a deployed model's quality dropped. How did you find out?
  • How did you decide between batch and real-time predictions on a past project? What did you give up?

1: Below the bar

Describes only training the model. Has no plan for serving, monitoring, or updating it, and can't explain design choices.

3: Meets the bar

Describes the main parts from data to serving. Names some trade-offs and a basic monitoring plan, but misses failure cases.

5: Above the bar

Ties design to business needs, compares options unprompted, and plans for monitoring, retraining, rollback, and cost before problems show up.

Data handling

Checks, cleans, and safeguards training data so models learn from reliable, representative examples.

Score for Data handling

Questions

  • Tell me about a time you found a problem in a dataset. How did you spot it?
  • Describe how you split data for training and testing. What did you guard against?
  • Tell me about a time data changed after launch. How did you respond?

1: Below the bar

Trusts data as given and splits it randomly without checking. Can't say how bad data would show up.

3: Meets the bar

Checks for missing values, duplicates, and leakage, and documents fixes. Reacts to data changes once they cause visible problems.

5: Above the bar

Audits data before modeling, designs splits to prevent leakage, and sets up checks that catch drift early.

Experimentation

Designs and runs controlled tests to learn whether a change actually improves results.

Score for Experimentation

Questions

  • Tell me about an experiment you ran to test a change. How did you set it up?
  • Describe a time results looked better but you doubted them. What did you do?
  • Tell me about an experiment that failed. What did you decide afterward?

1: Below the bar

Changes several things at once and compares by gut feel. Reports one good number without checking if it is real.

3: Meets the bar

Compares against a baseline and changes one thing at a time. Checks results for noise when prompted and records outcomes.

5: Above the bar

Defines success before starting, controls for noise unprompted, and explains what the result does and doesn't prove. Turns failures into the next test.

Code quality

Writes ML code that teammates can read, test, reuse, and safely change over time.

Score for Code quality

Questions

  • Tell me about ML code you wrote that others had to maintain. What made it easy or hard?
  • Tell me about a time you tested or reviewed code in a model pipeline. What did you check?
  • Tell me about a time you cleaned up messy research code. What did you change first, and why?

1: Below the bar

Treats working notebooks as finished. Has no tests or reviews to describe and blames others for hard-to-read code.

3: Meets the bar

Writes readable, organized code and some tests. Reviews others' work and can describe one cleanup, though it was reactive.

5: Above the bar

Designs code for reuse and testing from the start. Tests data and model steps, gives useful review feedback, and explains trade-offs.

Overall recommendation

Decide on your own, before you compare notes with the rest of the panel.

Overall recommendation

Use this scorecard in PontaHR

With PontaHR, your team fills out scorecards on the candidate's profile. Compare ratings and feedback side by side when it's time to decide.

See how scorecards work in PontaHR, then try it with a guided free trial.

More interview scorecard templates

Hiring for a different role? Make your own scorecard