Use these questions to interview Data Scientist candidates at {{Company name}}. They are grouped by competency; assign the statistics and modeling groups to your most technical interviewers and probe every claim for the project behind it. The strongest signal comes from how candidates reason about validity and failure, not from naming the latest model architecture.

Statistics and experiment design

• Design an experiment to test whether a new onboarding flow improves retention. Walk me through unit of randomization, duration, and guardrails.
• Your test shows a three percent lift with a p-value just under the threshold. The PM wants to ship today. What do you say?
• Explain a time you caught an invalid experiment — contamination, peeking, selection bias. How did you catch it?
• When is an observational analysis good enough to make a call without an experiment?
• How would you measure the effect of something you cannot randomize, like a price change in one market?
• Explain confidence intervals to a non-technical stakeholder in two sentences.

What good looks like: strong candidates reason about validity threats before statistics — what could contaminate the test, whether the metric is sensitive enough, how long until novelty effects wash out — and they resist the pressure question with a concrete recommendation, not a lecture. Weak candidates recite formulas but have never killed an invalid experiment in real life.

Machine learning and modeling

• Walk me through a model you shipped to production end to end. What was the baseline, and did you beat it by enough to matter?
• How did you decide the simplest approach was not sufficient? Show me the escalation from heuristic to model.
• Tell me about a model that performed well offline and failed in production. What was the gap?
• How do you detect and handle training-serving skew and data drift?
• What features leaked in a project you have seen, and how was the leak found?
• How do you evaluate a model when the business cost of false positives and false negatives is very different?

What good looks like: the best answers start from the baseline and the decision the model serves — candidates who tried a heuristic first, who can name their leakage checks, and who monitor production performance against a business metric. Be wary of candidates who jump to complex architectures and cannot say what the model changed for the business.

Coding and data engineering fluency

• Walk me through the structure of your last analysis repository. How would a teammate reproduce your results?
• Live exercise: clean this messy dataset — mixed types, duplicated keys, suspicious outliers — and narrate your decisions.
• How do you decide what belongs in a notebook versus a tested pipeline?
• Describe the handoff of one of your models to engineering. What broke, and what did you standardize afterwards?
• What does code review look like on your current team, and what do you personally look for?

Business impact and product sense

• Which of your projects created the most measurable value? Walk me through the number.
• Tell me about a project you stopped because the expected impact did not justify the effort.
• How do you translate an ambiguous business question into a modeling problem? Give a real example.
• A stakeholder wants machine learning where a rule would do. How do you handle it?
• What is the most impactful analysis you did that involved no modeling at all?

Communication and collaboration

• Explain your most complex project to me as if I ran the sales team. Two minutes.
• Tell me about a time your results contradicted what leadership wanted to hear. What happened?
• How do you present model uncertainty and limitations without undermining trust in the work?
• Describe working with engineers and PMs on a shipped feature. What was your role in each phase?
• Tell me about a time you changed your analysis based on a colleague's challenge.