Machine learning interview questions and answers
Questions that test whether you can use machine learning, not only define it: accuracy that misleads, leakage, overfitting, and choosing between models.
1. A colleague says the new churn model is great because it's 82% accurate. What question would you ask, and why?
I'd ask what always predicting 'stays' would score. If about 81% of customers stay, the baseline is already 81%, so 82% is barely better than no model at all. I'd also check that the 82% was measured on customers the model never saw during training.
2. Explain why a model can score 100% on its training data and still be bad, and how a held-out set reveals it.
A flexible model can memorize every training example, noise included, so its training score measures memory rather than its ability to predict. A held-out set of customers it never saw asks the real question: how does it do on new cases? If the held-out score drops to near or below a simple baseline, it memorized instead of learning.
3. A colleague's fine-tuned model has falling training loss but rising validation loss after epoch 2. Explain what's happening and two things to try.
That's high variance: after epoch 2 the model is memorizing the training examples instead of the general task, so it gets better on them and worse on new data. I'd keep the epoch-2 checkpoint (early stopping), then try more or more varied training examples and stronger regularization such as weight decay, each of which reduces how much the model can bend to the specific examples.
4. Your team built an LLM prompt that flags support tickets as "legal risk". It's 97% accurate. Explain why that may not be good enough, and what you'd report instead.
If only about 3% of tickets are legal risks, a prompt that never flags anything is also 97% accurate, so the number says nothing. I'd label a sample, build the confusion matrix and report recall (how many real legal-risk tickets we caught) and precision (how many flags were real). Missing a legal threat is far worse than a false alarm, so I'd push recall high and accept lower precision.
5. Your training loss keeps falling, but the loss on validation data started rising after epoch 8. What's happening and what would you do?
The model has started overfitting: it keeps improving on the training data by fitting its noise, while getting worse on new data. I'd keep the weights from around epoch 8 where validation loss was lowest (early stopping), and consider regularization or more data. Training loss alone can't tell you this, which is why you track both.
6. A teammate scaled all 600 customers' features first, then ran 5-fold cross-validation, and got a slightly higher score than you. Explain the problem and the fix.
Scaling everything first means the means and standard deviations already include the validation folds, so information about the held-out customers leaks into training and the score is a little optimistic. The fix is to put the scaler and model in one pipeline and cross-validate the pipeline, so each fold's scaler learns only from that fold's training rows.
7. Explain why preparation steps should be fit on the training data only, using standardization as the example.
Standardizing means subtracting a mean and dividing by a standard deviation, and those numbers are learned from data. If they're computed on everything, the test set has influenced training, so the test score is a little too rosy. And in production you only have the training statistics. So learn them on the training rows and apply exactly those numbers to the test set and to new customers.
8. Your churn model has an AUC of 0.82, and finance wants to use its probabilities to forecast next quarter's lost revenue. Explain what else you need to check.
AUC says the model ranks customers well, not that its probabilities are right. A forecast sums the probabilities, so they need to be calibrated: I'd bucket customers by predicted probability on held-out data and check that each bucket's real churn rate matches (about 30% of the 0.3s churn, and so on). If they're off, I'd recalibrate the model before giving finance any numbers.
9. Your gradient-boosted churn model beats logistic regression by 0.01 AUC but is 50× slower and hard to explain to the retention team. How would you decide which to use?
First I'd check whether 0.01 AUC actually changes who gets flagged and what that's worth, and whether it's even outside the noise of the test set. If the business gain is small, the logistic regression is the better product: faster, easier to explain to the retention team, easier to maintain and debug. If the gain does matter, I might ship the boosted model and give the team simpler explanations per customer.
10. Explain why unscaled features make it hard to find a good learning rate.
If one feature is in hundreds of dollars and another is a 0-to-1 share, the loss changes far faster along the first weight than the second, so the surface is a long narrow valley. Any rate small enough not to blow up along the steep direction crawls along the flat one, and the path zig-zags. Standardizing the features rounds out the valley so one rate works for all weights; Adam's per-weight step sizes also help.
11. Explain how a decision tree decides its first question, in plain language.
It tries every feature and every cutoff it could ask about. For each candidate it splits the customers into two groups and measures how mixed each group still is, using Gini impurity weighted by group size. It picks the question that leaves the groups purest, then asks the same thing again inside each group, one greedy step at a time.
12. A retention offer costs $5 and keeping a churner is worth $150. Explain how you'd choose the threshold for sending the offer, using the model's probabilities.
Send the offer when its expected benefit beats its cost: if p is the churn probability and the offer saves the customer, that's p × $150 > $5, so p above about 0.03. If the offer only works some of the time, divide by that success rate. Either way the threshold should be far below 0.5, and I'd confirm it on held-out data by looking at how many flagged customers really churn.
13. Your lifetime-value model gives a coefficient of +$40 for being a loyalty-program member. Your manager says 'so enrolling customers adds $40 each'. How do you respond?
The +$40 says members are worth $40 more than otherwise similar non-members in our data, not that membership causes it. Customers who join are probably already more engaged in ways the model doesn't see, so enrolling everyone else may add much less. To know the real effect we'd need an experiment, like offering membership to a random half.
14. You're forecasting how many rain jackets to stock. Running out loses a $60 margin per missed sale; leftover stock costs $15 per jacket to discount later. Would plain MSE be the right loss? Explain.
No. MSE punishes being 10 jackets short exactly as much as being 10 over, but here a shortfall costs $60 per jacket and an excess only $15. The loss should be asymmetric: penalize under-forecasting about four times more, which in practice means forecasting a high quantile (around the 80th percentile) of demand rather than the average.
15. Explain why you must scale features before k-NN, using days since last order and loyalty membership as the example.
k-NN picks neighbors by distance, which adds up the squared differences in each feature. A difference of 100 days contributes 10,000 while a membership difference contributes at most 1, so neighbors are chosen almost entirely by recency and membership is ignored. Standardizing both to mean 0 and spread 1 lets each feature count.
16. Your manager wants to segment 50,000 customers "with AI". Explain how you'd do it with clustering and how you'd know the segments are any good.
I'd pick behavioral features like order frequency, basket size, discount use and recency, scale them, and run k-means for k from 2 to 8. The elbow and silhouette scores narrow k down, then I'd profile each option, averaging features plus churn and lifetime value per cluster, and choose the k whose groups are clearly different and each suggest a campaign. There's no accuracy to check, so I'd confirm the segments are stable by re-running on last quarter's data and useful by seeing whether they really respond differently.
17. A colleague shows a t-SNE plot of customer embeddings and says, "Segment A is twice as big and much further from B than from C, so A is our most distinct segment." Explain what's wrong and how to check the claim properly.
t-SNE keeps neighbors together but doesn't preserve how big clusters are or how far apart they sit, and a different perplexity or seed can rearrange the map. To check the claim I'd count how many customers are in each segment, and measure distances between segment centroids in the original embedding space (or how easily a classifier tells the segments apart). Those numbers, not the picture, say which segment is most distinct.
18. Explain how a language model's final layer is a multiclass classifier.
At each position the model turns its hidden state into one score per token in its vocabulary with a linear layer, and softmax turns those into probabilities: a 100,000-class classifier. Training compares that distribution with the token that actually came next using cross-entropy, the same loss and gradient as softmax regression.
19. You're building a model to predict which website visitors will buy today. List two features you'd use and one you must not, with reasons.
I'd use pages viewed so far in the session and whether they're a returning visitor, since both are known while they browse. I must not use 'reached the order confirmation page' or 'payment method': those exist only because they bought, so they'd leak the answer and the model would fail in real use.
20. Suggest three engineered features for predicting whether an email subscriber will unsubscribe, and say why each might help.
Days since they last opened an email, since recency of engagement is the strongest warning sign; emails received per week, to capture fatigue; and the change in open rate this month versus their own average, because a falling trend says more than a level. All three are known before they unsubscribe.
21. Explain overfitting to a colleague using a prompt-engineering example.
It's when something fits the examples you tested on rather than the general task. With prompts: you keep adding special-case instructions until all 30 eval examples pass, and the prompt now handles those 30 well but behaves oddly on the 31st. A held-out set you only check at the end, and more varied examples, show whether the improvements are real.