Deep learning interview questions and answers
Questions on how neural networks train and fail: backpropagation, activations, vanishing gradients, optimizers, normalization, and debugging a model that won't learn.
1. A teammate says their new model "doesn't learn" and wants to try a bigger network. Describe what you'd check first, in order, and why.
Before touching the architecture I'd check the starting loss against ln(number of classes), which catches badly scaled outputs. Then I'd try to overfit 16 examples: if the setup can't even memorize those, the problem is in the model, loss or loop, and a bigger network won't help. Next I'd print some inputs with their labels after preprocessing and compare against a simple baseline. Finally I'd look at the learning rate and per-layer gradient norms, changing one thing at a time with a fixed seed.
2. Explain backpropagation to a marketing colleague, using an analogy if it helps.
The network makes a prediction, and we measure how wrong it was. Backpropagation then works backwards from that error through each layer, working out how much each weight contributed to it: a bit like attribution, but for every weight in the model. Each weight is then nudged slightly in the direction that would have reduced the error. The clever part is that one backward sweep gives every weight's nudge at once, so this works even with billions of weights.
3. Explain to a colleague why deep networks trained badly with sigmoid activations and better with ReLU.
Training sends the error signal backwards through every layer, and at each one it gets multiplied by the activation's slope. A sigmoid's slope is at most 0.25 and close to 0 when the neuron is saturated, so after many layers the signal reaching the early layers is tiny and they barely learn. ReLU's slope is exactly 1 for positive inputs, so the signal passes through undiminished. The catch is that a ReLU stuck on negative inputs passes nothing at all, which leaky ReLU and GELU fix.
4. Explain why a 50-layer network can be trained today but couldn't easily be in the 1990s.
Backpropagation multiplies one factor per layer, and with the sigmoid activations and simple initialization of the time those factors were mostly well below 1, so after dozens of layers almost no gradient reached the early layers. Since then a stack of fixes keeps the factors near 1: ReLU-type activations with slope 1, initialization scaled to the layer width, normalization layers that reset the scale at every layer, and residual connections that give gradients a direct path. Clipping catches the occasional explosion. With all of them, 50 or even hundreds of layers train fine.
5. An untrained 10-class network reports a loss of about 2.3 on its first batch. Explain why that number is reassuring rather than alarming.
An untrained network with small random weights gives roughly equal probability to all 10 classes, about 0.1 each. Cross-entropy is −log of the correct class's probability, and −log(0.1) = ln 10 ≈ 2.30. So 2.3 is exactly what a correctly wired, untrained network should report. If it started at 8 or 20, I'd suspect a bug or badly scaled initial weights.
6. Explain why training on small random batches works even though each batch gives only an approximate gradient.
Each batch is a random sample of the data, so its average gradient points in roughly the same direction as the gradient over all the data: right on average, just noisy. It's also much cheaper, so instead of one exact step per pass you get dozens or hundreds of approximate ones, and the errors largely cancel out over many steps. The noise can even help by jostling the model out of poor spots; a smaller learning rate or bigger batch reduces it when it's too much.
7. A colleague's fine-tuning run diverges (loss becomes nan) with Adam at a learning rate of 0.01. Explain what's likely happening and what to change.
Adam moves every weight by roughly the learning rate each step, so 0.01 means large jumps, far too big for pretrained weights that are already in a good place; a few such steps can blow the loss up to nan. I'd drop the learning rate to something like 1e-5 to 1e-4 and add a short warmup, and clip gradients as a safety net. If it still diverges, then I'd look at the data and the numeric precision.
8. Explain why a very deep network without normalization might not learn at all, and how normalization layers help.
In a deep network each layer's output scale depends on all the layers before it, so small imbalances compound: activations drift until units saturate or blow up, and the gradients reaching the early layers become tiny or huge. A normalization layer re-centers and re-scales the activations at every layer, then lets the network adjust them with learned γ and β. Every layer keeps seeing well-behaved numbers, so gradients flow, larger learning rates become safe, and the starting weights matter less.
9. Explain why dropout reduces overfitting, using the idea of an ensemble.
Every training step switches off a different random set of neurons, so each step trains a different sub-network, and all of them share the same weights. At test time the full network, with the scaling built in, behaves roughly like the average of all those sub-networks, and averaging many models is a classic way to cut overfitting (like a random forest averaging trees). It also stops neurons from relying on particular partners, so the network learns features that hold up when parts are missing.
10. A teammate's 30-layer network outputs NaN after the first step. Explain how initialization could cause this and how you'd check.
If the weights are too large for the layer width, every layer multiplies the size of the activations, and after 30 layers they overflow to infinity, which turns into NaN in the loss and gradients. I'd run one batch through the freshly initialized network and print the spread of the activations after each layer: if it grows layer by layer, initialization is the culprit. The fix is to scale weights by fan-in (He for ReLU, Xavier for tanh) or use the framework's defaults, and then check the learning rate too.
11. Explain why a convolutional network needs far fewer parameters than a fully connected network for images, and why that's an advantage beyond saving memory.
A fully connected layer gives every pixel its own weight for every unit, so the parameter count grows with the image size. A convolutional filter is tiny, say 3 × 3 per channel, and the same filter is reused at every position. Besides saving memory, that means far less to learn from the same data, so less overfitting, and it bakes in a sensible assumption: a pattern learned in one part of the image is recognized anywhere else it appears.
12. Explain the two main limitations of RNNs that transformers addressed.
First, an RNN is a chain: step t can't start until step t−1 is done, so a long sequence can't be processed in parallel and training is slow on GPUs. Second, the entire past has to fit into one fixed-size hidden vector, and the training signal fades across many steps, so the network struggles to connect things that are far apart. Transformers replace the chain with attention: every position looks directly at all earlier positions in one parallel step.
13. Explain, without math, how a two-layer network can learn a pattern a single neuron can't.
A single neuron can only ask one weighted yes/no question, which splits the data with a straight line. A hidden layer asks several such questions at once, for example 'is at least one signal on?' and 'are both on?'. The output neuron then combines those answers, 'at least one but not both', which no single straight line could express. Each layer turns the data into new features until the final decision becomes simple.
14. Explain how a framework like PyTorch can compute gradients for a model nobody differentiated by hand.
As the model runs, the framework records every small operation it performs, like multiplies, additions and ReLUs, as a graph. Each of those operations has a simple, known local derivative. To get gradients, it walks the graph backwards from the loss, multiplying local derivatives along each path (the chain rule) and adding the contributions where a value was used more than once. It keeps the forward-pass values it needs for that, which is why training is memory-hungry.
15. Explain the difference between model.eval() and torch.no_grad(), and why evaluation code normally uses both.
model.eval() changes how certain layers behave: dropout stops dropping and batch norm uses its stored averages, but gradients are still tracked. torch.no_grad() stops PyTorch from recording the computation graph, which saves memory and time, but leaves layer behavior alone. Evaluation wants both: the layers in their inference behavior so the predictions are right, and no graph because we won't call backward. Then model.train() switches back before the next training step.
16. Walk through what happens, in order, during one manual training step in PyTorch, and why each part is needed.
First the forward pass: run the model on a batch and compute the loss, while autograd records every operation. Then clear the previous gradients, because PyTorch adds new gradients to old ones. loss.backward() walks the recorded graph and stores the gradient of the loss in each parameter's .grad. Finally, inside torch.no_grad() so it isn't recorded, move every parameter a small step against its gradient. Repeating this over many batches is training.
17. Explain what the shape (16, 128, 768) usually means inside a language model, and what multiplying it by a (768, 3072) weight does.
It's a batch of 16 sequences, each 128 tokens long, with a 768-number feature vector for every token. Multiplying by a 768 × 3072 weight matrix applies the same linear layer to every one of those token vectors independently, giving shape (16, 128, 3072): each token now has 3,072 features. That's exactly the expansion step inside a transformer block's MLP.
18. Your company wants to fine-tune an open 7B model and asks whether one 24 GB GPU is enough. Explain how you'd reason about it.
Running the model is fine: 7 billion parameters at 2 bytes each is about 14 GB. Fully fine-tuning it is not: with mixed precision and Adam you need around 16 bytes per parameter, roughly 112 GB before activations. So on one 24 GB card I'd use parameter-efficient fine-tuning, LoRA, or QLoRA with the base model quantized to 4 bits (about 3.5 GB), training only small adapter matrices, and keep the batch size and sequence length modest because activations add to it. Otherwise we'd need to rent bigger or multiple GPUs.
19. A colleague says, "Our lead-scoring model is a single neuron; let's just add more input signals to make it smarter." Explain when that will and won't help.
A single neuron adds up weighted signals, so it can only draw a straight-line boundary: every signal pushes the score up or down on its own. More signals help if their effects really add up. But if what matters is a combination, like company size only mattering for certain industries, a weighted sum can't express it, however many signals you add. Then you'd need interaction features you build by hand, or a model with layers that can learn combinations.