Prompt engineering interview questions and answers
Questions on writing and testing prompts: system prompts, examples, structured output, chaining, evals, hallucination and prompt injection.
1. Your prompt worked well on the first review you tried. Why isn't that enough, and what would you do before using it in a real workflow?
One good reply could be luck, and real reviews vary: short, sarcastic, off-topic. I'd collect a handful of test inputs including the awkward ones, run the prompt on all of them, and check the outputs with code where I can. Then I'd rerun the same set every time I change the prompt.
2. Explain the difference between a system prompt and a user message, with one example of what belongs in each for an email-writing assistant.
The system prompt is the standing brief I write once and send with every request, such as "Write in our brand's plain, friendly voice for past customers; never use ALL CAPS; keep emails under 150 words." The user message is the specific job, which changes each time, such as "Write an email announcing 30% off rain jackets through Sunday."
3. You're picking five examples for a review classifier. What makes a good set, and what could go wrong with a careless one?
A good set covers every label at least once, varies length and wording, and includes the borderline cases where I'd expect confusion, all formatted like the real input. A careless set, say four positive reviews that all start with "Love", teaches the model to lean positive and copy that opening, and a mislabeled example teaches the wrong rule.
4. In your own words: why does splitting a task into a chain of calls often give more reliable results than one big prompt?
Each call has a single focused job, so it's more likely to be done well. Between calls, my code can check the output and retry or fix just the failing step instead of hoping one big prompt got everything right. And when something goes wrong, I can see which step caused it.
5. Structured output guarantees the reply matches your schema. Why should your code still validate the data before acting on it?
The schema only promises the right fields and types. It can't say a discount must be 5 to 50 percent, that a not-a-fit lead shouldn't get a demo, or that a company name isn't blank, and the model can fill valid fields with wrong values. Replies can also be cut off or refused. So I check stop_reason, parse, then run my own rule checks before anything acts on the data.
6. Why shouldn't you keep editing a prompt until every example in your eval passes?
Because the prompt ends up tailored to those exact examples, so a perfect score no longer tells you how it'll do on new inputs. Keep some examples aside as a held-out set you only check at the end, like a holdout group in a campaign test.
7. Explain the lethal trifecta and one design change that breaks it for an AI assistant that reads customer emails and your CRM.
An agent is dangerous when it can read private data, sees untrusted content and can send data out, because instructions hidden in the untrusted content can make it ship the private data somewhere. For an email assistant with CRM access, I'd remove the outbound leg: it writes drafts and a person sends them, and it can't make web requests.
8. When would you raise effort, and when would you lower it? Give one example of each.
I'd raise effort for hard, multi-step work, like deciding how to split a budget across campaigns, because more thinking improves the answer. I'd lower it for simple, high-volume work like tagging reviews, where extra thinking adds time and output-token cost without improving results.
9. Explain how a chat app makes a model seem to remember earlier messages, and why very long conversations need special handling.
The model keeps nothing between requests. The app saves every message in a list and sends the whole list each time, so the model rereads the conversation as a transcript. That means every turn resends more tokens, which costs more and takes longer, and a long enough chat could exceed the context window. So apps trim to recent messages, summarize older turns, or pin key facts in the system prompt.
10. Describe the "quotes first" technique and why it reduces hallucination.
You ask the model to first copy, word for word, the sentences from the documents that answer the question, then to answer using only those quotes, or say it doesn't know if there are none. It keeps the answer anchored to the actual text, and because quotes must match the source exactly, your code can verify them.
11. Explain overfitting to a colleague using a prompt-engineering example.
It's when something fits the examples you tested on rather than the general task. With prompts: you keep adding special-case instructions until all 30 eval examples pass, and the prompt now handles those 30 well but behaves oddly on the 31st. A held-out set you only check at the end, and more varied examples, show whether the improvements are real.
12. Before using an LLM judge to approve 500 product descriptions, how would you check it's trustworthy?
I'd label 40 descriptions myself against the rubric, run the judge on the same 40 and measure how often we agree. If agreement is low I'd read the disagreements, make the vague criteria more specific or give the judge the guide it needs, and measure again. Once it's around 0.9, I'd still spot-check a random sample of its passes.