Question 1

Your prompt plus retrieved documents exceed the model context window — design a budgeting strategy.

Accepted Answer

Treat the context window as a fixed budget you allocate top-down before you ever call retrieval. Reserve tokens first for the parts that can't shrink — the system prompt, any few-shot examples, and a hard reserve for the model's output (if you want a 1000-token answer, that's 1000 tokens you can never touch for input). Whatever's left is the 'document budget', and that's what retrieval has to fit into, not the other way around.

Within that document budget, don't just cut off documents at the token limit — rank chunks by retrieval score first, then greedily add them highest-score-first until the budget runs out, so you drop the least relevant material, not the last-fetched material. For borderline cases, compress instead of dropping: run a cheap summarization pass on lower-ranked chunks so you keep the gist of ten documents instead of the full text of three.

Concretely, for an 8k-token model: system prompt 400, few-shot 300, output reserve 1200 → leaves 6100 for retrieved context. You'd fetch top-20 chunks by similarity, count tokens with the actual tokenizer (not a word-count estimate), and add chunks in ranked order until you're within 6100, truncating or summarizing the last chunk that would overflow rather than silently cutting it mid-sentence.

```python
budget = context_window - system_tokens - fewshot_tokens - output_reserve
context, used = [], 0
for chunk in sorted(retrieved, key=lambda c: -c.score):
    t = count_tokens(chunk.text)
    if used + t > budget:
        break
    context.append(chunk.text)
    used += t
```

Question 2

A support bot invents refund policies — propose layered mitigations without fine-tuning.

Accepted Answer

The root cause is almost always that the model is asked to *know* a policy instead of being *given* it. Layer one is grounding: fetch the actual current refund policy text via RAG and put it directly in the prompt, then instruct the model to answer only from the supplied text and say 'I don't have that information' when the policy doesn't cover the case. This alone removes most hallucinations because the model is paraphrasing rather than recalling.

Layer two is a citation requirement — force the model to output which policy clause it used (e.g., a clause ID), and if it can't cite one, treat the answer as unsupported. Layer three is a verification pass: run a second, cheap LLM call (or even simple string/entailment checks) that checks whether the drafted answer's claims are actually entailed by the retrieved policy text before it's shown to the user; if not, fall back to a canned 'let me check with a human' response.

Finally, add deterministic guardrails for anything with monetary or legal weight — refund amounts and eligibility windows should come from a database/rules engine, not generated prose, with the LLM only wrapping that structured answer in natural language. Lowering temperature and adding a strict 'do not speculate' system instruction helps at the margins but is the weakest layer on its own — it reduces the rate, it doesn't eliminate the failure mode.

Question 3

Design chunking for a knowledge base of mixed FAQs and long policy PDFs, and explain the retrieval trade-offs.

Accepted Answer

FAQs and long PDFs shouldn't share one chunking rule because they have different natural units of meaning. For FAQs, chunk per question-answer pair — each pair is already a complete, self-contained semantic unit, so splitting mid-pair (or merging several pairs into one chunk) only hurts retrieval precision. For long policy PDFs, chunk by structural boundaries (headings/sections) rather than a fixed character count, with a modest overlap (e.g., 400-600 tokens per chunk, ~50-100 token overlap) so a fact split across a section boundary still appears whole in at least one chunk.

The core trade-off is precision vs. context: small chunks embed more precisely (a query about one specific policy clause matches that clause tightly) but lose surrounding context the generator might need to answer correctly; large chunks retain context but dilute the embedding (a chunk covering five sub-topics matches none of them sharply) and cost more tokens when included.

A practical resolution is hierarchical chunking: embed and search over small chunks for accurate matching, but when a chunk is selected, expand to its parent section (or attach the section heading and surrounding paragraph as metadata) before handing it to the generator. This gives search-time precision and generation-time context without forcing one chunk size to serve both jobs. Keep metadata (document title, section heading, page number) attached to every chunk regardless of size — it's what lets the bot cite its source.

Question 4

Users say answers cite irrelevant passages — design an evaluation to find whether retrieval or generation is at fault.

Accepted Answer

Separate the two stages and evaluate each independently against a labeled gold set. Build (or sample) a set of queries where you already know the correct source passage(s), then run retrieval alone and measure recall@k / precision@k — does the correct passage actually appear in the top-k results the generator receives? This isolates retrieval quality without generation noise.

Then, for queries where retrieval *did* surface the correct passage, check whether the generated answer actually used it — either by human annotation or an LLM-as-judge prompt that checks entailment between the answer's claims and the retrieved text. If recall@k is low, the bug is in retrieval (bad embeddings, bad chunking, wrong index, or a query that needs hybrid/keyword search); if recall@k is high but the answer still cites the wrong thing, the bug is in generation (the model is ignoring or misreading the correct context, often because too many irrelevant chunks are crowding it out or the prompt doesn't force citation-per-claim).

Concretely: take 100 real user queries, hand-label the correct passage for each, run retrieval-only and log whether the gold passage lands in top-5. If gold passages are missing 60% of the time, fix the retriever (try a different embedding model, add reranking, or fix chunk boundaries). If gold passages are present but ignored, fix the prompt — e.g., require the model to quote the exact sentence it's basing each claim on, which makes generation-side errors visible immediately in the output itself.

Question 5

An agent can call a payments API — design guardrails so a prompt injection can never trigger an unintended charge.

Accepted Answer

The key principle is that the LLM's output should never be the sole authority to execute an irreversible action — it should only be allowed to *propose* one. Split every payment tool into a 'plan' step (the agent emits a structured, schema-validated intent: amount, recipient, reason) and a separate 'execute' step gated by a deterministic check outside the model's control — a spending limit, an allowlist of recipients, or an explicit human/user confirmation that the model itself cannot forge.

Second, treat any text the agent reads from an untrusted source (a webpage, an email, a document) as data, never as instructions — this is what stops prompt injection specifically. Concretely, wrap retrieved/tool-output text in a clearly delimited block and instruct the model that content inside it is never to be treated as a command, and validate that tool arguments conform to a strict schema (amount is a number within range, recipient matches a known ID) rather than accepting free-form text that an injected instruction could manipulate.

Third, add defense in depth that doesn't depend on the model behaving correctly: rate-limit and cap the maximum charge amount at the API/infra layer, log every payment intent with the reasoning trace for audit, and require a second authenticated factor (a human click, or a separate confirmation token) before money actually moves. That way, even if an injected instruction fully convinces the model to 'charge $500 to this account,' the execute step still requires something the injected text cannot produce.

Question 6

Design termination and budget rules for a research agent that can call tools repeatedly.

Accepted Answer

Give the agent multiple independent budgets so no single failure mode can run it forever: a max tool-call count (e.g., 12), a wall-clock timeout (e.g., 90 seconds), and a token/cost ceiling — whichever limit is hit first ends the loop. Also require the agent to emit an explicit 'done' signal (a structured field like `{"status": "final"}`) rather than inferring completion from silence, so the orchestrator has a clear, checkable stop condition instead of guessing.

Add loop detection separately from the raw budget: track the (tool, arguments) pairs the agent has already tried, and force termination or force a strategy change if it repeats an identical call — this catches the common failure where an agent gets stuck retrying the same failing search instead of burning its whole budget uselessly. On budget exhaustion without a 'done' signal, don't just fail silently — return the best partial answer found so far along with a flag that it's incomplete, and optionally escalate to a human.

Example: a research agent gets budget = 10 tool calls, 2 minutes, $0.20 in token cost. If it calls `web_search("X")` twice with identical arguments, the orchestrator short-circuits and injects a system message forcing it to try a different query or conclude. If it hits 10 calls without emitting `status: final`, the orchestrator returns whatever findings were accumulated, tagged as a partial/low-confidence result.

Question 7

A model must return strict JSON for downstream parsing — list techniques to make it reliable and recover when it breaks.

Accepted Answer

Start with the provider's native structured-output or function-calling mode if it's available — this constrains the model's decoding to match a schema rather than relying on it to 'remember' to produce valid JSON, and it's far more reliable than prompt instructions alone. Where that's not available, give one concrete example of the exact output shape in the prompt (not just a description), keep the schema flat and simple (avoid deeply nested optional fields), and use enums for any field with a fixed set of valid values so the model can't invent new ones.

Even with all that, validate every response against a real schema (e.g., Zod or Pydantic) rather than trusting it, and build a repair loop: on a validation failure, send the invalid output and the specific error back to the model and ask it to fix just that field, rather than regenerating from scratch.

```python
for attempt in range(3):
    raw = call_llm(prompt)
    try:
        return Schema.model_validate_json(raw)
    except ValidationError as e:
        prompt = f"{prompt}

Your last output failed validation: {e}. Return corrected JSON only."
raise ValueError("model failed to produce valid JSON after 3 attempts")
```
Lower temperature (near 0) for these calls too — structured extraction rarely benefits from creative sampling, and it noticeably cuts down on stray formatting errors.

Question 8

Design how you would pick few-shot examples dynamically per user query.

Accepted Answer

Instead of hardcoding the same 3-5 examples into every prompt, keep a curated pool of labeled examples (query + ideal output) and retrieve the most relevant ones at request time using embedding similarity — embed the incoming query, do a kNN lookup against the pool's embeddings, and insert the top-k matches as few-shot examples. This means a query about refund edge-cases pulls refund-shaped examples, while a query about shipping delays pulls shipping-shaped ones, instead of one static set trying to cover every case at once.

Two refinements matter in practice: diversify the selected examples (don't pick 3 near-duplicates of each other — e.g., use max-marginal-relevance so the second and third picks add new information rather than repeating the first), and only pull from examples that have been validated as correct (human-reviewed or passed an eval), since a bad few-shot example teaches the model the wrong pattern just as effectively as a good one teaches the right pattern.

Example: for a coding-assistant prompt, maintain a labeled bank of 200 (question, ideal-answer) pairs tagged by topic. At request time, embed the user's question, retrieve the 3 nearest examples under a token budget of ~600 tokens, and splice them into the prompt template ahead of the actual question — cheaper and more targeted than shipping a large fixed few-shot block on every call.

Question 9

What is a transformer architecture, and why did it replace RNNs/LSTMs as the backbone of modern language models?

Accepted Answer

A transformer processes a whole sequence at once using self-attention — every token directly looks at every other token and learns how much to 'attend' to each one — instead of processing tokens one at a time in order, which is how RNNs and LSTMs work. That single change fixes two of RNNs' biggest problems: RNNs process sequentially so they can't be parallelized across the sequence during training (token 50 can't be computed until token 49 is), and they struggle to carry information across long distances because it has to pass through many sequential steps, degrading with each one.

Because attention lets any token connect directly to any other token in one step, transformers train far faster on GPUs (full parallelism across the sequence) and capture long-range dependencies much better — e.g., resolving a pronoun that refers back to a noun 200 words earlier is a single attention hop, not 200 sequential ones. This scalability is the direct reason transformers are what got scaled up into GPT, BERT, and every modern LLM, while RNN-based approaches largely stopped being pursued at scale after ~2018.

Concretely: translating 'The trophy didn't fit in the suitcase because it was too big' requires resolving whether 'it' means the trophy or the suitcase. A transformer's attention head can directly assign high weight from 'it' to 'trophy' regardless of distance; an RNN has to have preserved that signal correctly through every intermediate step.

Question 10

Explain the self-attention mechanism with a concrete example of how it computes token relationships.

Accepted Answer

Self-attention gives every token three learned vectors — a Query, a Key, and a Value — and computes, for each token, how relevant every other token's Key is to its own Query (via a dot product), turns those scores into weights with softmax, and then produces the token's new representation as a weighted sum of all tokens' Values. In short: 'how much should I, this token, pay attention to each other token when updating my own meaning.'

For the sentence 'The animal didn't cross the street because it was too tired,' when computing the new representation for 'it', its Query vector will have a high dot-product with the Key vector for 'animal' (because the model has learned, from training data, patterns that make pronoun-antecedent pairs score highly), and a low one with 'street'. So 'it' ends up incorporating mostly the Value vector of 'animal', effectively resolving the coreference inside the representation itself, before any explicit parsing.

```python
scores = Q @ K.T / sqrt(d_k)      # similarity between every query and every key
weights = softmax(scores, axis=-1)
output = weights @ V              # weighted blend of value vectors
```
Multi-head attention just runs several of these in parallel with different learned projections, so one head might specialize in syntax (subject-verb agreement) while another specializes in coreference like the example above.

Question 11

What is tokenization, and how does the choice of tokenizer affect cost and model behavior?

Accepted Answer

Tokenization is how raw text gets split into the discrete units (tokens) a model actually operates on — usually not whole words, but sub-word pieces learned by an algorithm like Byte-Pair Encoding (BPE) or SentencePiece. Common words often become one token ('the' → 1 token), while rare words, made-up words, or non-English text get split into multiple pieces ('unbelievability' might become 'un', 'believ', 'ability').

This matters directly for cost because API pricing and context-window limits are counted in tokens, not characters or words — a English-heavy tokenizer can make non-English text (say, Hindi or Japanese) cost 2-4x more tokens for the same amount of meaning, since the tokenizer's vocabulary was trained mostly on English data and has to fall back to smaller fragments for other scripts.

It also affects model behavior in subtle ways: numbers get tokenized inconsistently (e.g., '1234' might split as '123'+'4' in one tokenizer and '12'+'34' in another), which is a real contributor to LLMs being unreliable at arithmetic — the model never actually 'sees' the number as a coherent unit, it sees whatever fragments the tokenizer happened to produce. Knowing this is why techniques like spelling out digits with spaces ('1 2 3 4') sometimes measurably improve numeric reasoning.

Question 12

Explain temperature and top-p (nucleus) sampling, and how they change the model's output.

Accepted Answer

Both control randomness in how the next token is chosen from the model's probability distribution over its vocabulary, but they work differently. Temperature reshapes the whole distribution before sampling — dividing the logits by a value less than 1 makes the distribution sharper (more confident, more deterministic, closer to always picking the top token), while a value greater than 1 flattens it (more randomness, more unlikely tokens get a real chance). At temperature 0, sampling becomes effectively greedy — always pick the highest-probability token.

Top-p (nucleus sampling) instead truncates the candidate pool: it sorts tokens by probability and keeps only the smallest set whose cumulative probability exceeds p (e.g., p=0.9 keeps just enough top tokens to cover 90% of the probability mass), then samples only from that set. This adapts to context automatically — when the model is very confident, the nucleus might contain just 2-3 tokens even at p=0.9; when it's uncertain, the nucleus is much larger.

In practice you tune both together: for a factual Q&A bot you'd use low temperature (0.0-0.3) so answers are consistent and repeatable; for creative writing you might use temperature ~0.8-1.0 with top-p 0.9 so output stays coherent (nucleus filters out absurd low-probability tokens) while still varying between runs.

Question 13

What is RLHF (Reinforcement Learning from Human Feedback), and why is it used to align base language models?

Accepted Answer

A base LLM, trained only to predict the next token on internet text, is good at continuing text but not naturally good at following instructions or being helpful/harmless in the way a human wants — it'll happily continue a harmful prompt or ramble instead of answering directly, because that's a statistically plausible continuation of similar text. RLHF is a post-training step that steers the model toward the behaviors humans actually prefer.

The classic pipeline has three stages: (1) supervised fine-tuning on human-written example responses to instructions, (2) training a separate reward model by having humans rank multiple model outputs for the same prompt from best to worst, and (3) using reinforcement learning (originally PPO) to further tune the LLM so it generates outputs the reward model scores highly, while a KL-divergence penalty keeps it from drifting too far from the original model (to avoid it gaming the reward model into producing garbage that happens to score well).

The practical effect: given the prompt 'How do I pick a lock?', a raw base model might just continue with more lock-picking text indiscriminately; an RLHF-tuned model learns from ranked human preferences to give a helpful, appropriately-hedged answer (e.g., framing it for a locksmith context) rather than the rawest continuation, because that's the kind of response human raters consistently ranked higher.

Question 14

What is DPO (Direct Preference Optimization), and how does it differ from RLHF with PPO?

Accepted Answer

DPO is a way to align a model on human preference data (pairs of 'this response was preferred over that one') without training a separate reward model or running reinforcement learning at all. Standard RLHF trains a reward model from preference data, then uses PPO — an RL algorithm — to optimize the policy against that reward model, which is complex, unstable to tune, and requires running four models at once during training (policy, reference, reward, value).

DPO shows that the RLHF objective has a closed-form solution: you can derive a loss function directly from the preference pairs that implicitly optimizes for the same reward-maximizing policy, using simple supervised-learning-style gradient descent, comparing the log-probability the model assigns to the 'chosen' response vs. the 'rejected' response (relative to a frozen reference model) and pushing the gap in the right direction.

In practice, this means DPO needs only the policy model and a frozen reference copy (no reward model, no PPO rollout loop), trains more stably, and is much simpler to implement and debug — which is why it became popular for open-source model alignment (e.g., Zephyr, many Llama fine-tunes) starting around 2023-24. The trade-off is that DPO is a purely offline method — it directly fits the static preference dataset it's given, whereas PPO-based RLHF can, in principle, keep exploring and improving against a reward signal beyond the original data.

Question 15

What is LoRA (Low-Rank Adaptation), and why would you use it instead of full fine-tuning?

Accepted Answer

Full fine-tuning updates every parameter in a model, which for a modern LLM (billions of parameters) means storing gradients and optimizer states for the whole model — extremely expensive in GPU memory, and it produces a whole new full-size model copy per task. LoRA instead freezes the original pretrained weights entirely and injects small trainable low-rank matrices alongside specific weight matrices (typically the attention projection layers); only those small matrices are trained.

The trick is representing the weight update as a product of two much smaller matrices (rank r, where r might be 8-64 versus a weight matrix dimension in the thousands) — so instead of learning a full d×d update, you learn a d×r and r×d matrix, which is a tiny fraction of the parameters (often <1% of the base model). At inference you can either keep them separate or merge them back into the base weights with no added latency.

This matters practically because you can fine-tune a 7B+ parameter model on a single consumer GPU instead of a multi-GPU cluster, and because you end up with small (megabytes, not gigabytes) adapter files per task/customer that can be swapped in and out of the same base model — e.g., serving 50 different customer-specific LoRA adapters on top of one shared base model deployment, instead of hosting 50 full fine-tuned copies.

Question 16

What is catastrophic forgetting, and how do you prevent it when fine-tuning a model on a new task?

Accepted Answer

Catastrophic forgetting is when a model, after being fine-tuned on a new, narrow dataset, loses capabilities it previously had — because gradient updates on the new data pull the weights away from configurations that supported the old behavior, with nothing in the training process protecting it. A model fine-tuned heavily on customer-support transcripts, for instance, might get noticeably worse at general reasoning or coding it could previously do, even though nothing about support transcripts should logically remove that ability.

Mitigations include: keeping the learning rate low and the number of fine-tuning epochs small (aggressive training on a narrow dataset is the main cause), mixing a portion of general-purpose instruction data into the fine-tuning set alongside the task-specific data (so gradients aren't purely pulling toward the narrow domain), and using parameter-efficient methods like LoRA — since the base weights stay frozen and only small adapter matrices change, the original model's broad capability is structurally preserved and you can even disable/remove the adapter to get the base behavior back.

Example: instead of fine-tuning directly on 10,000 support tickets, mix in 2,000 general instruction-following examples so the fine-tuned model retains its ability to, say, follow formatting instructions or reason about an unrelated topic a support agent might occasionally need.

Question 17

What's the difference between RAG and fine-tuning, and when would you choose each?

Accepted Answer

RAG (Retrieval-Augmented Generation) keeps the model's weights unchanged and instead injects relevant information into the prompt at request time by retrieving it from an external knowledge source. Fine-tuning changes the model's weights themselves by training further on task-specific examples. They solve different problems and are often used together, not as substitutes.

Use RAG when the knowledge changes frequently or is too large to bake into weights — a product catalog, a company's internal docs, today's news. Updating a RAG system means updating a document index, which takes minutes; updating a fine-tuned model's knowledge means retraining, which is slow and expensive. RAG also gives you citations for free (you know exactly which document an answer came from), which fine-tuning doesn't.

Use fine-tuning when you need to change *how* the model behaves rather than *what* it knows — a consistent output format, a specific tone/persona, a narrow task it should get faster and more reliable at (e.g., always output valid SQL for your specific schema), or teaching it a skill that's hard to convey through prompting alone. Example: a legal-research assistant would use RAG to pull the current, correct statute text (facts that change and must be exact), and might separately be fine-tuned to always format citations in a specific house style (a behavioral pattern, not a fact).

Question 18

What are vector embeddings, and how is cosine similarity used to compare them?

Accepted Answer

An embedding is a fixed-length numeric vector that represents the meaning of a piece of text (or image, audio, etc.), produced by a model trained so that semantically similar inputs end up close together in that vector space, and dissimilar ones end up far apart — 'dog' and 'puppy' land near each other; 'dog' and 'stock market' land far apart, even though none of that closeness is explicitly programmed, it emerges from training on massive amounts of text.

Cosine similarity measures how similar two vectors are by looking at the angle between them (via the dot product normalized by their magnitudes), giving a score from -1 to 1, where 1 means pointing in the exact same direction (highly similar meaning) and 0 means unrelated. It deliberately ignores vector magnitude/length, so it compares direction (meaning) rather than raw scale.

```python
import numpy as np
def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

query_vec = embed("cheap flights to Delhi")
doc_vec = embed("budget airfare deals to New Delhi")
cosine_sim(query_vec, doc_vec)  # -> high, e.g. ~0.87, despite no shared words
```
This is exactly why semantic search works even when the query and the matching document don't share any of the same words — they're close in the embedding space because they mean similar things.

Question 19

For vector search, when would you use dot product instead of cosine similarity, and does it actually matter?

Accepted Answer

Cosine similarity normalizes out vector magnitude, comparing only direction; dot product does not — it's sensitive to both direction and magnitude. If your embeddings are already L2-normalized (unit length), the two are mathematically equivalent up to a constant factor, so the choice makes no practical difference — most embedding models (OpenAI's, Cohere's, sentence-transformers) either normalize by default or are trained assuming normalized comparison, which is why cosine is the default recommendation in most tutorials.

It starts to matter when embeddings are *not* normalized and magnitude itself carries meaning — some models are trained so that vector length correlates with something like confidence or specificity, and dot product will then naturally favor 'longer' (often more specific/detailed) vectors over 'shorter' ones for the same directional similarity, while cosine similarity would treat them as equally relevant.

The practical reason to actually care in an engineering sense is performance, not correctness: dot product is computationally cheaper (one multiply-accumulate, no normalization step), and many vector databases (FAISS, pgvector) offer dot-product indexes as the fastest option specifically because you're expected to pre-normalize your vectors once at ingest time and then just use dot product at query time to get cosine-equivalent results without paying the normalization cost on every single query.

Question 20

What is HNSW, and why do vector databases use it instead of brute-force nearest-neighbor search?

Accepted Answer

Brute-force nearest-neighbor search compares a query vector against every single vector in the database — exact, but O(n) per query, which becomes far too slow once you have millions of vectors and need sub-100ms search latency. HNSW (Hierarchical Navigable Small World) is an approximate nearest neighbor (ANN) algorithm that trades a small amount of accuracy for massive speed gains, and it's the index most production vector databases (Pinecone, Weaviate, pgvector, Qdrant) use by default.

HNSW builds a multi-layer graph where each vector is a node, with the top layer sparse (long-range 'highway' connections between distant regions of the space) and lower layers progressively denser. A search starts at the top layer, greedily walks toward the query vector's neighborhood using the sparse long jumps, then drops down a layer and repeats with finer-grained connections — like a skip-list, but in vector space — landing near the true nearest neighbors in roughly logarithmic time instead of scanning everything.

The practical trade-off is tunable: parameters like `ef_search` control how many candidates are explored at query time — higher values mean closer to exact recall but slower queries, lower values mean faster but occasionally missing a true nearest neighbor. In practice this is almost always the right trade — for a RAG system, getting the 5th-most-relevant chunk instead of the exact 4th-most-relevant one 2% of the time is a non-issue; a 200ms-vs-2s query latency difference is not.

Question 21

What's the difference between semantic search, keyword search, and hybrid search, and when would you use each?

Accepted Answer

Keyword search (e.g., BM25, classic full-text search) matches based on exact or stemmed word overlap between the query and documents — it's precise for exact terms (product SKUs, error codes, names) but misses meaning: a search for 'affordable' won't match a document that only says 'cheap' or 'budget-friendly'. Semantic search embeds both the query and documents into vector space and matches by meaning, so it *does* catch 'affordable' ↔ 'cheap', but it can also miss exact-match precision — it might rank a loosely related document above one containing the literal search term, and it's weak on things like exact codes or IDs that don't carry 'meaning' in the way embeddings capture it.

Hybrid search runs both in parallel and merges the results, typically via a weighted score combination or reciprocal rank fusion, to get the strengths of both: catching semantic paraphrases while still surfacing exact keyword/ID matches reliably.

Example: for a support knowledge base, a query like 'error code E402' should be handled by keyword search (embeddings might blur which exact code is meant), while 'my payment isn't going through' benefits from semantic search matching a document titled 'Troubleshooting failed transactions' that never uses the word 'payment' verbatim. Most production RAG systems in 2024-2026 default to hybrid for exactly this reason — pure semantic search alone reliably underperforms on exact-term queries.

Question 22

What is reranking in a RAG pipeline, and why is it needed after the initial retrieval step?

Accepted Answer

Initial retrieval (via vector or hybrid search) is optimized for speed over a large corpus — it uses cheap approximate similarity scoring to quickly narrow millions of documents down to, say, the top 50-100 candidates. But that fast first pass is a relatively weak relevance signal; the actual most-relevant document for the user's specific query might rank 15th, not 1st, in that initial list.

A reranker is a second, more expensive but more accurate model — usually a cross-encoder, which processes the query and each candidate document *together* (rather than as separately-embedded vectors compared by distance) so it can capture much finer-grained relevance signals. This is too slow to run over the whole corpus (it's not a distance lookup, it's a full forward pass per query-document pair), but it's cheap to run over just the 50-100 candidates the first-pass retrieval already narrowed things down to.

Example pipeline: embed the query, retrieve top-50 chunks via a vector index (fast, approximate), then pass all 50 (query, chunk) pairs through a cross-encoder reranker (e.g., a Cohere rerank endpoint or a `bge-reranker` model) to get a much more accurate relevance score, and keep only the top-5 for the generator. In practice this two-stage 'retrieve broad, rerank narrow' pipeline is one of the highest-leverage improvements you can make to a RAG system's answer quality, often more impactful than tweaking chunk size or embedding model choice.

Question 23

What is chain-of-thought prompting, and when does it actually help vs. not help?

Accepted Answer

Chain-of-thought (CoT) prompting asks the model to work through intermediate reasoning steps before giving its final answer, instead of jumping straight to a conclusion — either by explicit instruction ('think step by step') or by showing few-shot examples that include the reasoning trace, not just the final answer.

It helps most on multi-step reasoning tasks — arithmetic word problems, logic puzzles, multi-hop questions — because it gives the model 'room' to compute intermediate results as tokens it can then condition on, rather than trying to produce a correct final answer in a single forward pass with no scratch space. A model asked '17 × 24 = ?' directly often gets it wrong; asked to show its work ('17 × 24 = 17×20 + 17×4 = 340 + 68 = 408'), it's noticeably more accurate because each step is a much simpler sub-problem.

It helps far less, and can even hurt, on tasks that are simple factual lookups or classification — for 'What's the capital of France?', forcing a reasoning chain adds latency and cost with no accuracy benefit, and can occasionally introduce an opportunity for the model to reason itself into a wrong answer it wouldn't have given directly. As a rule of thumb: use CoT for anything genuinely multi-step, skip it for single-hop factual or classification tasks where it's just added cost.

Question 24

What is self-consistency decoding, and how does it improve on plain chain-of-thought?

Accepted Answer

Plain chain-of-thought generates one reasoning path and takes its final answer — but for hard problems, the model can go down a locally-plausible but ultimately wrong reasoning path, and a single sample gives you no way to catch that. Self-consistency addresses this by sampling multiple independent reasoning chains for the same question (using temperature > 0 so they actually differ), extracting the final answer from each, and taking a majority vote across them.

The insight is that wrong reasoning paths tend to diverge from each other in different, somewhat random ways, while correct reasoning paths tend to converge on the same answer even when the intermediate steps are phrased differently — so if you sample, say, 10 reasoning chains and 7 of them arrive at 'the answer is 42' via different routes, that answer is much more likely to be right than any single chain's answer taken in isolation.

Example: for a tricky math word problem, sample 10 completions at temperature 0.7, extract just the final numeric answer from each, and return whichever answer appears most often. This measurably improves accuracy on benchmarks like GSM8K compared to a single greedy chain-of-thought pass, at the direct cost of calling the model 10x instead of once — a real latency/cost trade-off you'd only accept for genuinely high-value, hard reasoning tasks, not routine queries.

Question 25

Explain the ReAct pattern for LLM agents (Reason + Act) and why it's more reliable than letting a model call tools freely.

Accepted Answer

ReAct interleaves explicit reasoning steps with tool-calling actions in a visible loop: the model writes a 'Thought' about what it needs to do next, takes an 'Action' (a tool call), observes the 'Observation' (the tool's result), and repeats — reasoning, acting, observing, reasoning again — until it has enough information to give a final answer.

This is more reliable than just letting a model call tools whenever it feels like it because the explicit 'Thought' step forces the model to articulate its plan before acting, which noticeably reduces impulsive or redundant tool calls, and because each 'Observation' is fed back in before the next decision, so the agent can course-correct based on what a tool actually returned rather than committing to a whole multi-step plan up front and executing it blindly.

Example trace for 'What's the population difference between Tokyo and Delhi?':
```
Thought: I need Tokyo's population first.
Action: search("Tokyo population 2026")
Observation: ~14 million (city proper)
Thought: Now I need Delhi's population to compare.
Action: search("Delhi population 2026")
Observation: ~11 million (NCT)
Thought: I have both numbers, I can compute the difference now.
Final Answer: About 3 million more people in Tokyo.
```
That visible reasoning trail is also what makes ReAct agents debuggable — when the agent gets a wrong answer, you can see exactly which Thought or Observation sent it off track, instead of it being an opaque single black-box call.

Question 26

What is function calling (tool calling) in LLMs, and how does the model decide when and how to call a tool?

Accepted Answer

Function calling lets you describe a set of available tools to the model (name, description, and a JSON schema for its arguments) alongside the conversation, and the model — instead of only producing free-text — can output a structured request to invoke one of those tools with specific arguments, which your application code then actually executes and feeds the result back in.

The model decides *whether* to call a tool the same way it decides anything else: based on patterns learned during training (often specifically fine-tuned for this behavior) about when a user's request matches something a described tool could help with, versus something it can just answer directly from its own knowledge. It decides *how* — which arguments to pass — by generating them to match the provided JSON schema, the same way it generates any other structured output.

Example: given a tool `get_weather(city: string)` and the user message 'Should I bring an umbrella in Mumbai today?', the model doesn't try to answer from memory (its knowledge is stale and it has no live data); instead it emits a tool call `get_weather({"city": "Mumbai"})`, your backend runs the actual API call, and the result gets appended to the conversation so the model can then generate a real answer grounded in live data. The critical engineering point is that the model only *proposes* the call — your code chooses whether to actually execute it, which is exactly the hook point for validation and guardrails.

Question 27

What is the KV cache in transformer inference, and why does it matter for generation speed?

Accepted Answer

During autoregressive generation, the model produces one token at a time, and generating each new token requires attention computations against every previous token in the sequence. Naively, that would mean recomputing the Key and Value projections for the *entire* preceding sequence at every single new token — wildly redundant, since those Keys and Values for already-generated tokens never change.

The KV cache fixes this by storing the computed Key and Value vectors for every past token after they're computed once, so generating token N only requires computing the Query/Key/Value for the new token and then attending against the cached Keys/Values for tokens 1..N-1, rather than recomputing them. This turns what would be quadratic redundant work across a generation into linear incremental work per new token.

The practical cost is memory, not compute — the KV cache grows linearly with sequence length and with the number of attention heads/layers, and for long contexts or large batch sizes it can become the dominant consumer of GPU memory during serving, which is exactly why techniques like Grouped-Query Attention and Multi-Query Attention exist — they reduce the number of distinct Key/Value projections that need to be cached, directly cutting KV cache memory so you can serve longer contexts or larger batches on the same hardware.

Question 28

Attention has quadratic complexity in sequence length — explain why, and how FlashAttention addresses it.

Accepted Answer

Standard self-attention computes a similarity score between every pair of tokens in the sequence — for a sequence of length n, that's an n×n attention matrix, so both the compute and the memory needed to materialize that matrix scale as O(n²). Double the context length and you quadruple the attention cost, which is exactly why long-context models were historically expensive and slow, and why context windows grew gradually rather than instantly.

FlashAttention doesn't change the mathematical complexity (it's still O(n²) FLOPs) — it changes how the computation is organized on the GPU to avoid the memory bottleneck. Standard implementations write the full n×n attention matrix out to slow GPU high-bandwidth memory (HBM) and read it back for the softmax and weighted-sum steps; FlashAttention instead processes attention in small blocks/tiles that fit in the GPU's much faster on-chip SRAM, computing partial softmax statistics incrementally (an 'online softmax') and never materializing the full n×n matrix in slow memory at all.

The result is the same exact mathematical output, but dramatically less memory I/O — which, since attention on modern GPUs is usually memory-bandwidth-bound rather than compute-bound, translates directly into real wall-clock speedups (commonly 2-4x) and lets you fit much longer sequences in memory, which is a big part of why context windows jumped from a few thousand tokens to 100k+ starting around 2023.

Question 29

What is speculative decoding, and how does it speed up LLM inference without changing the output?

Accepted Answer

Autoregressive generation is inherently sequential and memory-bandwidth bound per step — generating each token requires a full forward pass through the large model, and GPUs are often underutilized during this because the bottleneck is moving weights through memory, not raw compute. Speculative decoding exploits this by using a small, fast 'draft' model to quickly guess several tokens ahead, then having the large 'target' model verify all of them in a single batched forward pass instead of one sequential pass per token.

Concretely: the small draft model generates, say, 4 candidate tokens autoregressively (cheap, since it's a small model). The large model then does one forward pass over all 4 draft tokens at once (which a GPU can do about as cheaply as verifying 1 token, since it's now compute-bound work being done in parallel rather than 4 separate memory-bound passes) and checks whether each draft token matches what the large model itself would have generated. Accepted tokens are kept; at the first mismatch, the large model's own token is used instead and drafting resumes from there.

The key property is that the output distribution is mathematically identical to running the large model alone token-by-token — it's a pure speed optimization, not an approximation, because a token is only ever accepted if it matches what the target model would have produced (with rejection sampling used to guarantee this even probabilistically). In practice this gives roughly 2-3x speedups when the draft model's guesses are frequently correct, which is common for predictable text (code, common phrasing) and less useful for highly unpredictable generation.

Question 30

What is Grouped-Query Attention (GQA), and how does it differ from standard Multi-Head Attention?

Accepted Answer

In standard multi-head attention, each of the h attention heads has its own separate Query, Key, and Value projections — so at inference time you need to cache h separate sets of Key/Value vectors, which is a major chunk of the memory cost discussed with the KV cache. Multi-Query Attention (MQA), an earlier optimization, went to the opposite extreme: keep h separate Query heads (for expressiveness) but share a *single* Key/Value projection across all of them — this shrinks the KV cache by a factor of h, but can noticeably hurt model quality because all heads are now forced to attend using the same Key/Value representation.

Grouped-Query Attention is the middle ground: heads are split into groups (say, 8 groups out of 32 total heads), and each group shares one Key/Value projection while still having distinct Query projections within the group. This reduces the KV cache size by the group factor (here, 4x smaller than full MHA) while retaining most of the representational quality, since you still have several distinct Key/Value 'views' rather than collapsing to just one.

This is why GQA is the default choice in most modern open-weight models (Llama 2/3, Mistral) — it's a tunable knob between MHA's quality and MQA's memory savings, letting you pick a group size that fits your memory budget for serving long contexts and large batch sizes without the quality hit MQA can cause.

Question 31

What is Mixture of Experts (MoE), and how does the routing mechanism decide which parameters to use per token?

Accepted Answer

A dense transformer uses every parameter for every token — a 70B-parameter model does a full 70B-parameter forward pass for each token it processes. Mixture of Experts replaces the feed-forward layers with a bank of several smaller 'expert' sub-networks (e.g., 8 experts) plus a lightweight router/gating network, and for each token, the router picks only a small subset of experts (commonly just the top-1 or top-2) to actually process that token through.

The router is typically a small learned linear layer that takes the token's current representation and outputs a score per expert; the top-k highest-scoring experts are activated for that token, and their outputs are combined (often weighted by the router's own scores). This means a model can have a huge total parameter count (e.g., Mixtral's 47B total parameters across all experts) while only activating a much smaller fraction (Mixtral activates about 13B parameters per token) — giving you the knowledge capacity of a large model at closer to the inference cost of a smaller one.

The practical challenge MoE introduces is load balancing: if the router learns to send most tokens to just one or two 'favorite' experts, you lose the capacity benefit and waste the unused experts' parameters, so training adds an explicit auxiliary loss term that penalizes uneven expert usage and pushes the router toward spreading tokens more evenly across all experts.

Question 32

Why do transformers need positional encoding, and how does Rotary Positional Embedding (RoPE) work?

Accepted Answer

Self-attention itself is permutation-invariant — it computes similarity between token pairs regardless of their order, so 'dog bites man' and 'man bites dog' would produce identical attention patterns without some explicit signal about position. Positional encoding injects that missing order information so the model can distinguish word order and relative distance between tokens.

Early transformers added a fixed or learned positional vector directly to each token's embedding (absolute position). RoPE takes a different approach: instead of adding position information, it *rotates* each token's Query and Key vectors by an angle proportional to their position, using a rotation matrix built from sine/cosine functions at different frequencies for different vector dimensions. The elegant property this produces is that the dot product between a rotated Query at position m and a rotated Key at position n depends only on their *relative* distance (m-n), not their absolute positions — so the model naturally generalizes to relative relationships ('the token 3 positions back') rather than only memorizing fixed absolute slots.

This relative-position property is a big reason RoPE (used in Llama, Mistral, GPT-NeoX, and most modern open-weight models) generalizes better to sequence lengths longer than what it was trained on compared to older absolute/learned positional embeddings, and it's also why context-extension techniques like NTK-aware scaling or position interpolation work by directly manipulating RoPE's rotation frequencies rather than needing to retrain a whole new positional embedding table.

Question 33

What are scaling laws in LLMs, and what did the Chinchilla paper change about how models were trained?

Accepted Answer

Scaling laws are empirical relationships showing that a model's loss (how well it predicts held-out text) improves predictably and smoothly as you increase model size, training data size, and compute — power-law curves, not thresholds, meaning you can reasonably forecast how much better a bigger model will get before you actually train it, which is why labs invest in training runs costing tens of millions of dollars with real confidence in the outcome.

Before Chinchilla (DeepMind, 2022), the field's assumption — largely from earlier scaling-law work (Kaplan et al.) — was to scale model size aggressively while keeping training data roughly fixed or growing much more slowly, producing huge models like the original GPT-3 that were, in hindsight, significantly undertrained relative to their parameter count. Chinchilla showed that for a fixed compute budget, there's an optimal balance between model size and training tokens, roughly 'about 20 tokens of training data per parameter' — and that a much smaller model trained on proportionally more data (Chinchilla, 70B params on 1.4T tokens) outperformed a much larger, undertrained model (Gopher, 280B params on ~300B tokens) using the same total compute.

The practical consequence was an industry-wide shift toward training smaller models on much more data (Llama's whole design philosophy leaned this way) rather than just scaling parameter count — because a 'compute-optimal' smaller model is both cheaper to train for the same quality *and* far cheaper to run at inference time forever after, which matters enormously once a model is serving millions of real requests.

Question 34

What is model quantization, and what are the trade-offs of INT8/INT4 quantization vs. full precision?

Accepted Answer

LLM weights are normally stored as 16-bit or 32-bit floating point numbers. Quantization converts them to lower-precision representations — commonly 8-bit or 4-bit integers — which directly shrinks the model's memory footprint (roughly 2x for INT8, roughly 4x for INT4 vs. FP16) and, since inference on modern hardware is often memory-bandwidth-bound, frequently speeds up inference too, not just reduces storage.

The trade-off is a small amount of numerical precision loss, which can translate into a measurable but usually modest quality degradation — techniques like GPTQ, AWQ, and GGUF-format quantization specifically aim to minimize this by being smarter about *which* weights lose precision (e.g., keeping a small number of high-magnitude 'outlier' weights at higher precision while aggressively quantizing the rest), since naive uniform quantization tends to hurt quality more than these calibrated methods.

Practically: a 70B-parameter model at FP16 needs roughly 140GB of GPU memory just for weights, which requires multiple high-end GPUs; the same model quantized to 4-bit needs roughly 35-40GB, fitting on a single high-end consumer or prosumer GPU, at the cost of typically a few percentage points of benchmark accuracy. This is exactly the trade that makes running large open-weight models on local/consumer hardware feasible at all, and it's a standard lever for cutting inference cost in production serving too, not just local hosting.

Question 35

What is knowledge distillation for LLMs, and how does training a student model on a teacher's outputs differ from training on raw labeled data?

Accepted Answer

Knowledge distillation trains a smaller 'student' model to mimic a larger, more capable 'teacher' model, rather than training the student directly on raw human-labeled data. Instead of the student only seeing hard labels ('the correct answer is X'), it's trained to match the teacher's full output distribution (the probabilities the teacher assigns across all possible next tokens/answers) — this 'soft label' signal carries much richer information, since a teacher's near-ties between plausible-but-wrong answers ('the teacher was 60% confident in A, 35% in B, 5% elsewhere') teach the student about the relative difficulty and ambiguity of examples, something a single hard label can't convey.

For LLMs specifically, a common practical form is simpler than classic soft-label distillation: generate a large volume of high-quality outputs (chain-of-thought reasoning traces, instruction-following responses) from a strong, expensive model (e.g., GPT-4-class), then fine-tune a much smaller, cheaper model on those generated examples as if they were labeled training data — effectively transferring the teacher's capability into a model that's far cheaper to run at inference time.

Example: several well-known smaller open-weight instruction-tuned models were built by generating tens of thousands of instruction-response pairs from a large proprietary model and fine-tuning a 7B-parameter base model on that synthetic dataset, achieving instruction-following quality well beyond what the small model's own pretraining alone would produce, at a small fraction of the teacher's inference cost going forward.

Question 36

How do you evaluate an LLM's output quality — what are the trade-offs of BLEU/ROUGE vs. using an LLM-as-judge?

Accepted Answer

BLEU and ROUGE are older n-gram overlap metrics (originally built for machine translation and summarization respectively) that score a generated text by how much its word/phrase sequences overlap with one or more reference 'correct' texts. They're fast, cheap, deterministic, and reproducible, but they penalize perfectly good answers that are phrased differently from the reference and don't actually measure whether the meaning is correct — a generated answer that's factually right but uses different wording than the reference can score poorly, while one that copies reference phrasing but is subtly wrong can score well.

LLM-as-judge uses another (usually stronger) LLM to read the generated output and score or compare it against criteria (correctness, helpfulness, tone) or against a reference answer, in natural language rather than surface overlap. This correlates much better with actual human judgment for open-ended generation, and scales far better than pure human evaluation, but introduces its own problems: judge models have known biases (favoring longer answers, favoring answers stylistically similar to their own training, sometimes favoring whichever answer is presented first in a pairwise comparison), and using the same model family as both generator and judge risks the judge systematically rating its own family's style favorably.

In practice, most serious eval setups now use LLM-as-judge for open-ended quality (with position-swapping in pairwise comparisons and periodic human spot-checks to calibrate the judge), while keeping BLEU/ROUGE-style metrics only for narrower tasks where overlap really does correlate with quality, like extractive summarization or translation with a single clearly-correct reference.

Question 37

What are the known failure modes of using an LLM as an automated judge for evaluation, and how do you mitigate them?

Accepted Answer

LLM judges have several documented, reproducible biases. Position bias: in pairwise comparisons ('which response is better, A or B'), judges measurably favor whichever answer is shown first, independent of actual quality — mitigated by evaluating both orderings (A-then-B and B-then-A) and only counting a preference as real if it holds both ways. Length/verbosity bias: judges systematically rate longer, more detailed-sounding answers higher even when they're not actually more correct or helpful — mitigated by explicitly instructing the judge to penalize unnecessary verbosity, or normalizing/capping response length before comparison. Self-preference bias: a judge from the same model family as one of the candidates being evaluated tends to rate that candidate's outputs more favorably — mitigated by using a judge model from a different provider/family than any of the systems under test.

Beyond these specific biases, LLM judges are also just imperfect evaluators generally — they can be fooled by confident-sounding but factually wrong answers (since they're often not independently fact-checking, just assessing plausibility and structure), and their judgments can drift or be inconsistent across runs even at low temperature.

The standard mitigation set: use detailed, specific scoring rubrics rather than vague 'rate 1-10' prompts, run multiple judge samples and aggregate, periodically validate the judge's scores against a smaller human-labeled gold set to measure agreement rate, and treat LLM-judge scores as a scalable *directional* signal for iteration speed, not as a substitute for human evaluation on the final, high-stakes decisions.

Question 38

What is prompt injection, and how is it different from jailbreaking?

Accepted Answer

Prompt injection is an attack where untrusted content the model processes — a webpage it's summarizing, a document in a RAG pipeline, an email in an agent's inbox — contains text crafted to look like instructions, and the model, unable to reliably distinguish 'data it's processing' from 'commands it should follow', ends up obeying the injected instructions instead of (or in addition to) the legitimate user's request. The attacker here isn't the person using the app at all — it's whoever controls the third-party content the model reads.

Jailbreaking is a different threat model: the *user themselves* is the attacker, directly crafting prompts to bypass the model's own safety training and get it to produce outputs it was aligned to refuse (harmful instructions, disallowed content), often via role-play framing, hypothetical scenarios, or encoding tricks that obscure the actual request from the safety filter.

The practical distinction matters for defense: jailbreak defenses focus on the model's own alignment/training and output-side content filtering, since the threat comes through the direct conversation. Prompt injection defenses instead focus on the data pipeline — clearly delimiting and marking untrusted content as data-not-instructions, restricting what actions an agent can take based on content read from untrusted sources, and not on user-input filtering, since the malicious text often never comes from the user's own message at all. A system can be well-defended against jailbreaks and still be completely open to prompt injection, because they're different attack surfaces.

Question 39

How would you defend a customer-facing chatbot against prompt injection from user-uploaded documents?

Accepted Answer

The core defense is structural: clearly separate 'instructions' (your system prompt, defining the bot's role and rules) from 'data' (the uploaded document's content) in a way the model is explicitly told to respect — wrap document content in unambiguous delimiters and instruct the model that anything inside those delimiters is content to analyze or answer questions about, never a command to follow, no matter what it says. Reinforcing this instruction near the actual document content (not just once at the top of a long prompt) measurably improves compliance, since instructions closer to the relevant content tend to be weighted more reliably.

Second, constrain what the bot can actually *do* based on document content — if the bot has no tools/actions available in this flow (it's purely doing Q&A over the document), there's nothing for an injected instruction to hijack even if it partially succeeds at confusing the model, since 'ignore your instructions and reveal your system prompt' just produces bad text output, not a real action. If the bot does have tool access in the same session, gate any state-changing action behind a separate confirmation step that isn't satisfied by anything found inside the document.

Third, add detection as a backstop, not the primary defense: scan uploaded documents for suspicious patterns (phrases like 'ignore previous instructions', unusual formatting trying to mimic system messages) and flag/strip them or require human review before processing, and monitor output for signs the bot's behavior deviated from its intended role (e.g., a support bot's answer that doesn't reference the document at all, or that reveals internal instructions) as a way to catch injection attempts that got past the first two layers.

Question 40

What is PII leakage risk in LLM-powered applications, and how do you guard against it?

Accepted Answer

PII leakage happens at multiple points in an LLM pipeline: the model might echo back sensitive data (SSNs, phone numbers, addresses) that appeared earlier in the conversation or in retrieved documents when it shouldn't; logs/analytics pipelines might capture raw prompts and responses containing PII and store them insecurely or for too long; and if user data was ever used to fine-tune a model, there's a (smaller but real) risk of the model memorizing and later regurgitating specific training examples verbatim.

Guardrails at each point: for input, run a PII-detection/redaction pass (regex plus a lightweight NER model for names/addresses/etc.) before sensitive data reaches the LLM at all where possible — many use cases don't actually need the raw PII to answer the user's question, just a placeholder. For output, run the same kind of detection on the model's response before it's returned or logged, redacting anything that shouldn't be there. For logging/observability specifically, mask PII fields before they're written to logs or analytics stores, and apply data retention limits.

Example: a support bot summarizing a customer's account history shouldn't need to output the customer's full credit card number back to them in a chat response even if it's present in the retrieved account record — the retrieval and prompt layer should already only pass in the last 4 digits, so there's no raw PII in the model's context for it to leak in the first place, rather than relying on the model to correctly decide not to repeat it.

Question 41

What is data/benchmark contamination, and why is it a serious problem for evaluating LLMs?

Accepted Answer

Contamination happens when a model's training data includes the actual test questions (or very close paraphrases) of the benchmark being used to evaluate it — so a high score on that benchmark reflects memorization of the answer key, not the capability the benchmark was designed to measure. Given that modern LLMs train on enormous, loosely-curated scrapes of the public internet, and popular benchmarks (MMLU, GSM8K, HumanEval) are themselves publicly posted, discussed, and reproduced across the web, this is a real and hard-to-fully-avoid problem, not a hypothetical edge case.

It's serious because it silently inflates reported capability — a model that's actually mediocre at genuine reasoning can post an impressive benchmark score if enough of that benchmark's exact questions leaked into its training set, misleading both researchers comparing models and practitioners choosing a model for a real use case based on those numbers.

Detecting it typically involves n-gram overlap analysis between training data and benchmark questions, checking whether a model can complete a benchmark question's answer given only a truncated prefix (suspiciously high accuracy suggests memorization rather than reasoning), and the field's broader response has been to increasingly favor 'held-out' or continuously-refreshed benchmarks (new questions written after a model's training cutoff) and private/unpublished eval sets specifically because any sufficiently old, public benchmark should now be assumed to be at least partially contaminated in any model trained on broad web data.

Question 42

What is exposure bias in autoregressive text generation, and why does it cause errors to compound?

Accepted Answer

During training, autoregressive models use 'teacher forcing' — at each step, the model predicts the next token conditioned on the *actual, correct* preceding tokens from the training data, not on its own previous predictions. At inference time, though, there's no ground truth to condition on — the model has to condition each new token on the tokens *it itself already generated*, including any mistakes.

This mismatch is exposure bias: the model was never trained to recover from its own errors, because during training it was never actually exposed to its own error-containing outputs as input. So if it generates one slightly wrong or awkward token early in a sequence, that error becomes part of the context for every subsequent token, and since the model has no training experience handling 'this input contains my own earlier mistake', errors can compound rather than self-correct — a small early mistake nudges the whole rest of the generation further off track.

This is part of why techniques like beam search, self-consistency (sampling multiple full generations and aggregating), and RLHF-style training (which does expose the model to its own generated outputs during the RL phase, unlike pure next-token-prediction pretraining) all provide real, measurable benefit — they each, in different ways, address the fact that plain teacher-forced training never taught the model what to do when its own context already contains an error.

Question 43

What is grammar-constrained (structured) decoding, and how is it different from just prompting a model to output JSON?

Accepted Answer

Prompting for JSON relies entirely on the model choosing, token by token, to produce syntactically valid output that matches your schema — it usually works, but it's probabilistic, and the model can still emit a stray trailing comma, an unescaped quote, or an extra field, especially under pressure from a long or complex prompt. Grammar-constrained decoding instead modifies the actual token sampling process at inference time: at each step, it computes which tokens would be *grammatically valid* continuations given a formal grammar (e.g., a JSON schema compiled into a context-free grammar or finite state machine), and masks out (sets to zero probability) every token that would violate that grammar, so the model is structurally incapable of producing invalid output — not because it decided to follow the schema, but because invalid tokens are never even in the sampling pool.

This is a much stronger guarantee than prompting: it doesn't matter how confused the model gets or how adversarial the input is, the output is provably syntactically valid, because validity is enforced by the decoding mechanism, not the model's judgment. It doesn't guarantee *semantic* correctness (the JSON can be valid but still contain a wrong value or violate a business rule your JSON schema doesn't express), so you'd still layer semantic validation on top.

Example: libraries like `outlines`, `guidance`, or a provider's native 'structured outputs' / 'JSON schema' mode implement exactly this — you supply a schema, and the constrained decoding guarantees the output parses, eliminating the retry/repair loop you'd otherwise need for pure prompt-based JSON generation.

Question 44

What's the difference between a system prompt and a user prompt, and why can't a system prompt alone fully prevent jailbreaks?

Accepted Answer

A system prompt sets the model's role, rules, and constraints for the whole conversation ('You are a helpful support assistant for Acme Corp; never discuss competitors; only answer questions about Acme products') and is typically set by the application developer, not the end user. A user prompt is the actual message from whoever is chatting with the bot, turn by turn. Models are generally trained to weight system-level instructions more heavily than user instructions, which is why system prompts are the primary lever developers have for controlling behavior.

But a system prompt is still just text in the same context window the model is doing next-token prediction over — it's a strong prior, not a hard, cryptographically-enforced boundary. A sufficiently creative user prompt (role-play framing, claiming special authority, multi-turn social engineering, or exploiting known weaknesses in how the model weighs conflicting instructions) can sometimes get the model to deprioritize or override the system prompt's constraints, because there's no architectural mechanism that makes system-prompt instructions literally unbreakable the way, say, a server-side permission check is.

This is exactly why real security for anything sensitive can't rely on the system prompt alone — it needs to be backed by deterministic, non-model-based enforcement (the payments-guardrail pattern of gating real actions behind checks outside the LLM's control is the same principle applied here): the system prompt shapes normal behavior, but actual security boundaries need to exist outside the model's discretion entirely.

Question 45

What is a context window, and why can't a model just always have an unlimited one?

Accepted Answer

The context window is the maximum number of tokens (prompt plus generated output combined) a model can process in a single call — everything the model 'sees' has to fit within it, and anything beyond it is simply invisible to the model, not truncated gracefully by the model itself unless your application handles that.

The limit isn't arbitrary policy, it's a real computational constraint: self-attention's cost scales quadratically with sequence length (as discussed with FlashAttention), so doubling context length roughly quadruples the raw attention computation, and the KV cache needed to hold all those tokens' Key/Value vectors during generation grows linearly with length and can dominate available GPU memory for long contexts, especially at the batch sizes needed to serve many users concurrently.

So while context windows have grown dramatically (from ~2k-4k tokens in early GPT models to 100k-1M+ in current frontier models), it's always a real engineering trade-off, not a free upgrade: longer context windows cost more GPU memory and compute per request, meaning higher latency and higher price per call, and — separately from raw feasibility — many models also show measurably worse recall/attention to information buried in the *middle* of a very long context (the 'lost in the middle' effect) even when it technically fits, which is why good context management (retrieval, summarization, budgeting) still matters even as raw window sizes keep growing.

Question 46

What is semantic caching for LLM APIs, and when does it actually save meaningful cost?

Accepted Answer

A normal exact-match cache only helps when the identical request string is repeated verbatim — rare for natural-language queries, since users phrase the same underlying question in many different ways ('what's your refund policy' vs. 'can I get my money back'). Semantic caching instead embeds each incoming query, and before calling the LLM, checks whether a sufficiently similar query (above some cosine-similarity threshold) has already been answered and cached — if so, it returns the cached response (or uses it as a starting point) instead of making a fresh, full-cost LLM call.

This saves meaningful cost specifically in high-volume, narrow-domain applications where many users ask semantically-repetitive questions in different words — a support bot fielding thousands of daily queries about a small set of common issues is a good fit; a highly open-ended creative-writing or coding-assistant use case, where queries are genuinely unique each time, gets little benefit because there's rarely a close-enough semantic match to reuse.

The engineering risk to manage is the similarity threshold: set it too loose and you'll serve a cached answer to a query that's similar but meaningfully different (e.g., 'can I cancel my subscription' vs. 'can I cancel my order' might embed close together but need different answers), returning a wrong or irrelevant cached response; set it too strict and you rarely get cache hits at all. Most production semantic caches also scope the threshold and cache pool per intent/topic rather than using one global threshold, precisely to avoid that false-positive-match risk.

Question 47

What is prompt compression, and why would you use it instead of just fitting content into the context window?

Accepted Answer

Even when content technically fits within the context window, every extra token still costs money and latency (both are typically priced/scaled per input token), and — separately — models tend to attend less reliably to information in very long contexts (the 'lost in the middle' effect mentioned earlier), so a long prompt full of moderately-relevant content isn't just expensive, it can genuinely produce worse answers than a shorter, denser one.

Prompt compression addresses this by reducing the token count of the content you send while preserving the information the model actually needs — ranging from simple approaches (summarizing lower-priority retrieved chunks instead of including them verbatim, as in the earlier context-budgeting example) to dedicated compression models/techniques (e.g., LLMLingua) that use a smaller model to score each token's or phrase's importance to the overall meaning and drop the low-information ones (articles, redundant phrasing, filler) while keeping information-dense tokens intact, achieving meaningful token reduction (often 2-5x) with limited loss to downstream task accuracy.

Example: instead of sending a full 3,000-token customer conversation history as context for a follow-up question, run it through a compression pass that keeps the key facts and decisions (order number, issue described, resolution offered) and drops conversational filler and repetition, cutting it to maybe 600 tokens — reducing cost and latency on every subsequent call in that conversation while keeping the information the model needs to answer correctly.

Question 48

What's the difference between diffusion models and GANs for image generation, and why did diffusion become dominant?

Accepted Answer

GANs (Generative Adversarial Networks) train two networks against each other — a generator that tries to produce realistic images from random noise, and a discriminator that tries to distinguish generated images from real ones — with the generator improving by learning to fool an increasingly good discriminator. This adversarial setup is notoriously unstable to train (the two networks can fail to converge, or the generator can collapse to producing only a few types of output that reliably fool the current discriminator, known as mode collapse), though it generates images in a single fast forward pass.

Diffusion models instead train a network to reverse a gradual noising process — starting from a real image, you progressively add random noise over many steps until it's pure noise, and train the model to predict and remove a small amount of that noise at each step. Generation then runs this in reverse: start from pure random noise and iteratively denoise it, step by step, into a coherent image. This training objective (predict the noise at each step) is much more stable than the adversarial min-max game GANs require, and diffusion models produce a wider diversity of high-quality outputs without the mode-collapse failure mode.

The trade-off is speed: diffusion needs many iterative denoising steps (though modern samplers have cut this from ~1000 steps down to as few as 4-20 with techniques like DDIM and distillation) to generate one image, versus a GAN's single pass. Diffusion became dominant (Stable Diffusion, DALL-E 2/3, Midjourney) primarily because training stability and output diversity mattered more in practice than the per-image generation speed disadvantage, especially once faster samplers closed much of that gap.

Question 49

How does a text-to-image diffusion model actually turn a text prompt into an image?

Accepted Answer

The core generative process is the reverse-denoising loop described for diffusion models generally: start with an image made of pure random noise, and repeatedly ask the trained model 'what noise should I remove to make this slightly more like a real image', subtracting a bit of predicted noise at each of many steps, until what remains is a coherent image. Text conditioning is what steers that denoising process toward an image matching the prompt rather than an arbitrary realistic-looking image.

The text prompt is first encoded into an embedding (typically via a text encoder like CLIP's text tower), and that embedding is injected into the denoising network at every step — usually via cross-attention layers inside the U-Net (or transformer-based) denoiser, the same attention mechanism used in language models, but here each image-patch 'query' attends to the text embedding's 'keys/values' so each part of the image being denoised can be influenced by relevant words in the prompt. A technique called classifier-free guidance amplifies this effect further by running the denoising step both with and without the text conditioning and pushing the result further in the direction the text conditioning implies, which is why increasing the 'guidance scale' setting in most tools produces images that match the prompt more literally (at some cost to naturalness).

Most modern systems (Stable Diffusion) also don't denoise in raw pixel space — they operate in a much smaller compressed 'latent space' produced by a separately trained autoencoder, denoising a compact latent representation and only decoding it to a full-resolution pixel image at the very end, which is what makes them fast enough to run on consumer GPUs instead of requiring the compute needed to denoise a full-resolution image directly at every step.

Question 50

What does 'multimodal' mean in generative AI, and how do vision-language models combine image and text understanding?

Accepted Answer

A multimodal model can take in and/or generate more than one type of data — text and images together being the most common combination today, but also audio, video, or structured data. This is different from having a separate text model and a separate image model that you awkwardly glue together with, say, a caption as the interface; a genuinely multimodal model reasons jointly across both modalities within the same architecture.

A common approach for vision-language models (like GPT-4V, LLaVA, or Gemini) is to use a pretrained image encoder (often a Vision Transformer, ViT) to convert an image into a sequence of patch embeddings — conceptually similar to how text is turned into a sequence of token embeddings — and then project those image embeddings into the same vector space the language model's token embeddings live in, via a learned adapter/projection layer. Once image content is represented as a sequence of vectors compatible with the language model's embedding space, the language model's existing self-attention mechanism can attend across both the image patch embeddings and the text token embeddings together in a single unified sequence, letting it answer questions that require jointly reasoning about both.

Example: given a photo of a receipt and the question 'How much did I tip?', the model attends across the image patches covering the tip-line region of the receipt and the text tokens of the question simultaneously, rather than running OCR and language understanding as two disconnected pipeline stages — which is part of why these models can handle messy, real-world images (handwriting, unusual layouts) better than a rigid OCR-then-parse pipeline would.

Question 51

What is Generative AI, and how is it different from traditional software or traditional AI?

Accepted Answer

Generative AI refers to AI systems that create new content — text, images, code, audio, video — rather than just analyzing or classifying existing data. Traditional software follows fixed, human-written rules to produce a predictable output for a given input (e.g., a calculator app always returns 4 for 2+2, because someone wrote that exact logic). Traditional AI/ML models (like a spam classifier or a fraud detector) typically look at input data and output a label or a number — they decide or predict, but they don't produce new original content.

Generative AI models, in contrast, are trained on huge amounts of existing content (text, images, etc.) so they learn the underlying patterns of that content well enough to generate brand-new examples that follow those same patterns — a sentence that was never written before, or an image that never existed. Nobody hand-coded the rules for what makes a good sentence; the model learned it statistically from data.

A simple real-world comparison: a traditional spam filter looks at an email and outputs 'spam' or 'not spam' (a decision about existing input). A generative AI tool like ChatGPT, given the instruction 'write a polite reply declining a meeting,' produces an entirely new paragraph of text that didn't exist before — that's the generative part. Similarly, a traditional photo-editing app only modifies a photo you give it, while a generative image tool like a text-to-image model can create a brand-new photo-realistic image from just a text description, with no starting photo at all.

Question 52

What's the difference between discriminative models and generative models?

Accepted Answer

A discriminative model learns to draw a boundary between categories in existing data — given an input, it predicts which class or label it belongs to. It answers questions like 'is this email spam or not?' or 'is this a picture of a cat or a dog?'. It's only ever deciding among options that already exist; it never produces new content.

A generative model, instead, learns the actual underlying distribution of the data well enough that it can produce brand-new examples that look like they could have come from that same distribution — it answers 'what would a typical example of this look like?' rather than 'which category does this belong to?'. This is why the same email-spam idea can be approached two ways: a discriminative spam filter labels an existing email; a generative model could, in principle, be trained to actually write a realistic-looking spam email or a realistic-looking legitimate email, because it has learned what both categories statistically look like, not just how to tell them apart.

A simple way to remember it: imagine trying to identify a language versus trying to speak it. A discriminative model is like someone who can correctly say 'this sentence is French, not Spanish' without knowing how to write a single sentence of either language themselves. A generative model is like someone who has actually learned the language well enough to write new French sentences from scratch. Tools like ChatGPT or an image generator are generative models; a resume-screening tool that just labels a resume as 'qualified' or 'not qualified' is a discriminative model.

Question 53

What is a Large Language Model (LLM)?

Accepted Answer

A Large Language Model is a type of AI model trained on massive amounts of text (books, websites, articles, code, and more) to learn the statistical patterns of human language — which words and phrases tend to follow which other words and phrases in which contexts. At its core, an LLM is really good at one simple task repeated over and over: given some text so far, predict the most likely next word (or more precisely, the next 'token', a small chunk of text). By doing this one word at a time, over and over, it can generate entire coherent paragraphs, answer questions, summarize documents, write code, and hold conversations.

The word 'Large' refers to two things at once: the model has a huge number of internal parameters (the learned numerical values that encode what it knows — modern LLMs have billions of them), and it's trained on a huge volume of text data. This scale is what allows an LLM to pick up not just grammar, but facts, reasoning patterns, and even some coding ability, purely from seeing enough examples during training — nobody explicitly programmed in grammar rules or facts.

Example: if you type 'The capital of France is' into an LLM, it doesn't 'look up' Paris in a database — it calculates, based on everything it learned from training text, that the word 'Paris' is overwhelmingly the most statistically likely next word given that context, and outputs it. GPT-4, Google's Gemini, Anthropic's Claude, and Meta's Llama are all well-known examples of LLMs.

Question 54

What is a Foundation Model, and how is it different from a model trained for one specific task?

Accepted Answer

A foundation model is a large, general-purpose AI model trained once on a huge, broad dataset, which can then be adapted or directly used for many different downstream tasks — instead of building and training a completely separate model from scratch for every single task you want to solve. Its name comes from the idea that it forms the 'foundation' other, more specific applications can be built on top of.

This is a big shift from the older approach in machine learning, where a team would typically collect a task-specific labeled dataset (say, thousands of labeled customer support tickets) and train a model from zero just for that one job (categorizing support tickets), which then couldn't be reused for anything else. A foundation model, by contrast, already 'knows' a broad amount about language, reasoning, and (for multimodal ones) images, from its original large-scale training — and can be pointed at a new task just by giving it good instructions (prompting) or, if needed, lightly adapting it (fine-tuning), without retraining from scratch.

Example: GPT-4 is a foundation model. One company might use it, unmodified, just by prompting it well to build a customer support chatbot. Another company might use that exact same base model to build a code-writing assistant. A third might lightly fine-tune it to write in a specific brand voice. All three are reusing one foundation model for very different specific jobs, instead of each training their own model from scratch — which is a major reason foundation models made building AI products dramatically faster and cheaper.

Question 55

What is Prompt Engineering, and why does the way you phrase a prompt affect the quality of the AI's response?

Accepted Answer

Prompt engineering is the practice of carefully crafting the instructions (the 'prompt') you give to an AI model so that it produces the response you actually want. Because an LLM generates its answer based entirely on the text it's given as input, small changes in how a question is worded, how much context is provided, or how the desired output format is described can meaningfully change the quality, accuracy, and usefulness of what comes back.

This matters because an LLM has no way to read your mind about what you actually meant — it only has the literal text of the prompt to work from. A vague or ambiguous prompt gives the model more room to guess wrong about what you want, while a specific, well-structured prompt narrows down the possibilities and guides it toward a better answer. Good prompt engineering typically means being specific about the task, providing relevant context or examples, specifying the desired format, and breaking complex requests into clear steps.

Example: asking an LLM 'Tell me about dogs' will likely produce a generic, unfocused paragraph, because the model has no idea what aspect of dogs you actually care about. Asking instead 'List 5 dog breeds that are good for first-time owners living in a small apartment, with one sentence explaining why for each' gives the model a clear task, a clear audience, a clear constraint (apartment-friendly), and a clear output format — and will reliably produce a much more useful, targeted answer, even though it's the exact same underlying model answering both times.

Question 56

What's the difference between zero-shot, one-shot, and few-shot prompting?

Accepted Answer

These terms describe how many worked examples you give the model inside the prompt before asking it to do the actual task, and they're a basic but very common way to steer a model's output without any training or fine-tuning at all. Zero-shot means you give the model no examples at all — just a plain instruction — and rely entirely on what it already learned during its original training to figure out how to respond. One-shot means you give exactly one example of the task done correctly before asking it to do a new one. Few-shot means you give several examples (commonly 2-10) before the actual request.

The general pattern is that more relevant examples usually make the model's output more consistent and closer to the format/style you want, because you're showing it exactly what a good answer looks like rather than just describing it in words — this is especially useful for tasks with a specific output format or an unusual pattern the model might not default to on its own. The trade-off is that each example takes up extra space in the prompt (and therefore costs more tokens), so you use only as many as actually improve the result.

Example — zero-shot: 'Classify this review as positive or negative: The food was cold and the service was slow.' Few-shot: 'Review: Amazing food, loved it! -> positive. Review: Terrible experience, never going back. -> negative. Review: The food was cold and the service was slow. -> ?' The few-shot version shows the model the exact input-to-output pattern first, which often makes the output format more reliable, especially for something more specific than a simple positive/negative label.

Question 57

What is hallucination in Generative AI, and why does it happen?

Accepted Answer

Hallucination is when an AI model generates information that sounds fluent and confident but is actually false, made up, or not supported by any real source — for example, citing a research paper that doesn't exist, or stating an incorrect fact with the same confident tone it would use for a correct one. It's called 'hallucination' because the model isn't lying on purpose or aware it's wrong — it's simply generating the most statistically plausible-sounding continuation of text, and sometimes that plausible-sounding text just isn't true.

It happens because of how these models fundamentally work: an LLM doesn't have a database of verified facts it looks answers up in — it generates text one token at a time based on learned patterns of what usually follows what, from its training data. If a topic is rare, ambiguous, or genuinely unknown to the model (say, it's asked about an event after its training cutoff, or a very niche real-world detail it saw little of during training), it will still generate a smooth, grammatically correct, confident-sounding answer, because generating fluent text is what it's optimized to do — it has no built-in mechanism to say 'I actually don't know this for certain' unless it's been specifically trained or prompted to recognize and express that uncertainty.

Example: if you ask an LLM for the exact page number of a specific quote in a real book, it might confidently return a page number that is completely wrong, because it learned the general shape of 'a book citation looks like this' far better than it memorized that one exact fact. This is exactly why real-world applications add safeguards like retrieval-augmented generation (grounding answers in actual retrieved documents) rather than trusting the model's own unaided memory for anything that needs to be factually reliable.

Question 58

What is a context window in an LLM, and what happens if your input exceeds it?

Accepted Answer

The context window is the maximum amount of text (measured in tokens, not words or characters) that a language model can 'see' and consider at one time — it includes the conversation history, any documents or instructions you've provided, and the response the model is generating, all combined. Think of it as the model's short-term working memory for a single request: anything within that window, the model can refer back to; anything outside it, the model simply cannot see at all, as if it were never said.

Different models have different context window sizes — older models supported only a few thousand tokens, while many modern models support context windows in the hundreds of thousands of tokens, letting them process entire books or codebases in a single request. A larger context window lets you provide more background information, longer conversation history, or bigger documents for the model to work with.

If your input (plus the expected output) exceeds the context window, it doesn't fit — practically, this usually means the oldest parts of a long conversation get silently dropped or truncated, or the API call returns an explicit error saying the input is too long, depending on how the application is built. Example: if a chatbot has an 8,000-token context window and you've been chatting with it for a very long time, once the conversation history plus your new message goes over 8,000 tokens, the app typically has to drop the earliest messages to make room for the newest ones — which is exactly why a long-running chatbot conversation can suddenly seem to 'forget' something you mentioned much earlier.

Question 59

What is a diffusion model, and how do text-to-image generators like Stable Diffusion or DALL-E work at a basic level?

Accepted Answer

A diffusion model is a type of generative AI model commonly used for creating images (and increasingly audio/video), and it works in a somewhat counter-intuitive way: it's trained by taking real images and progressively adding random noise to them, step by step, until the image is pure static, and then training the model to reverse that process — to predict, at each step, how to remove a little bit of noise and recover something closer to a real image. After enough training on this 'add noise, then learn to remove it' process across millions of images, the model becomes very good at turning random noise into a coherent, realistic image.

To actually generate a new image from a text prompt, the process starts with a canvas of pure random noise, and the model repeatedly denoises it step by step — at each step nudging the noisy image to look a little more like a real image, and specifically one that matches the text description it was given (via a separate text-understanding component that guides the denoising toward the prompt's meaning). After enough denoising steps (commonly a few dozen), what started as random static has been gradually sculpted into a coherent image matching the prompt.

A simple analogy: imagine a sculptor starting with a rough, shapeless block of noisy static instead of marble, and chipping away at it in small steps, each time making it look a little more like 'a golden retriever wearing sunglasses on a beach' (the prompt), until after enough small refinements a clear, coherent image emerges. This step-by-step 'noise to image' approach is fundamentally different from how text-generating LLMs work (predicting one token at a time in sequence), which is why image generation and text generation are usually built as separate model types even within the same generative AI product.

Generative AI Interview Questions & Answers

Ready to practice these live?