Stop 22 · Interview PreparationTechnical · 3/8

LLM/GenAI Concepts

In short

Tokens, context windows, temperature, hallucination, RAG vs fine-tuning: interviewers now expect every AI engineer to explain these simply. Most candidates can use an LLM; fewer can say what it's actually doing.

A fast-growing interview category — expect questions on tokenization, context windows, RAG vs. fine-tuning, hallucination, and prompt engineering trade-offs.

The purple blob sits at an exam desk holding one small sheet covered in tiny squiggles.

Think of it like this

One A4 sheet in the exam

Imagine an exam where you may bring one A4 sheet of notes. Everything you need must fit on that sheet; what doesn't fit, you can't use. That sheet is the context window. And when you don't know an answer but write something confident anyway, that's a hallucination.

Watch it happen

How an LLM writes one word

Text

Tokens

Context

Scores

Sample

1/5Your prompt arrives as plain text: "Chai is ready, come".

The big ideas

What you need to know

Tokens, not words

Limits and pricing are in tokens. Other languages and code often use more tokens than English prose.

Temperature controls randomness

Low temperature picks likely tokens, good for extraction. Higher gives variety, good for brainstorming.

Hallucinations are built in

The model predicts plausible text, not checked facts. Grounding with RAG and asking for citations reduces it.

RAG vs fine-tuning

RAG adds fresh knowledge at question time. Fine-tuning changes behaviour, tone or format. Many systems use both.

See it in code

Walk through the code

python

1import numpy as np2import tiktoken3 4enc = tiktoken.get_encoding("cl100k_base")5ids = enc.encode("Chai is ready, come fast!")6print(len(ids), ids[:5])          # tokens, not words or characters7 8def sample_probs(logits, temperature):9    z = np.array(logits) / temperature10    p = np.exp(z - z.max())11    return p / p.sum()12 13logits = [2.0, 1.0, 0.1]          # scores for three candidate next tokens14print(sample_probs(logits, 0.5))  # sharper: top token dominates15print(sample_probs(logits, 1.5))  # flatter: more random

1/3Load a tokenizer and see how a sentence becomes token IDs.

Two ideas you can show in code: counting tokens with a real tokenizer, and how temperature changes the odds before a token is picked.

Save yourself the pain

Common mistakes

Oops

Saying temperature 0 makes the model accurate.

Do this instead

It makes output more repeatable, not more correct. A wrong answer at temperature 0 is just consistently wrong.

Oops

Treating context window and memory as the same thing.

Do this instead

The model only sees what's in this request. Chat apps fake memory by resending or summarising earlier messages.

Your mission

Explain five concepts simply

~35 min

Quick check

A chatbot needs to answer questions about a product catalogue that changes daily. What's the best fit?

Go deeper

1Be ready to explain RAG vs. fine-tuning trade-offs clearly and with a concrete example.
2Practice explaining how a Transformer generates text token by token.
3Stay current — this area changes fast, so review recent developments before interviewing.

More technical at this stop