Stop 11 · Large Language ModelsTopics · 8/9

Emergent Abilities (Overview)

In short

Small models fail at some tasks almost completely, and then bigger models suddenly pass them. Researchers called these 'emergent abilities'. Whether they're truly sudden is still debated, which makes this fun to understand.

Capabilities (like multi-step reasoning) that appear only past a certain model scale, not present in smaller models trained the same way — a debated but influential concept in LLM research.

The blob watches a pressure cooker on a stove suddenly whistle with a big puff of steam.

Think of it like this

The pressure cooker whistle

You put dal on the stove. For minutes nothing seems to happen. Then, all at once, the whistle goes. But heat was building steadily the whole time; you just couldn't see it. Some 'sudden' model abilities may work the same way.

Watch it happen

Exact-match score vs model size (illustrative)

small

medium

large

x-large

1/5Four models of growing size try a multi-step arithmetic task.

The big ideas

What you need to know

The original claim

A 2022 paper listed tasks where performance stayed near random until models passed a certain scale.

The metric matters

A 2023 paper argued many jumps come from strict metrics. Smoother metrics often show steady improvement.

Examples people cite

Multi-step arithmetic, following few-shot examples and chain-of-thought reasoning improving a lot in larger models.

Practical lesson

Don't assume a small model can't do a task. Test it, try better prompts, and measure with sensible metrics.

Save yourself the pain

Common mistakes

Oops

Treating 'emergence' as magic or proof of consciousness.

Do this instead

It's an observation about benchmark curves, and part of it depends on how you measure.

Oops

Concluding a task needs the biggest model after one failed prompt.

Do this instead

Try few-shot examples, step-by-step prompting or a fine-tuned small model before paying for a giant one.

Your mission

Test sizes yourself

~30 min

Quick check

Why do some researchers doubt that emergent abilities are truly sudden?

Go deeper

1Read examples of emergent abilities cited in LLM research.
2Understand the ongoing debate about whether 'emergence' is real or a measurement artifact.
3Connect this to why capability jumps between model generations can feel discontinuous.