Stop 08 · Data Cleaning & WranglingTopics · 9/9

Documenting Data Cleaning Decisions

In short

Three months later someone asks why your revenue is 4% lower than finance's. If your cleaning decisions are written down, it's a two-minute answer. If not, it's a lost week.

Writing down what you changed and why — missing-value handling, outlier decisions, deduplication logic — so your analysis is reproducible and defensible when someone questions a number.

The jaadu stirs a cooking pot while writing notes in a small diary.

Think of it like this

Grandma's recipe diary

Her diary doesn't just say 'dal'. It says 'less salt for Papa, soaked overnight because the new batch is hard'. The why matters as much as the what. A cleaning log is your recipe diary: what you changed, why, and how many rows it touched.

Watch it happen

Building a cleaning log, entry by entry

push(Removed 16 exact dups)

Removed 16 exact dups

Top of the stack is coloured

1/6Entry 1: what you did and the count. 1,200 rows became 1,184.

The big ideas

What you need to know

What, why, impact

Each entry: the change, the reason, and how many rows or how much value it affected.

Record judgement calls

Dropping vs filling, capping vs keeping. Note the option you rejected and why.

Keep it near the code

A markdown cell in the notebook or a README beside the script. Not in someone's memory.

Data dictionary

List each column, its meaning, type and allowed values. Your future self will thank you.

See it in code

Walk through the code

text

1## Cleaning log: orders_oct_2026.csv2Source: app export, pulled 2026-10-27, 1,200 rows3 4| # | Step                         | Why                               | Rows   |5|---|------------------------------|-----------------------------------|--------|6| 1 | Removed exact duplicates     | File loaded twice by export job   | -16    |7| 2 | Dropped rows, no order_id    | Can't match to payments           | -9     |8| 3 | Mapped city variants         | Bangalore/BLR -> Bengaluru        | 41 fix |9| 4 | Flagged minutes > 42 (IQR)   | Kept, flagged; agreed with ops    | 12 flg |10 11Final: 1,175 rows. Revenue vs finance: -0.6% (refund timing).

1/4Name the file, its source, when it was pulled and the starting row count.

A cleaning log in markdown. Short, specific, and it answers 'why don't the numbers match?' before anyone asks.

Save yourself the pain

Common mistakes

Oops

Writing 'cleaned the data' as the only note.

Do this instead

List each step with a reason and row counts. Vague notes are as good as none.

Oops

Keeping the log in a separate doc nobody can find.

Do this instead

Keep it with the notebook or code, and link it from any report that uses the data.

Your mission

Log a past cleaning

~20 min

Quick check

Which log entry is most useful to a teammate reading it later?

Go deeper

1Add a 'data cleaning log' section to your next analysis, listing every transformation made.
2Practice explaining a cleaning decision to a skeptical stakeholder in one sentence.
3Keep your raw and cleaned data separate so the transformation is always auditable.