Stop 03 · Python for Data ScienceTopics · 7/9

Working With APIs

In short

Lots of useful data isn't in a CSV. It lives behind APIs: weather, prices, public records. Knowing how to fetch it with Python makes your projects stand out.

Using the requests library to pull data directly from a web API — a common way to source real datasets when nothing suitable exists as a static file.

The mascot waiter carries a covered dish on a tray out of a kitchen hatch.

Think of it like this

Ordering through a waiter

You don't walk into the restaurant kitchen. You read the menu, tell the waiter exactly what you want, and they bring it back on a plate. An API is that waiter: the docs are the menu, your request is the order, and JSON is the plate.

Watch it happen

Fetching every page of results

Your notebook

Weather API

GET ?city=Pune&page=1

1/6You ask for the first page of results, with the city as a query parameter.

The big ideas

What you need to know

Requests and responses

You send a URL with parameters. You get a status code, like 200 for success, and a body, usually JSON.

Keys stay secret

Many APIs need an API key. Keep it in an environment variable or Colab secrets, never in shared code.

Results come in pages

Loop through pages until the API returns nothing, or follow its 'next' link.

Respect rate limits

Too many calls too fast gets you a 429. Wait and retry, and cache what you've already fetched.

See it in code

Walk through the code

python

1import time, requests2import pandas as pd3URL = "https://api.example.com/v1/weather"   # placeholder API4rows, page = [], 15while True:6    r = requests.get(URL, params={"city": "Pune", "page": page}, timeout=10)7    if r.status_code == 429:                  # too many requests: wait8        time.sleep(5)9        continue10    r.raise_for_status()11    batch = r.json()["results"]12    if not batch:13        break14    rows += batch15    page += 116df = pd.DataFrame(rows)

1/4Send a GET request with query parameters and a timeout so it can't hang forever.

A reusable pattern for paged APIs. Swap in a real API's URL and field names from its docs.

Save yourself the pain

Common mistakes

Oops

Pasting your API key directly into a notebook you share on GitHub.

Do this instead

Read it from an environment variable or Colab's Secrets panel, and revoke any key you've leaked.

Oops

Calling the API again every time you rerun the notebook.

Do this instead

Save the results to a CSV or Parquet file once, then load from disk while you explore.

Your mission

Pull live data

~30 min

Quick check

An API returns status code 429. What does it mean?

Go deeper

1Use the requests library to pull data from a free public API.
2Practice parsing a JSON API response into a Pandas DataFrame.
3Learn basic API concepts: endpoints, parameters, and authentication keys.