Stop 02 · Math & Statistics for AITopics · 2/13

Matrix Operations

In short

A neural network layer is basically one matrix multiplication. Learn how multiply, transpose and solve work, and the maths inside linear regression and transformers stops feeling like magic.

Matrix multiplication, transposition and inversion — the operations that power linear regression, neural network layers, and embedding lookups.

The purple blob at a mess counter lines up plates of rotis with price tokens and an abacus.

Think of it like this

The hostel mess bill

Each student ate some rotis, dal and sweets this week. The mess has one price for each. Your bill is rotis × price + dal × price + sweets × price. Doing that for every student at once, quantities table times price list, is matrix multiplication.

Watch it happen

Aman's bill: one row times prices

4 × 10

2 × 30

1 × 25

1/5Aman's row is [4 rotis, 2 dal, 1 sweet]. Prices are [10, 30, 25]. Pair them up by position.

The big ideas

What you need to know

* and @ are different

a * b multiplies matching positions. a @ b is matrix multiplication, which sums products along rows and columns.

Inner sizes must match

(3, 4) @ (4, 2) works and gives (3, 2). (3, 4) @ (3, 2) fails. The middle numbers must agree.

Transpose flips rows and columns

A.T turns a (3, 2) matrix into (2, 3). You'll use it to line up shapes before multiplying.

Solve instead of invert

To solve Ax = b, use np.linalg.solve(A, b). It's faster and more accurate than computing the inverse yourself.

See it in code

Walk through the code

python

1import numpy as np2 3qty = np.array([[4, 2, 1],     # Aman: rotis, dal, sweets4                [6, 1, 0],     # Bela5                [2, 2, 2]])    # Chirag6price = np.array([10, 30, 25])7 8print(qty @ price)             # [125  90 130]  bill per student9print(qty.T @ np.ones(3))      # [12. 5. 3.]  total of each item10print(qty * 2)                 # every quantity doubled, same shape11 12A = np.array([[2, 1],13              [1, 3]])14b = np.array([5, 10])15print(np.linalg.solve(A, b))   # [1. 3.]  so 2x + y = 5, x + 3y = 10

1/4A 3 × 3 quantity matrix and a price vector of length 3.

qty @ price gives each student's bill. qty.T flips the table so you can total each item instead.

Save yourself the pain

Common mistakes

Oops

Using * when you meant matrix multiplication and getting a matrix instead of a vector.

Do this instead

Use @ (or np.matmul) for matrix multiplication. Use * only for element-wise work.

Oops

Assuming A @ B equals B @ A.

Do this instead

Order matters. Matrix multiplication is not commutative, and often only one order even fits the shapes.

Your mission

Compute your own mess bill

~15 min

Quick check

X has shape (100, 5) and w has shape (5,). What shape is X @ w?

Go deeper

1Practice matrix multiplication by hand on small matrices before trusting NumPy.
2Learn why matrix multiplication isn't commutative and what that implies for neural network layers.
3Understand matrix shapes carefully — shape mismatches are the most common bug in ML code.