The math you actually need

Vectors: data as points in space

A vector is an ordered list of numbers. In machine learning, everything becomes one. A house is [3 bedrooms, 120 m2, built 1995]. A server at one moment is [CPU 0.7, memory 0.4, 240 requests per second]. A sentence is turned by a language model into an embedding: a vector of several hundred or a few thousand numbers, arranged so that texts with similar meaning land close together. The number of entries is the vector's dimension.

Thinking of a vector as a point, or as an arrow from the origin to that point, gives you geometry for free. Similar things are near each other. Questions such as "which documents are most like this query?" become questions about distance and angle. Three operations cover nearly everything.

a = [3, 4]        b = [4, 3]

dot product   a . b  = 3x4 + 4x3            = 24
norm (length) |a|    = sqrt(3^2 + 4^2)      = 5          |b| = 5
cosine        cos    = a . b / (|a| |b|)    = 24 / 25    = 0.96

distance      |a - b| = |[-1, 1]| = sqrt(1 + 1)          = 1.414

The dot product multiplies matching entries and adds the results. It is large when two vectors point the same way and have large entries, zero when they are perpendicular, and negative when they point in opposite directions. It is the single most common computation in machine learning: a neuron is a dot product, and so is attention in a transformer.

Cosine similarity is the dot product with the lengths divided out, so it measures only direction. It runs from 1, meaning the same direction, through 0, meaning unrelated, to -1, meaning opposite. It is the standard way to compare embeddings, because for text the direction carries the meaning and the length mostly does not. If vectors are first normalised to length 1, cosine similarity is simply the dot product, which is why vector databases often store normalised embeddings.

import numpy as np

a = np.array([3.0, 4.0])
b = np.array([4.0, 3.0])

a @ b                                   # dot product -> 24.0
np.linalg.norm(a)                       # length -> 5.0
(a @ b) / (np.linalg.norm(a) * np.linalg.norm(b))   # cosine -> 0.96
np.linalg.norm(a - b)                   # Euclidean distance -> 1.414...

# compare one query against many documents at once
docs = np.array([[4.0, 3.0], [-3.0, -4.0], [4.0, -3.0]])
docs_n = docs / np.linalg.norm(docs, axis=1, keepdims=True)
q_n = a / np.linalg.norm(a)
docs_n @ q_n                            # -> [ 0.96, -1.0, 0.0 ]

Matrices: transforming many vectors at once

A matrix is a grid of numbers with a shape of rows by columns. It plays two roles. As data, a matrix is a table with one row per example and one column per feature, which is exactly a NumPy array or a DataFrame. As a transformation, a matrix is a machine that turns one vector into another, and this is what the weights of a model are.

Multiplying a matrix by a vector takes the dot product of each row of the matrix with the vector. Each output number is therefore a weighted mix of all the inputs.

W = | 1  2 |      x = | 5 |
    | 3  4 |          | 6 |

W x = | 1x5 + 2x6 |  =  | 17 |
      | 3x5 + 4x6 |     | 39 |

Shape rule:   (m x n) @ (n x p)  ->  (m x p)     the inner sizes must match
              (2 x 2) @ (2 x 1)  ->  (2 x 1)

The shape rule is the most practical thing to take from linear algebra. When a deep learning library reports a shape mismatch, which it will, the fix is to write down the shape at every step and find where the inner dimensions disagree. Note also that order matters: in general A @ B is not equal to B @ A.

One layer of a neural network is exactly this, applied to a whole batch of inputs at once, with a bias vector added and a simple non-linear function applied to the result.

import numpy as np

rng = np.random.default_rng(0)

X = rng.normal(size=(32, 10))        # a batch of 32 examples with 10 features each
W = rng.normal(size=(10, 4))         # weights: 10 inputs -> 4 outputs
b = np.zeros(4)                      # one bias per output

Z = X @ W + b                        # (32, 10) @ (10, 4) -> (32, 4); b is broadcast
H = np.maximum(0, Z)                 # ReLU: the non-linearity, applied element-wise
H.shape                              # (32, 4)

W.T.shape                            # transpose swaps rows and columns -> (4, 10)

Without the non-linear step, stacking layers would be pointless, because two matrix multiplications in a row are equivalent to one. The non-linearity is what lets a network represent curves, thresholds and interactions. The deep learning module builds on this.

Probability: reasoning under uncertainty

Models do not output facts. A classifier outputs a probability, and your data is a sample, not the whole truth. A few concepts let you reason about both.

data: 2, 4, 4, 4, 5, 5, 7, 9

mean                = 40 / 8                         = 5
variance            = average squared distance from the mean
                    = (9+1+1+1+0+0+4+16) / 8         = 4
standard deviation  = sqrt(variance)                 = 2

median              = middle value (average of 4 and 5)  = 4.5

The mean is pulled about by outliers and the median is not, which is why the SRE track insists on percentiles for latency. The standard deviation measures spread in the same units as the data. One detail catches people out: np.std divides by n by default, as above, while pandas .std() divides by n minus 1, which is the better estimate when your data is a sample. For the data above they give 2.0 and about 2.14.

Conditional probability, written P(A | B), is the probability of A given that B is known to be true. Bayes' rule lets you reverse a condition, and it produces the most important counter-intuitive result in applied machine learning.

An anomaly detector watches 10,000 hosts. 1% are really faulty.
It catches 99% of faulty hosts, and wrongly flags 5% of healthy ones.
A host is flagged. How likely is it to be faulty?

  faulty:   100 hosts   -> 99% flagged  ->   99 true alarms
  healthy: 9,900 hosts  ->  5% flagged  ->  495 false alarms

  P(faulty | flagged) = 99 / (99 + 495) = 99 / 594 = 16.7%

Bayes' rule:  P(A|B) = P(B|A) x P(A) / P(B)
            = 0.99 x 0.01 / (0.99 x 0.01 + 0.05 x 0.99) = 0.0099 / 0.0594 = 0.167

A detector that is "99% accurate" on faulty hosts still produces five false alarms for every real one, because faults are rare. This is the base rate effect. It explains alert fatigue from the SRE track in a single calculation, and it is why accuracy is a misleading measure whenever one class is rare. The next module turns this into precision and recall.

• A distribution describes how likely each value is. The normal distribution, the bell curve, describes many natural measurements. Latency, income and file sizes are not normal: they are heavy-tailed, with rare, very large values.
• Expected value is the long-run average: each outcome multiplied by its probability, added up. It is what a loss function averages over your data.
• Independence means that knowing one event tells you nothing about another. Much of statistics assumes it, and real data often violates it, as with requests from one user, or readings from one host over time.
• Correlation measures how two variables move together, from -1 to 1. It says nothing about which causes which, or whether a third factor drives both.

Gradients: how models learn

Training a model means finding the parameter values that make its predictions least wrong. "Wrong" is measured by a loss function, a single number that is large for bad predictions and small for good ones. Training is therefore a search for the lowest point of the loss, across millions of parameters.

The derivative of a function is its slope: how much the output changes when the input is nudged. The gradient is the same idea for a function of many parameters: a vector holding one slope per parameter, which together point in the direction of steepest increase. To reduce the loss, step the opposite way. That is gradient descent, and it is how every neural network learns.

loss(w) = (w - 3)^2          minimum at w = 3
slope   = 2 x (w - 3)

update rule:  w_new = w - learning_rate x slope         learning_rate = 0.1

  w = 0.000   slope = -6.000   ->  w = 0 - 0.1 x (-6.000)    = 0.600
  w = 0.600   slope = -4.800   ->  w = 0.6 + 0.480           = 1.080
  w = 1.080   slope = -3.840   ->  w = 1.08 + 0.384          = 1.464
  ...each step covers 20% of the remaining distance to 3

def slope(w: float) -> float:
    return 2 * (w - 3)


w, learning_rate = 0.0, 0.1
for step in range(50):
    w = w - learning_rate * slope(w)

print(round(w, 4))        # 3.0: converged to the minimum

The learning rate is the most important setting in training. Too small, and learning takes for ever. Too large, and each step overshoots the minimum. In the example above, a learning rate of 1.0 jumps from 0 to 6 and back indefinitely, and 1.1 moves further away with every step until the numbers overflow. A loss that bounces around or turns into NaN usually means that the learning rate is too high.

• With many parameters, the computation is identical, with one slope per parameter. Backpropagation is the algorithm that calculates all of them efficiently, by applying the chain rule backwards through the network. Frameworks such as PyTorch do it for you automatically.
• Stochastic gradient descent estimates the gradient from a small random batch of examples instead of the whole dataset. Each step is noisy but cheap, so you can take many more of them.
• Optimisers such as Adam adapt the step size for each parameter and add momentum. They are the usual default.
• Real loss surfaces have many valleys, and gradient descent finds a good one, not necessarily the best. In practice that is enough.

Softmax and cross-entropy

Two formulas sit at the output of nearly every classifier, and of every language model, which is a classifier over the next token. A network's final layer produces raw scores called logits, which can be any real numbers. Softmax converts them into a probability distribution: exponentiate each, which makes them all positive, then divide by the total, which makes them add up to 1.

logits              [ 2.0,    1.0,    0.0  ]
exponentiate        [ 7.389,  2.718,  1.000 ]      sum = 11.107
softmax             [ 0.665,  0.245,  0.090 ]      sums to 1

cross-entropy loss = -log(probability given to the correct class)

  correct class given 0.90  ->  -log(0.90) = 0.105     confident and right: small loss
  correct class given 0.50  ->  -log(0.50) = 0.693
  correct class given 0.10  ->  -log(0.10) = 2.303     confident and wrong: large loss

import numpy as np


def softmax(logits: np.ndarray) -> np.ndarray:
    shifted = logits - logits.max()          # subtracting the max avoids overflow
    exps = np.exp(shifted)
    return exps / exps.sum()


probs = softmax(np.array([2.0, 1.0, 0.0]))   # [0.665, 0.245, 0.090]
loss = -np.log(probs[0])                     # true class is index 0 -> 0.408

Cross-entropy punishes confident mistakes severely and rewards confident correct answers, which pushes the model towards calibrated probabilities. This is the loss that large language models are trained on, and perplexity, a figure often quoted for language models, is just the exponential of the average cross-entropy.

Softmax also explains temperature, a setting you will meet when working with language models. Dividing the logits by a temperature before the softmax changes how peaked the distribution is. A temperature below 1 sharpens it, so the most likely token dominates and the output becomes predictable. A temperature above 1 flattens it, so less likely tokens are chosen more often and the output becomes more varied. The LLM module returns to this.