Neural networks & deep learning

From a neuron to a network

A single neuron takes several numbers as input, multiplies each by a weight, adds them up together with a bias, and passes the result through an activation function. That is a dot product followed by a simple non-linear step, which you met in the maths module. A layer is many neurons looking at the same inputs, which is one matrix multiplication. A network is layers stacked so that each feeds the next.

one layer:    h = activation(x @ W + b)

input (10 features) -> [Linear 10->64] -> ReLU -> [Linear 64->32] -> ReLU -> [Linear 32->2] -> logits

parameters: (10x64 + 64) + (64x32 + 32) + (32x2 + 2) = 704 + 2,080 + 66 = 2,850

The activation function matters more than it looks. Without it, stacking layers would achieve nothing, because several matrix multiplications in a row collapse into a single one, and the network could only ever draw straight lines. The non-linearity lets each layer bend the space, and stacking layers lets the network compose simple bends into very complex shapes. ReLU, which is simply max(0, x), is the usual default. GELU is a smoother relative used in transformers.

The "deep" in deep learning refers to having many layers. What makes depth useful is that layers build on one another. In an image model, early layers respond to edges, middle layers to textures and shapes, and late layers to whole objects. Nobody programmed those features. They emerged because they reduced the loss. That is the central difference from the classical machine learning of the previous module, where you engineer the features by hand: a deep network learns its own features from raw data, which is why it dominates for images, audio and text, and why it needs so much data and computation.

The training loop

Training repeats four steps: run the inputs through the network to get predictions, which is the forward pass; measure how wrong they are with a loss function; calculate how each weight contributed to that error, which is the backward pass; and nudge every weight slightly in the direction that reduces the loss, which is the optimiser step. The backward pass uses backpropagation, the chain rule applied layer by layer from the output back to the input. PyTorch records the operations of the forward pass and calculates all the gradients for you.

import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

torch.manual_seed(0)
device = "cuda" if torch.cuda.is_available() else "cpu"


class Classifier(nn.Module):
    def __init__(self, n_features: int, n_classes: int):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(n_features, 64), nn.ReLU(), nn.Dropout(0.2),
            nn.Linear(64, 32), nn.ReLU(),
            nn.Linear(32, n_classes),          # raw logits; no softmax here
        )

    def forward(self, x):
        return self.net(x)


model = Classifier(n_features=10, n_classes=2).to(device)
loss_fn = nn.CrossEntropyLoss()                # applies softmax internally
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=1e-4)

# X_train: float32 tensor (N, 10).  y_train: int64 tensor (N,) of class indices
train_loader = DataLoader(TensorDataset(X_train, y_train), batch_size=64, shuffle=True)

for epoch in range(20):
    model.train()                              # enables dropout
    for xb, yb in train_loader:
        xb, yb = xb.to(device), yb.to(device)
        optimizer.zero_grad()                  # 1. clear gradients from the last step
        logits = model(xb)                     # 2. forward pass
        loss = loss_fn(logits, yb)             # 3. how wrong?
        loss.backward()                        # 4. backward pass: compute gradients
        optimizer.step()                       # 5. update the weights

    model.eval()                               # disables dropout
    with torch.no_grad():                      # no gradients needed for evaluation
        val_logits = model(X_val.to(device))
        val_loss = loss_fn(val_logits, y_val.to(device)).item()
        val_acc = (val_logits.argmax(dim=1).cpu() == y_val).float().mean().item()
    print(f"epoch {epoch:2d}  train_loss={loss.item():.3f}  val_loss={val_loss:.3f}  val_acc={val_acc:.3f}")

Deep learning runs on GPUs because the work is overwhelmingly matrix multiplication, which a GPU performs thousands of times in parallel. The practical constraint is usually GPU memory, which has to hold the weights, the gradients, the optimiser's state and the activations of the current batch. "CUDA out of memory" is the error you will meet most often, and the first remedy is a smaller batch size.

Keeping a deep network honest

A network with millions of parameters can memorise a training set with ease, so everything the previous module said about overfitting applies with extra force. The essential habit is to plot the training loss and the validation loss together as training proceeds.

• Early stopping: keep the weights from the epoch with the best validation loss, and stop when it has not improved for several epochs. It is simple and effective.
• Dropout randomly zeroes a share of activations during training, so that the network cannot depend on any single path.
• Weight decay is L2 regularisation: it gently pulls weights towards zero.
• Data augmentation creates plausible variations of the training examples, such as flipped or cropped images, or noise added to audio. It is effectively free data.
• Normalise the inputs, and use normalisation layers such as LayerNorm or BatchNorm inside deep networks, which makes training much more stable.
• Overfit one batch first. Before a long run, check that the model can drive the loss to nearly zero on a single small batch. If it cannot, there is a bug in the model or the data pipeline, and no amount of training will fix it.

Transformers at a glance

The transformer, introduced in 2017, is the architecture behind essentially all modern language models, and increasingly behind vision and audio models too. You do not need to be able to implement one. You do need its vocabulary, because it explains how language models behave, what they cost, and where their limits come from.

1. Tokens. Text is cut into pieces called tokens: common words, fragments of rarer words, punctuation. Each token is an integer ID from a fixed vocabulary of tens of thousands of entries. The model never sees letters or words, only token IDs.
2. Embeddings. Each token ID is looked up in a table that gives a vector of several hundred or several thousand numbers. These vectors are learned during training, and tokens used in similar ways end up with similar vectors.
3. Positional information. Attention, the next step, has no built-in sense of order, so information about each token's position is added. Without it, "dog bites man" and "man bites dog" would look the same.
4. Self-attention. For each token, the model calculates how relevant every other token is to it, and builds a new vector for that token as a weighted mix of the others. In "the server crashed because it ran out of memory", attention is what lets the vector for "it" draw mostly on "server". The relevance scores are dot products, turned into weights by a softmax. Multi-head attention runs several of these in parallel, each free to track a different kind of relationship.
5. Feed-forward layer. Each token's vector then passes through a small ordinary network, such as you built above.
6. Repeat. Attention plus feed-forward is one block. Large models stack dozens of blocks, with residual connections and normalisation to keep training stable.
7. Output. A final layer turns the last vector into a score for every token in the vocabulary, and a softmax makes those scores probabilities. A language model is a classifier that predicts the next token.

Two consequences are worth holding on to. First, a transformer processes all the tokens of its input in parallel during training, unlike the older recurrent networks that read one token at a time. That is what made it practical to train on enormous amounts of text. Second, because every token attends to every other token, the cost of attention grows roughly with the square of the sequence length. That is the root of the context window limit, and of the way cost and latency rise as prompts get longer, both of which the next stage examines.

Fine-tuning a pretrained model

Training a large model from scratch requires vast data and computation, and almost nobody does it. Transfer learning is the normal route. A pretrained model has already learned general features from an enormous dataset: what edges and textures look like, or how language works. You adapt it to your task with a comparatively tiny dataset, because most of what it needs to know is already there.

import torch
from torch import nn

# `backbone` is any pretrained model whose final layer has been removed,
# returning a feature vector of size `feature_dim` for each input.

for param in backbone.parameters():
    param.requires_grad = False                # freeze: these weights will not change

head = nn.Linear(feature_dim, n_classes)       # new, randomly initialised, trainable
model = nn.Sequential(backbone, head)

# pass ONLY the trainable parameters to the optimiser
optimizer = torch.optim.AdamW(head.parameters(), lr=1e-3)

# Later, to fine-tune everything: unfreeze, and use a much smaller learning rate
for param in backbone.parameters():
    param.requires_grad = True
optimizer = torch.optim.AdamW(model.parameters(), lr=2e-5)

• Use a small learning rate when fine-tuning pretrained weights, often a hundred times smaller than for training from scratch. Large updates destroy what the model already knows, which is called catastrophic forgetting.
• Freeze first, then unfreeze. Train the new head by itself until it is sensible, then unfreeze the rest. Otherwise the random head sends large, meaningless gradients back through the good weights.
• Preprocess exactly as the pretrained model expects: the same tokeniser, the same image size and normalisation. A mismatch degrades results silently.
• LoRA (low-rank adaptation) keeps the original weight matrices frozen and learns a pair of small matrices alongside each one, whose product is added to it. Only a tiny fraction of the parameters is trained, so memory needs drop sharply and the result is a small adapter file instead of a full copy of the model. It is the most widely used member of the family called parameter-efficient fine-tuning (PEFT).
• Evaluate against the simple alternatives. Fine-tuning a language model is rarely the right first step. Prompting and retrieval are cheaper, faster to iterate on, and need no retraining when the base model improves. Fine-tune when you need a consistent style or format, a specialised skill, or a smaller and cheaper model that matches a larger one on a narrow task.