chapter 01 / ml-foundations · estimated time 90-120 min

Introduction to Machine Learning
and
Core Concepts

What Is Machine Learning: A Different Way to Write Programs

Traditional programming works like this: a human understands the problem, writes the solution as rules, and the computer executes those rules.

rules + data  ──▶  computer  ──▶  answers

But there is a class of problems where humans themselves can't articulate the rules. You can recognize that a photo contains a cat, but you can't write down the "if-else rules for cat-ness" — Pointy ears? What about Scottish folds? Has fur? What about hairless cats? The rules explode. Machine learning flips the process around:

data + answers  ──▶  computer  ──▶  rules (the model)

We don't tell the computer "how to do it." Instead, we give it lots of "input → correct output" examples and let it find for itself the function that maps inputs to outputs. That is the essence of machine learning: automatically finding a function from data.

A more rigorous definition comes from Tom Mitchell (1997):

Why is the phrase "finding a function" literally accurate?

Because every ML model is ultimately a mathematical function $f$: image classification is $f: \mathbb{R}^{224\times224\times3} \to \{1..1000\}$ (pixels to class); ChatGPT is $f: \text{sequence of preceding tokens} \to \text{probability distribution over the next token}$. "Training" means searching a vast space of functions, using data as clues, to find the best $f$. Every chapter that follows — from linear regression to GPT — is just a continual upgrade of two things: "what the function looks like" and "how we search for it."

The Three Learning Paradigms

Supervised Learning

The dataset consists of pairs $(x, y)$: each input $x$ comes with a human-annotated correct answer $y$ (the label). The model learns the mapping $f(x) \approx y$. Depending on the type of $y$, there are two cases:

• Regression: $y$ is a continuous value. Predicting house prices, temperature, stock prices.
• Classification: $y$ is a discrete category. Spam/not spam, cat/dog, benign/malignant.

Unsupervised Learning

You only have $x$, with no labels. The model discovers the structure inside the data on its own:

• Clustering: group similar samples together (user segmentation, news clustering). Representative algorithm: K-Means.
• Dimensionality reduction: compress high-dimensional data into low dimensions while preserving information (PCA, t-SNE, autoencoders).
• Density estimation / generation: learn the data distribution $p(x)$, then sample from it to generate new data — the foundation of diffusion models (Stable Diffusion).

Reinforcement Learning

There are no ready-made labels, but there is a reward signal. An agent acts within an environment, the environment feeds back rewards, and the goal is to learn a policy that maximizes long-term cumulative reward. AlphaGo, robot control, and — pay attention here — the core technique behind RLHF for training ChatGPT and behind training reasoning models (OpenAI's o-series, DeepSeek-R1), covered in detail in Chapter 8.

Paradigm	Data form	What is learned	Typical examples
Supervised learning	pairs $(x, y)$	mapping $f(x)\to y$	image classification, house-price prediction
Unsupervised learning	only $x$	structure/distribution of the data	clustering, PCA, generative models
Reinforcement learning	states, actions, rewards	optimal policy $\pi(a|s)$	AlphaGo, RLHF

The Language of Machine Learning: Data, Model, Loss

Let's settle the terminology first; all eleven chapters that follow rely on it:

• Sample: one piece of data, written $x^{(i)}$. The superscript $(i)$ denotes the $i$-th item.
• Feature: a number describing a sample. A house = [area, age, distance to subway] is 3 features, $x \in \mathbb{R}^3$, dimension $d=3$.
• Label: the target to predict, $y^{(i)}$.
• Dataset: $\mathcal{D} = \{(x^{(1)}, y^{(1)}), \dots, (x^{(n)}, y^{(n)})\}$, with $n$ samples in total.
• Model / hypothesis: a parameterized function $f_\theta(x)$. $\theta$ holds the parameters — a few numbers in linear regression, over a trillion numbers in GPT-4, but the concept is exactly the same.
• Hypothesis space: the set of all functions you can obtain as $\theta$ ranges over every possible value. Choosing a model = choosing a hypothesis space; training = searching that space for the optimal $\theta$.
• Loss function: $L(\theta)$, a function that quantifies "how bad the model's predictions are." Training = adjusting $\theta$ to minimize $L$.

Before we start deriving, watch a video that explains the "intuition of learning" better than anything else. It is nominally about neural networks, but the imagery of "what the network is learning, what the parameters are" will give you an intuitive anchor for all the math to come:

Linear Regression: Our First Complete Derivation

Now let's walk through the "three pieces" completely using the simplest possible model. Don't skip it just because it's simple — concepts like gradient descent, the loss surface, and the learning rate are seen most clearly here, and by the time you reach the Transformer in Chapter 6, the mathematical skeleton is still this same one.

Model

Assume the output is a linear function of the input. For a single feature:

$$\hat{y} = f_\theta(x) = wx + b$$

$w$ (weight/slope) and $b$ (bias/intercept) are the parameters, $\theta = (w, b)$. The hat $\hat{y}$ denotes the "predicted value," distinct from the true value $y$. With multiple features, we write it as a vector inner product $\hat{y} = \mathbf{w}^\top \mathbf{x} + b$.

Loss: Mean Squared Error (MSE)

For each sample, the error is $\hat{y}^{(i)} - y^{(i)}$. Square all the sample errors and take the average:

$$L(w, b) = \frac{1}{n} \sum_{i=1}^{n} \left( wx^{(i)} + b - y^{(i)} \right)^2$$

Why squared, rather than absolute value or fourth power?

Three reasons, each deeper than the last:
① Differentiability: $|e|$ is not differentiable at 0, whereas the square is smooth and differentiable everywhere, which makes computing gradients convenient.
② Penalty structure: the square's penalty for large errors grows quadratically — twice the error, four times the penalty. A fourth power penalizes too aggressively and gets completely hijacked by outliers (in the Playground below you can add two outliers and see with your own eyes just how scared squared loss already is of outliers).
③ Probabilistic interpretation (the deepest): if you assume the noise follows a Gaussian distribution $y = wx + b + \varepsilon,\ \varepsilon \sim \mathcal{N}(0, \sigma^2)$ and do maximum likelihood estimation on the parameters, write out the log-likelihood and negate it, what remains is exactly the MSE. In other words, "minimizing MSE = maximum likelihood under an assumption of Gaussian noise." The same recipe (assume a distribution → maximum likelihood → loss function) will derive cross-entropy in Chapter 2 and the pretraining objective of LLMs in Chapter 7. Loss functions are never picked at random.

The Analytical Solution: The Normal Equation

$L(w,b)$ is a quadratic function of the parameters — a bowl-shaped surface with a unique lowest point. Calculus tells us: at the lowest point, the partial derivatives are zero. Solve directly:

$$\frac{\partial L}{\partial w} = \frac{2}{n}\sum_{i=1}^n \left(wx^{(i)} + b - y^{(i)}\right)x^{(i)} = 0$$ $$\frac{\partial L}{\partial b} = \frac{2}{n}\sum_{i=1}^n \left(wx^{(i)} + b - y^{(i)}\right) = 0$$

Two equations, two unknowns; solving (derivation: the second equation gives $b = \bar{y} - w\bar{x}$, substitute into the first and simplify):

$$w^* = \frac{\sum_i (x^{(i)} - \bar{x})(y^{(i)} - \bar{y})}{\sum_i (x^{(i)} - \bar{x})^2} = \frac{\text{Cov}(x, y)}{\text{Var}(x)}, \qquad b^* = \bar{y} - w^*\bar{x}$$

The multi-feature case is more elegant in matrix form. Stack all samples into a matrix $X \in \mathbb{R}^{n \times d}$ (one sample per row, with a column of all ones appended to absorb $b$), and stack the labels into $\mathbf{y} \in \mathbb{R}^n$, so $L(\mathbf{w}) = \frac{1}{n}\|X\mathbf{w} - \mathbf{y}\|^2$. Take the gradient with respect to $\mathbf{w}$ and set it to zero:

$$\nabla_\mathbf{w} L = \frac{2}{n} X^\top (X\mathbf{w} - \mathbf{y}) = 0 \;\;\Longrightarrow\;\; \boxed{\mathbf{w}^* = (X^\top X)^{-1} X^\top \mathbf{y}}$$

This is the famous Normal Equation.

Gradient Descent: The Engine of All Deep Learning

Take a different approach: instead of solving the equations directly, start from a random position and walk downhill one small step at a time until you reach the bottom of the valley.

The gradient $\nabla_\theta L$ is a vector pointing in the direction in which the loss increases fastest. So walk in the opposite direction:

$$\theta_{t+1} = \theta_t - \eta \, \nabla_\theta L(\theta_t)$$

$\eta$ is the learning rate — how big a step to take each time. For our linear regression, just reuse the partial derivatives we computed in the last section, and each update step becomes:

$$w \leftarrow w - \eta \cdot \frac{2}{n}\sum_i (\hat{y}^{(i)} - y^{(i)})\, x^{(i)}, \qquad b \leftarrow b - \eta \cdot \frac{2}{n}\sum_i (\hat{y}^{(i)} - y^{(i)})$$

Three Ways to Feed the Data

Variant	Data used per step	Characteristics
Batch gradient descent (BGD)	all $n$ samples	accurate gradient, stable direction, but expensive per step; infeasible when data is large
Stochastic gradient descent (SGD)	1 sample	extremely fast per step but the direction jitters a lot; the noise actually helps escape local pits
Mini-batch	$B$ samples (e.g. 32~512)	the practical standard. GPU-parallel friendly, with moderate noise. LLM training batches can reach millions of tokens

Learning Rate: The Most Important Hyperparameter

• $\eta$ too small: convergence is extremely slow, wasting compute.
• $\eta$ too large: each step overshoots the bottom and lands higher up on the opposite side; the loss rises instead of falling — it diverges.
• For quadratic loss it can be proven that convergence requires $\eta < 2/\lambda_{\max}$ (where $\lambda_{\max}$ is the largest eigenvalue of the Hessian). In practice no one computes this — people use learning-rate schedules (warmup + cosine decay, covered in the LLM training details of Chapter 7) and adaptive optimizers (Adam, derived in Chapter 3).

Don't just read the words — go to the lab below, drag the learning-rate slider all the way to the right, and watch divergence happen with your own eyes. That image is more unforgettable than any formula.

First watch two videos to cement this. The first uses "going downhill" to explain gradient descent thoroughly; the second is StatQuest, breaking down every arithmetic step, perfect if you want to follow along and compute by hand:

Interactive Lab: Train a Model with Your Own Hands

In SGD mode, why does the loss curve keep jittering at a low level and never decrease further?

SGD looks at only one random sample per step, so the gradient is an unbiased but noisy estimate of the true gradient. Near the optimum, the true gradient approaches 0, but the noise of the single-sample gradient is still there, so the parameters perform a random walk around the optimum and the loss oscillates on a "noise floor." The fix is learning-rate decay: the noise magnitude is proportional to $\eta$, so gradually shrinking $\eta$ lets it converge deeper. This is precisely one of the reasons LLM training uses cosine-decay schedules.

Generalization: The Real Challenge of Machine Learning

So far we've only made the loss small on the training data. But the real purpose of training is for the model to perform well on data it has never seen — this is called generalization. This is the essential distinction between ML and plain optimization.

• Underfitting: the model is too simple to even fit the training data well. Fitting parabolic data with a straight line.
• Overfitting: the model is too complex and memorizes even the noise in the training data. The training error approaches 0, but it's a disaster on new data. A 20th-degree polynomial threaded through every single point.

The Discipline of Splitting Data

Set	Purpose	Typical proportion
Training set	update parameters $\theta$	~80%
Validation set	tune hyperparameters (learning rate, model size), early stopping	~10%
Test set	use only once, at the very end, to report final performance	~10%

Why separate validation and test? Because you'll repeatedly tune hyperparameters based on validation performance — tune too much, and the hyperparameters "secretly fit" the validation set, inflating the validation score. The test set is a "judge" that never participated in any decision. When data is scarce, use K-fold cross-validation: split into K folds, take turns holding one out for validation, and average. In the LLM era this same discipline persists, becoming the problem of "benchmark contamination" — once benchmark questions leak into the pretraining data, the scores become untrustworthy.

The Bias-Variance Decomposition

Underfitting/overfitting can be written as precise mathematics. Suppose the true relationship is $y = g(x) + \varepsilon$, with noise $\varepsilon$ of mean 0 and variance $\sigma^2$. At a fixed point $x$, taking the expectation over "models $\hat{f}$ trained on different training sets," the expected squared error decomposes exactly into three terms (derivation: add and subtract $\mathbb{E}[\hat{f}(x)]$ then expand the square; the cross term has expectation zero):

$$\mathbb{E}\big[(y - \hat{f}(x))^2\big] = \underbrace{\big(g(x) - \mathbb{E}[\hat{f}(x)]\big)^2}_{\text{bias}^2} + \underbrace{\mathbb{E}\big[(\hat{f}(x) - \mathbb{E}[\hat{f}(x)])^2\big]}_{\text{variance}} + \underbrace{\sigma^2}_{\text{irreducible error}}$$

• Bias: how far the model is from the truth on average — the systematic error caused by the model being too simple. Underfitting = high bias.
• Variance: how much the model changes when you swap in a different batch of training data — its sensitivity to the randomness of the data. Overfitting = high variance.
• Irreducible error: the noise inherent in the data itself, which even a divine model can't eliminate.

Hands-On Code: A From-Scratch NumPy Implementation

Translate all the math of this chapter into 30 lines of code. Read it line by line; each line corresponds to a formula from earlier:

python · linear_regression_from_scratch.py

# Linear regression from scratch: normal equation vs gradient descent
import numpy as np

rng = np.random.default_rng(42)

# ---- Make data: y = 2.5x + 1.0 + Gaussian noise ----
n = 200
x = rng.uniform(0, 10, size=n)
y = 2.5 * x + 1.0 + rng.normal(0, 1.5, size=n)

# ---- Method 1: normal equation  w* = (XᵀX)⁻¹Xᵀy ----
X = np.column_stack([x, np.ones(n)])        # append a column of 1s to absorb the bias b
w_closed = np.linalg.solve(X.T @ X, X.T @ y) # more stable than explicit inversion
print(f"Analytical   w={w_closed[0]:.4f}, b={w_closed[1]:.4f}")

# ---- Method 2: batch gradient descent ----
w, b = 0.0, 0.0          # random starting point (here we use 0)
eta = 0.01               # learning rate
for step in range(2000):
    y_hat = w * x + b                 # forward: prediction
    err = y_hat - y                   # error vector
    grad_w = 2 * np.mean(err * x)     # ∂L/∂w = (2/n)Σ err·x  "error × input"
    grad_b = 2 * np.mean(err)         # ∂L/∂b = (2/n)Σ err
    w -= eta * grad_w                 # take a step along the negative gradient
    b -= eta * grad_b
    if step % 500 == 0:
        print(f"step {step:4d}  MSE={np.mean(err**2):.4f}  w={w:.3f} b={b:.3f}")

print(f"Grad descent w={w:.4f}, b={b:.4f}   # matches the analytical solution ✓")

In production it's a one-liner (but now you know what happens behind that one line):

python · sklearn version

from sklearn.linear_model import LinearRegression
model = LinearRegression().fit(x.reshape(-1, 1), y)
print(model.coef_[0], model.intercept_)   # ≈ 2.5, 1.0

Why does the code use np.linalg.solve instead of np.linalg.inv followed by a multiplication?

Numerical stability. inv explicitly inverts and then multiplies, and the error gets amplified by the condition number twice; solve uses LU decomposition to solve the linear system directly, which is faster and more stable. When features are highly correlated (multicollinearity), $X^\top X$ is nearly singular, and this difference becomes a chasm. Engineering details are part of technical skill too.

Finally, take a glimpse of where all this leads. This talk by Karpathy (a former founding member of OpenAI and former Director of AI at Tesla) is the best bridge from "this chapter to LLMs" — you'll find that the "training is compression" and "next-word prediction" he describes have exactly the skeleton you just finished learning: model + loss + optimization:

Chapter Quiz