Contents
The map of the book.
Eight Parts, in dependency order. 97 of 97 chapters are written. Every chapter is available.
Part I
Mathematical Language
The notation and mental objects everything else uses.
9 of 9 chapters written
- I.1 Scalars, quantities, units, powers and roots available
- I.2 Coordinates and points available
- I.3 Vectors: components, magnitude, and direction reference chapter available
- I.4 Algebraic expressions, equations and inequalities available
- I.5 Functions as input → transformation → output available
- I.6 Essential function families available
- I.7 Sets, intervals, and relations available
- I.8 Indexed quantities and summation available
- I.9 The notation map for ML available
Part II
Linear Algebra
Matrices as transformations: the strongest foundation in the book.
19 of 19 chapters written
- II.1 Vector addition and subtraction available
- II.2 Scalar multiplication and vector scaling available
- II.3 Norms, magnitude, and normalization available
- II.4 Dot product, angle, and orthogonality available
- II.5 Projection and distance to a direction available
- II.6 Linear combinations and span available
- II.7 Linear independence, basis, and coordinates available
- II.8 Matrices as linear transformations available
- II.9 Matrix × vector available
- II.10 Matrix × matrix as composition available
- II.11 Transpose and orthogonality available
- II.12 Determinant: orientation and area/volume scaling available
- II.13 Linear systems as intersecting constraints available
- II.14 Gaussian elimination and row operations available
- II.15 Rank and invertibility available
- II.16 Inverse matrices available
- II.17 Eigenvalues and eigenvectors available
- II.18 Orthogonal bases and least-squares geometry available
- II.19 Singular Value Decomposition available
Part III
Calculus
Change, local approximation, and sensitivity made visible.
15 of 15 chapters written
- III.1 Limits through approach available
- III.2 Continuity and local behavior available
- III.3 Derivative as secant → tangent available
- III.4 Derivative as rate of change and local linear model available
- III.5 Derivative rules available
- III.6 Chain rule available
- III.7 Multivariable functions and partial derivatives available
- III.8 Gradient and directional derivatives available
- III.9 Jacobian available
- III.10 Computational graphs and automatic differentiation available
- III.11 Hessian and curvature available
- III.12 Integrals as accumulation available
- III.13 Fundamental derivative/integral connection available
- III.14 Numerical differentiation available
- III.15 Numerical integration available
Part IV
Probability
Uncertainty through sample spaces, evidence, and repetition.
16 of 16 chapters written
- IV.1 Random experiments, outcomes, and sample spaces available
- IV.2 Events and probability mass available
- IV.3 Counting and finite probability where needed available
- IV.4 Conditional probability as restricted sample space available
- IV.5 Independence available
- IV.6 Bayes' theorem as probability-mass update available
- IV.7 Random variables available
- IV.8 PMF, PDF, and CDF available
- IV.9 Bernoulli and Binomial distributions available
- IV.10 Uniform and Gaussian distributions available
- IV.11 Expectation available
- IV.12 Variance and standard deviation available
- IV.13 Joint, marginal, and conditional distributions available
- IV.14 Covariance and correlation available
- IV.15 Law of Large Numbers available
- IV.16 Central Limit Theorem available
Part V
Statistics
From observed data to estimates, intervals, and likelihood.
10 of 10 chapters written
- V.1 Data summaries: mean, median, mode available
- V.2 Spread: variance, standard deviation, robust interpretation available
- V.3 Population, sample, and sampling variability available
- V.4 Estimators and estimation available
- V.5 Confidence intervals through repeated sampling available
- V.6 Covariance matrices and multivariate spread available
- V.7 Regression fundamentals available
- V.8 Likelihood available
- V.9 Maximum Likelihood Estimation available
- V.10 Maximum A Posteriori estimation available
Part VI
Optimization
Learning as repeated parameter updates governed by an objective.
10 of 10 chapters written
- VI.1 Objective functions and optimization landscapes available
- VI.2 Local vs global optima and stationary points available
- VI.3 Convexity available
- VI.4 Gradient descent flagship chapter available
- VI.5 Learning rate and convergence behavior available
- VI.6 Momentum available
- VI.7 Stochastic and mini-batch gradient descent available
- VI.8 Adam intuition available
- VI.9 Constraints and projected/penalized intuition available
- VI.10 Numerical stopping and optimization diagnostics available
Part VII
Information Theory
Logarithms, entropy, and the loss behind token prediction.
8 of 8 chapters written
- VII.1 Logarithms and log probabilities in computation available
- VII.2 Self-information available
- VII.3 Entropy available
- VII.4 Cross entropy and negative log-likelihood available
- VII.5 KL divergence available
- VII.6 Logits and softmax available
- VII.7 Numerical stability: log-sum-exp and stable softmax available
- VII.8 From token logits to language-model loss available
Part VIII
Numerical Computing
Why equivalent expressions behave differently on real machines.
10 of 10 chapters written
- VIII.1 Floating-point representation available
- VIII.2 Rounding and representation error available
- VIII.3 Error propagation and conditioning available
- VIII.4 Catastrophic cancellation available
- VIII.5 Overflow and underflow available
- VIII.6 Stable algorithms available
- VIII.7 Numerical linear algebra realities available
- VIII.8 Sparse matrices and sparse structure available
- VIII.9 Vectorized computation available
- VIII.10 Reproducibility, random seeds, and nondeterminism available