Mathematical Foundations

Interactive Mathematical Foundations for Computing & Machine Learning · Book II

See the mathematics move.

A book where every formula has a handle. Pull on a vector, watch the numbers answer, then follow the derivation — from vectors and calculus to gradient descent and softmax.

Explore the map Start with I.1 Try the vector explorer

Try it: pull the tip of the arrow

Coordinate plane from −7 to 7 on both axes; +x points right and +y points up. The vector v runs from the origin (0, 0) to the point (3, 2). Its horizontal component is 3 and its vertical component is 2.

vector
(3, 2)
magnitude
√133.605551…
direction
0.588003 rad

= exact · ≈ rounded to 6 decimals
Drag the arrow tip and watch its coordinates, length and formulas update together. Chapter I.3, Vectors, builds on this.

One way of learning, in five moves

Every chapter follows the same order, so the mathematics always arrives with a mental picture already attached.

  1. See it

    Meet the idea as a picture you can touch before any symbol appears.

  2. Manipulate it

    Change one quantity and watch every consequence follow, in the same frame.

  3. Understand the mechanism

    Follow the same numbers through the inspector and the formula.

  4. Derive the mathematics

    Now the notation arrives, with something already in your hands to attach it to.

  5. Implement it

    Run it in Python and NumPy, checked against the browser model.

If a formula says v=(3,2)\mathbf{v}=(3,2), the arrow ends at (3,2)(3,2).

The picture, the inspector and the formula are three views of one mathematical state. Nothing on the page is a decorative approximation; if a rounded number appears, it is marked as one.

Eight Parts, in the order they depend on each other

Each emblem is drawn from real mathematics of its Part, not from a stock icon set.

Equations this book makes mechanical

By the end, each of these should read as a description of something you have already moved with your hands.

θk+1=θk−η ∇θL(θk)\theta_{k+1}=\theta_k-\eta\,\nabla_\theta L(\theta_k)

One step of learning: move the parameters a little against the gradient.

Part VI · Optimization

pi=ezi∑jezjp_i=\frac{e^{z_i}}{\sum_j e^{z_j}}

Softmax turns unconstrained scores into a probability distribution.

Part VII · Information Theory

−∑iyilog⁡pi-\sum_i y_i\log p_i

Cross entropy: how surprised a model is by the right answer.

Part VII · Information Theory

y=Ax+b\mathbf{y}=A\mathbf{x}+\mathbf{b}

A linear layer: a transformation of space, then a shift.

Part II · Linear Algebra

Your learning path

All 97 chapters are available. Begin at Part I or choose a topic to explore, then follow the worked steps and exercises.

Inside each chapter

  • Explore an interactive mathematical model
  • Understand formulas through derivations and worked steps
  • Try executable Python and NumPy examples
  • Practice with graded questions and solutions

Connect the Parts

  • Linear Algebra and Calculus → Optimization
  • Probability and Information Theory → token loss
  • Exact definitions → Numerical Computing

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available