Mathematical Foundations

II.12 · Linear Algebra

One signed number records area scaling, collapse, and orientation.

Drag the image columns and watch the unit square become a parallelogram. Its ordinary area is |det A|; the sign remembers whether the ordered frame was reversed.

Watch a unit square become signed area

The unit square becomes a parallelogram of area 2. The positive sign records preserved orientation.

matrix A
2×2 [2, 1; 0, 1]
signed area scale
2
ordinary area scale
2
orientation
preserved
basis independence test
independent

Exact expressions use =. Rounded computations use ≈. Every view uses this same state.

The unit square becomes a parallelogram of area 2. The positive sign records preserved orientation.

det⁡A=a11a22−a12a21\det A=a_{11}a_{22}-a_{12}a_{21}
det⁡A=2(1)−1(0)=2\det A=2(1)-1(0)=2
area=∣det⁡A∣=2\text{area}=|\det A|=2

from −4 to 4, up to 2 decimal places

from −4 to 4, up to 2 decimal places

from −4 to 4, up to 2 decimal places

from −4 to 4, up to 2 decimal places

Drag either matrix column. The transformed unit square, exact determinant, unsigned area, orientation, and independence status all update from one state.

Choose Area-one shear and move the slanted column horizontally; explain why base and height stay fixed. Choose Negative reflection and locate the orientation reversal. Finish with Zero collapse, then drag one column off the shared line.

Intuition: measure the image of one standard tile

The unit square has sides e1,e2\mathbf e_1,\mathbf e_2 and area one. A matrix sends those sides to its two columns, so the square becomes the parallelogram spanned by those columns. Its area tells how every planar area scales under the transformation.

Ordinary area cannot be negative, but geometry also has an ordered orientation. The determinant combines both ideas in one signed number.

Definition

For

A=[abcd],A=\begin{bmatrix}a&b\\c&d\end{bmatrix},

the determinant is

det⁡A=ad−bc.\det A=ad-bc.

Its magnitude ∣det⁡A∣|\det A| is the area scale. A positive sign preserves the orientation of the ordered standard basis; a negative sign reverses it. A zero determinant means the image has zero planar area.

Mechanics: read magnitude, sign, and zero separately

Three questions prevent common confusion:

  1. Is det⁡A\det A zero? Then the columns are dependent and the plane collapses to fewer dimensions.
  2. If nonzero, what is ∣det⁡A∣|\det A|? This is the area multiplier.
  3. What is its sign? This records whether orientation is preserved or reversed.

A determinant of −3-3 scales ordinary area by 33 and reverses orientation. A determinant of 0.20.2 preserves orientation while shrinking area.

Derivation: signed parallelogram area

For columns a1=(a,c)\mathbf a_1=(a,c) and a2=(b,d)\mathbf a_2=(b,d), the signed area is

ad−bc.a d-b c.

One way to see the subtraction is to place the parallelogram inside a surrounding rectangle and remove the corner triangles. Algebraically, the expression is linear in either column while the other is fixed, changes sign when columns swap, and equals one on the standard basis. Those properties exactly match signed area.

Zero determinant and basis independence

If the columns are collinear, the parallelogram has no height and hence no area. Conversely, nonzero area requires two distinct directions. Therefore, in two dimensions,

det⁡A≠0⟺the columns are linearly independent.\det A\ne0 \quad\Longleftrightarrow\quad \text{the columns are linearly independent}.

This completes the geometric interpretation previewed in II.7. It does not yet introduce general rank or inverse algorithms.

Reflections and singular maps

A reflection has negative determinant because it reverses the ordered frame. A projection of the plane onto a line has determinant zero because it destroys one independent direction. A shear can have determinant one even while changing lengths and angles: area preservation does not imply orthogonality.

Worked examples

For

A=[2101],A=\begin{bmatrix}2&1\\0&1\end{bmatrix},

the determinant is 22. The transformed unit square is a parallelogram of area 22 with preserved orientation. Swapping its columns gives determinant −2-2 and the same ordinary area.

Inspect the worked calculation

Transform the two sides of the unit square.

det⁡A=a11a22−a12a21\det A=a_{11}a_{22}-a_{12}a_{21}
matrix A
2×2 [2, 1; 0, 1]
signed area scale
2
ordinary area scale
2
orientation
preserved
basis independence test
independent
Transform the unit square, compute signed area, then separate magnitude from orientation.

For [1224]\begin{bmatrix}1&2\\2&4\end{bmatrix}, the determinant is zero. The second column is twice the first, so the square collapses onto a line—exactly the dependence witnessed in II.7.

Higher-dimensional meaning

In higher dimensions, the determinant’s magnitude scales oriented volume rather than planar area. A zero determinant still signals collapse into a lower-dimensional set, and a negative sign still records an orientation reversal. This chapter keeps the derivation and interactive geometry in two dimensions; later linear algebra can reuse the interpretation without pretending a 3D or higher-dimensional object fits this plane.

Executable reference code

The Python reference computes the exact 2D determinant and verifies the parallelogram area by the shoelace formula. The NumPy version compares signed determinant values for scaling, reflection, shear, and singular examples.

code/python/matrix_determinant.py
from fractions import Fraction as F


def det2(a):
    return a[0][0] * a[1][1] - a[0][1] * a[1][0]


assert det2(((F(2), F(1)), (F(0), F(3)))) == 6
assert det2(((F(0), F(1)), (F(1), F(0)))) == -1
assert det2(((F(1), F(2)), (F(2), F(4)))) == 0
code/python/matrix_determinant_numpy.py
import numpy as np

A = np.array([[2.0, 1.0], [0.0, 3.0]])
reflection = np.array([[0.0, 1.0], [1.0, 0.0]])
singular = np.array([[1.0, 2.0], [2.0, 4.0]])

assert np.isclose(np.linalg.det(A), 6.0)
assert np.isclose(np.linalg.det(reflection), -1.0)
assert np.isclose(np.linalg.det(singular), 0.0)

Explore

Hold the first column fixed and slide the second parallel to it; identify the zero crossing. Reverse one column and watch only the sign change. Double one column and confirm that the determinant doubles. Add a multiple of the first column to the second and observe that the determinant stays fixed.

Common mistakes

The determinant is not the sum or product of all entries. Its sign is not ordinary area, and a negative determinant does not mean “negative size.” Determinant one does not imply a rigid motion. Zero determinant does not mean the matrix is the zero matrix; it means its columns fail to provide all independent directions.

Numerical considerations

The lesson classifies zero exactly for entered decimal rationals. Floating-point determinants near zero can be poor standalone singularity tests because scale and conditioning matter. Numerical libraries use factorizations and problem-aware tolerances. The exact sign here describes the entered matrix, while measured coefficients may carry uncertainty beyond arithmetic roundoff.

Engineering and ML connections

Determinants track volume change in coordinate transforms, density corrections, and invertible models. The absolute value controls scale; the sign carries orientation. Direct determinant computation is rarely the best numerical way to solve a system, and a small determinant alone does not summarize all conditioning behavior.

Exercises

calculation

Compute signed area

For A = [[2, 1], [0, 3]], compute det A.

calculation

Separate sign and area

A 2D matrix has det A = −4. What is the area of the image of the unit square?

prediction

Read orientation

What does det A < 0 say when det A is nonzero?

Choose one answer

calculation

Connect to independence

Columns are (1, 2) and (2, 4). Compute their determinant.

prediction

Predict an area-one shear

For A = [[1, k], [0, 1]], what is det A?

Choose one answer

Summary and checkpoints

In two dimensions, det⁡A\det A is signed area scale. Check that you can separate zero, magnitude, and sign; connect zero to dependent columns; distinguish reflection from collapse; and explain the higher-dimensional volume interpretation without relying on a misleading 2D picture.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available