Mathematical Foundations

V.1 · Statistics

How much can one observation move the center?

Move a real observation and compare the mean, median and frequency counts. Ordered values, plotted observations and summaries share one exact dataset.

Move observations, follow the summaries

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

Select an observation below. Drag its handle or use Left/Right arrows; Shift takes a larger step. Numeric entry accepts two decimal places.

Ordered: 1, 2, 2, 3, 7

Single mode: 2.

Counts of actual observations
Counts of actual observationsOrdered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.31: approximately 12: approximately 23: approximately 17: approximately 1048

Observed values · counts · dashed marker is the observed mean

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

Observed mean
3
Observed median
2
Population variance of these observations
4.4
Usual sample variance
5.5
Population standard deviation
√110/52.097618…
Median absolute deviation (unscaled)
1

= exact finite-data/model quantity · ≈ binary64 approximation; empirical results describe this run

from −20 to 20, up to 2 decimal places

Mean = 3; median = 2. Single mode: 2.

xˉ=15∑ixi=3\bar x=\frac{1}{5}\sum_i x_i=3
vn=∑i(xi−xˉ)2n=4.4v_n=\frac{\sum_i(x_i-\bar x)^2}{n}=4.4
s2=∑i(xi−xˉ)2n−1=5.5s^2=\frac{\sum_i(x_i-\bar x)^2}{n-1}=5.5

Order actual observations

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

n=5n=5

Move a real observation and compare the mean, median and frequency counts. Ordered values, plotted observations and summaries share one exact dataset.

Intuition

The mean is a balance point: every observation contributes to the sum. The median asks where the ordered middle lies. A mode asks which observed value occurs most often. They answer different questions, so disagreement can be useful information. Try moving the selected 7 to 20 before reading on.

Definition

For a nonempty dataset x1,…,xnx_1,\ldots,x_n, the mean is xˉ=n−1∑ixi\bar x=n^{-1}\sum_i x_i. Order the observations as x(1)≤⋯≤x(n)x_{(1)}\leq\cdots\leq x_{(n)}. For odd nn, the median is x((n+1)/2)x_{((n+1)/2)}; for even nn, average x(n/2)x_{(n/2)} and x(n/2+1)x_{(n/2+1)}. Modes maximize observed frequency.

Mechanics

Select an observation by its numbered button, then drag its number-line handle, use arrow keys, or enter a value. Add or remove observations and watch the ordered list and histogram update. Values may repeat: the frequency view keeps counts while the number line can show coincident points. Reset restores [1,2,2,3,7].

Derivation

The mean uniquely minimizes ∑i(xi−a)2\sum_i(x_i-a)^2: expanding around xˉ\bar x gives ∑i(xi−a)2=∑i(xi−xˉ)2+n(a−xˉ)2\sum_i(x_i-a)^2=\sum_i(x_i-\bar x)^2+n(a-\bar x)^2. By contrast, a median minimizes the sum of absolute distances; for an even dataset a whole interval can minimize that objective, while our reported convention averages the middle pair.

Worked examples

For [1,2,2,3,7][1,2,2,3,7], the sum is 15, so the mean is 3. The middle observation is 2 and its frequency is 2: median and single mode are both 2. Replacing 7 by 20 gives mean 28/528/5 and the same median 2. For [1,1,3,3][1,1,3,3], both 1 and 3 are modes.

Worked example: Move observations, follow the summaries

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

Select an observation below. Drag its handle or use Left/Right arrows; Shift takes a larger step. Numeric entry accepts two decimal places.

Ordered: 1, 2, 2, 3, 7

Single mode: 2.

Counts of actual observations
Counts of actual observationsOrdered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.31: approximately 12: approximately 23: approximately 17: approximately 1048

Observed values · counts · dashed marker is the observed mean

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

Observed mean
3
Observed median
2
Population variance of these observations
4.4
Usual sample variance
5.5
Population standard deviation
√110/52.097618…
Median absolute deviation (unscaled)
1

= exact finite-data/model quantity · ≈ binary64 approximation; empirical results describe this run

from −20 to 20, up to 2 decimal places

Mean = 3; median = 2. Single mode: 2.

xˉ=15∑ixi=3\bar x=\frac{1}{5}\sum_i x_i=3
vn=∑i(xi−xˉ)2n=4.4v_n=\frac{\sum_i(x_i-\bar x)^2}{n}=4.4
s2=∑i(xi−xˉ)2n−1=5.5s^2=\frac{\sum_i(x_i-\bar x)^2}{n-1}=5.5

Order actual observations

Ordered observed values: 1, 2, 2, 3, 7. Single mode: 2. Mean = 3; median = 2. Single mode: 2.

n=5n=5

Replay the worked mathematical steps using the same canonical state and actual derived quantities.

Executable reference code

Run both files. The pedagogical program exposes the arithmetic; the NumPy companion uses array operations. Literal assertions check the worked example rather than testing a simulation against a desired statistical outcome.

code/python/statistics_summaries.py
"""Runnable summaries reference; exact mechanics / NumPy companion."""
from fractions import Fraction as F
from collections import Counter
x = [F(v) for v in [1,2,2,3,7]]
a = sorted(x)
mean = sum(x)/len(x)
median = a[len(a)//2]
counts = Counter(x)
modes = [v for v,c in counts.items() if c == max(counts.values())]
assert (mean,median,modes)==(F(3),F(2),[F(2)])
print(mean, median, modes)
code/python/statistics_summaries_numpy.py
"""Runnable summaries reference; exact mechanics / NumPy companion."""
import numpy as np
x=np.array([1,2,2,3,7],dtype=float)
values, counts=np.unique(x,return_counts=True)
modes=values[counts==counts.max()]
assert x.mean()==3 and np.median(x)==2 and modes.tolist()==[2]
print(x.mean(),np.median(x),modes)

Explore

Compare the outlier, singleton, multiple-mode and distinct-value presets. Predict the result before moving a point. Find two different datasets with the same mean but different medians; then explain why one number cannot reconstruct the observations.

Common mistakes and counterexamples

Do not choose a tied mode arbitrarily. When every distinct value occurs once, every value ties for maximal frequency; this explorer reports no repeated value and no uniquely determined mode. Some textbooks call every tied value a mode. This declared convention avoids implying a preferred value. A singleton has its own value as all three summaries.

Numerical considerations

Inputs contain at most two decimal places and lie between −20 and 20; datasets have 1–24 observations. Sorting, sums, medians and frequency equality use exact rational arithmetic. Rendering converts these exact values to binary64 coordinates. Rounded inspector companions use ≈, while observed fractions use =. The plot reports data values, not unknown population parameters.

Engineering and ML connections

Latency, sensor readings and feature distributions often have a long right tail. A mean can reveal total load, while a median describes a typical ordered observation. A mode is useful for discrete categories but depends on bin choices for continuous data; this chapter uses actual repeated values rather than inventing histogram modes.

Exercises

Predict, calculate and explain before checking the corrective feedback. Each solution states the relevant rule and its conditions.

calculation

Centers

For observations [1,2,2,3,7], give the mean, median and single mode.

calculation

Outlier effect

Replace 7 by 20 in [1,2,2,3,7]. Give the new mean and median.

calculation

Even median

For [1,2,5,8], give the median.

explanation

Tied modes

Which modes describe [1,1,3,3,5]?

Choose one answer

explanation

Distinct observations

For [1,2,3], what should this explorer report?

Choose one answer

Summary and checkpoints

Check that you can order observations, distinguish three definitions, calculate an even median, retain mode ties and explain an outlier. A summary compresses observations; always retain enough of the dataset to inspect what it hides.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available