How much probability-weighted spread remains after removing the center?

Predict which value contributes the largest weighted squared distance. Shift the distribution and decide which table columns should change and which should stay fixed.

Intuition: location does not specify spread

Two distributions can share a mean and have different spreads. A constant at $2$ and a fair choice between $0$ and $4$ both have mean $2$, but only the latter varies. We need a quantity that measures distance from the center rather than distance from zero.

Signed distances average to zero: negative deviations cancel positive ones. Squaring makes every distance contribution nonnegative and gives larger deviations greater influence. Weighting those squares by probability produces the variance; taking its nonnegative square root returns a spread in the original units.

Definition

Write $\mu=\mathbb E[X]$. The variance and standard deviation are

Variance is nonnegative. In a finite model it is zero precisely when every positive-mass value equals the mean; arbitrary zero-mass labels do not change it. If $X$ has units of milliseconds, variance has units of milliseconds squared, while standard deviation has units of milliseconds.

Finite models have finite moments. More generally, finite second moment guarantees a finite first moment and variance; distributions without a finite second moment require a separate discussion of infinite or undefined quantities.

Mechanics: compare distances on one scale

The canonical values are $[-2,0,4]$ with raw weights $[1,2,1]$, affine scale $1$, and shift $0$. Read the table’s mass, signed distance from the mean, and weighted squared contribution. The square drawings compare squared distances on one common length scale; the table shows how probability weights alter their contributions.

Use Shift by 2 and Scale by −2 to inspect a companion affine variable in the transformed expectation and variance rows. The plot, squares, and contribution table describe the original $X$, enabling comparison with the transformation. Zero variance changes the source values themselves to $2$. Reset restores the canonical model. Negative weights or an all-zero total are invalid, whereas zero spread is a valid state.

Derivation: the second-moment identity and affine law

Expand the centered square and use linearity:

For $Z=aX+b$, its mean is $a\mu+b$, so $Z-\mathbb E[Z]=a(X-\mu)$. Squaring gives

A shift changes location and leaves spread unchanged. A negative scale reverses the number line but cannot make variance or standard deviation negative. With scale zero, all values collapse to the shift and the spread is zero.

Worked examples

For the canonical model, $\mu=1/2$ and $\mathbb E[X^2]=5$. The signed deviations are $-5/2,-1/2,7/2$, whose squared values are $25/4,1/4,49/4$. Weighting them gives

Thus $\sigma=\sqrt{19}/2\approx2.17944947177$. The value $4$ contributes the largest weighted square, $49/16$, even though its mass is only $1/4$.

Shifting by $2$ changes the mean to $5/2$ and preserves variance $19/4$. Scaling by $-2$ gives mean $-1$, variance $19$, and standard deviation $\sqrt{19}$. Collapsing all values to $2$ gives mean $2$, second moment $4$, and variance $0$.

Executable reference code

The exact program calculates both centered weighted squares and the second-moment identity using Fraction, asserting that they agree. It verifies literal canonical values and sweeps affine scales and shifts, including collapse and reflection. The square root is reported approximately because $\sqrt{19}/2$ is irrational.

The NumPy program exposes a normalized probability vector, a center, and the dot product of centered squared distances. Its large-offset check demonstrates why centered squares are preferable to subtracting nearly equal floating-point moments. These are theoretical distribution calculations, not sample-variance estimates.

"""Exact theoretical variance by centered squares and the moment identity."""
from fractions import Fraction as F
from math import sqrt


def moments(values, weights):
    if not values or len(values) != len(weights):
        raise ValueError("Require matching nonempty values and weights")
    if any(not isinstance(x, (int, F)) or isinstance(x, bool) for x in (*values, *weights)):
        raise ValueError("Use integer or Fraction values and weights")
    total = sum(weights, F(0))
    if any(w < 0 for w in weights) or total <= 0:
        raise ValueError("Weights must be nonnegative with positive total")
    mean = sum((x*w for x, w in zip(values, weights)), F(0)) / total
    second = sum((x*x*w for x, w in zip(values, weights)), F(0)) / total
    variance = sum((w*(x-mean)**2 for x, w in zip(values, weights)), F(0)) / total
    assert variance == second - mean**2 >= 0
    return mean, second, variance


def main():
    x, w = [-2, 0, 4], [1, 2, 1]
    mean, second, variance = moments(x, w)
    assert (mean, second, variance) == (F(1, 2), F(5), F(19, 4))
    assert abs(sqrt(float(variance)) - 2.179449471770337) < 1e-14
    shifted = moments([v+2 for v in x], w)
    assert shifted[0] == F(5, 2) and shifted[2] == variance
    reflected = moments([-2*v for v in x], w)
    assert reflected[0] == -1 and reflected[2] == 19
    assert moments([2, 2, 2], w) == (F(2), F(4), F(0))
    for scale in (-2, 0, 1, 3):
        for shift in (-5, 0, 2):
            transformed = moments([scale*v + shift for v in x], w)
            assert transformed[0] == scale*mean + shift
            assert transformed[2] == scale**2 * variance
    assert moments([10**12 + v for v in x], w)[2] == variance
    assert moments([99, 2], [0, 1])[2] == 0
    for values, weights in (([], []), ([0], [0]), ([0, 1], [1, -1]), ([0, 1], [1]), ([float('nan')], [1])):
        try:
            moments(values, weights)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid exact weighted model accepted")
    print("E[X] =", mean, "E[X²] =", second, "Var(X) =", variance)
    print("σ = √19/2 ≈", sqrt(float(variance)))
    print("Shift +2: same variance; scale -2: variance 19; collapsed X=2: variance 0")


if __name__ == "__main__":
    main()

"""A centered weighted variance; raw weights describe a model, not a sample."""
import numpy as np


def moments(values, weights):
    x, w = np.asarray(values, dtype=float), np.asarray(weights, dtype=float)
    if x.ndim != 1 or x.size == 0 or w.shape != x.shape or not np.all(np.isfinite(x)):
        raise ValueError("Require finite nonempty values and matching weights")
    if not np.all(np.isfinite(w)) or np.any(w < 0) or not np.isfinite(w.sum()) or w.sum() <= 0:
        raise ValueError("Require nonnegative finite weights with positive finite total")
    p = w / w.sum()
    mean = np.dot(p, x)
    second = np.dot(p, x*x)
    centered = x - mean
    variance = np.dot(p, centered*centered)
    return float(mean), float(second), float(variance), float(np.sqrt(variance))


def main():
    x, w = np.array([-2., 0., 4.]), np.array([1., 2., 1.])
    mean, second, variance, sigma = moments(x, w)
    np.testing.assert_allclose([mean, second, variance, sigma], [0.5, 5, 4.75, 2.179449471770337], rtol=0, atol=1e-12)
    assert moments(x+2, w)[0] == 2.5 and moments(x+2, w)[2] == 4.75
    assert moments(-2*x, w)[2] == 19
    assert moments(np.full(3, 2.), w) == (2., 4., 0., 0.)
    for scale in (-2, 0, 1, 3):
        for shift in (-5, 0, 2):
            transformed = moments(scale*x + shift, w)
            np.testing.assert_allclose([transformed[0], transformed[2], transformed[3]], [scale*mean + shift, scale**2*variance, abs(scale)*sigma], rtol=0, atol=1e-12)
    assert moments(10**12 + x, w)[2] == 4.75
    assert moments([99, 2], [0, 1])[2] == 0
    for values, weights in (([], []), ([0], [0]), ([0, 1], [1, -1]), ([np.nan], [1]), ([0], [np.inf])):
        try:
            moments(values, weights)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid weighted model accepted")
    print("Weighted theoretical moments ≈", (mean, second, variance, sigma))
    print("For an observed sample only, np.var(sample, ddof=1) uses m-1 for m>1.")
    print("Centered squares avoid subtracting two large, nearly equal moments.")


if __name__ == "__main__":
    main()

Explore

Before choosing Shift by 2, predict the transformed mean and variance. Before choosing Scale by −2, predict transformed mean $-1$ and variance $19$, then check the transformed inspector rows. On paper, reconstruct that variable’s doubled deviation magnitudes, quadrupled squared distances, and doubled standard deviation from the original table.

Choose Zero variance and explain why a nonzero mean does not imply nonzero spread. Reset and increase an outer value’s weight or move it farther from the center. Track how the mean itself moves; computing deviations from the old mean would answer a different question.

Common mistakes and counterexamples

The second moment measures squares about zero, while variance measures squares about the mean. For a constant $X=2$, $\mathbb E[X^2]=4$ but variance is zero. Adding a constant also does not add its square to the variance: centering removes the same shift from every value.

Standard deviation is the nonnegative square root, not half the variance. For the canonical model it is $\sqrt{19}/2$, not $19/2$. A negative affine scale uses $a^2$ for variance and $|a|$ for standard deviation.

Numerical considerations

The exact identity $\mathbb E[X^2]-\mu^2$ is convenient algebraically. With large offsets and small spread, floating-point subtraction can lose significant digits or produce a tiny negative result. Centered weighted squares avoid that particular subtraction. Extremely large values can still overflow floating-point squares; finite input alone does not guarantee a representable second moment.

The stage uses a specified theoretical model, so normalize its raw weights by their total. For $m>1$ independent identically distributed observed samples with an unknown mean, the usual unbiased sample variance estimator is

This correction concerns estimation from observations, not the number of support labels or the raw model weight total. With one sample, that estimator is undefined. NumPy’s default var uses ddof=0; ddof=1 selects the usual sample denominator when its assumptions apply. Even then, the square root is not generally an unbiased estimator of population standard deviation.

Engineering and ML connections

Standardizing a feature uses its center and a positive spread scale. A zero-variance feature cannot be divided by its standard deviation and needs an explicit handling rule. Measurement-unit changes explain the squared scale law: converting milliseconds to seconds rescales variance by the conversion factor squared. Estimated data statistics and theoretical model moments should be labeled separately.

Exercises

Use exact rational variance answers. The choices test square-root units and the distinction between a model and an estimator.

Summary and checkpoints

Variance is the probability-weighted squared distance from the mean; standard deviation is its nonnegative square root. Check that you can recover $19/4$ and $\sqrt{19}/2$, preserve spread under a shift, square the affine scale, explain collapse, and keep theoretical moments distinct from sample estimates.