Where does a distribution balance when its values have unequal weights?

Predict the center before inspecting it. Compare the equal-label average with the probability-weighted average: the middle value has twice the mass of either outer value.

Intuition: a center of probability mass

Expectation is a weighted balance point, not a guarantee about the next result. Imagine each value carrying its probability mass on a number line. A larger mass contributes more strongly to the center. Negative values pull in the negative direction; they do not require negative probabilities.

For values $[-2,0,4]$ and raw weights $[1,2,1]$, the normalized masses are $[1/4,1/2,1/4]$. Their center is $1/2$, even though no outcome takes that value. The ordinary average of the three labels, $2/3$, describes a different equal-weight model.

Definition

For a finite random variable with values $x_i$ and probabilities $p_i$, its expectation is

For a function $g$, $\mathbb E[g(X)]=\sum_i p_i g(x_i)$. In particular, the second moment is $\mathbb E[X^2]=\sum_i p_i x_i^2$; it is generally different from $(\mathbb E[X])^2$.

For a continuous variable with density $f$, $\mathbb E[X]=\int_{-\infty}^{\infty}x f(x)\,dx$ when the expectation exists. A finite absolute first moment ensures this definition is well behaved; not every infinite-support distribution has one. The finite models in this stage always do.

Mechanics: normalize before summing

The stage begins with affine scale $1$ and shift $0$. Change values or nonnegative raw weights, then read each table row’s normalized mass and product $p_i x_i$. The formula, center marker, inspector, and trace use the same derived distribution. The total raw weight must be positive.

Scale and shift define a companion affine variable $aX+b$ while preserving its masses. Read its expectation and variance in the transformed inspector rows; the plot and weighted-sum table show the original $X$. The presets Zero variance, Shift by 2, and Scale by −2 expose a constant source variable or an affine comparison. Reset restores the canonical values, weights, scale, and shift.

Derivation: linearity does not need independence

For constants $a,b$,

For two jointly defined finite variables, sum over their common outcomes. Distributing the sum gives $\mathbb E[X+Y]=\mathbb E[X]+\mathbb E[Y]$. Independence is unnecessary: both values can depend on the same outcome. In general settings these identities require the relevant expectations to exist.

An event indicator $I_A$ is $1$ on $A$ and $0$ elsewhere, so $\mathbb E[I_A]=P(A)$. A binomial count is a sum of Bernoulli indicators; linearity gives mean $np$. Independence is required for its binomial distribution, but not for the expectation of a sum of indicators with given marginal success probabilities.

Worked examples

The canonical weighted calculation is

Define $Y=2X+1$ on the same outcome. This is a dependent pair, not a second independent draw. Its values are $[-3,1,9]$, with the same masses. Since $\mathbb E[Y]=2\mathbb E[X]+1=2$,

By contrast, $XY=2X^2+X$ gives $\mathbb E[XY]=21/2$, whereas $\mathbb E[X]\mathbb E[Y]=1$. Linearity concerns sums and constant multiples, not arbitrary products.

Executable reference code

The standard-library program normalizes raw weights with exact Fraction arithmetic and computes first and second moments by explicit sums. It constructs $Y$ from each value of $X$, then directly checks the dependent sum and product calculations. It also verifies weight-rescaling invariance, affine transformations, constant values, and invalid-input rejection.

The NumPy version uses normalized weight vectors and dot products. It exposes the array mechanics after the mathematical calculation, rather than treating an array routine as a definition of expectation. Neither program simulates samples to manufacture a theoretical answer.

"""Expectations from exact raw weights, including dependent linearity."""
from fractions import Fraction as F


def moments(values, weights):
    if not values or len(values) != len(weights):
        raise ValueError("Require matching nonempty values and weights")
    if any(not isinstance(x, (int, F)) or isinstance(x, bool) for x in (*values, *weights)):
        raise ValueError("Use integer or Fraction values and weights")
    total = sum(weights, F(0))
    if any(w < 0 for w in weights) or total <= 0:
        raise ValueError("Weights must be nonnegative with positive total")
    mean = sum((x*w for x, w in zip(values, weights)), F(0)) / total
    second = sum((x*x*w for x, w in zip(values, weights)), F(0)) / total
    return mean, second


def main():
    x, w = [-2, 0, 4], [1, 2, 1]
    mean, second = moments(x, w)
    assert (mean, second) == (F(1, 2), F(5))
    y = [2*v + 1 for v in x]  # Y shares X's outcome: these are dependent.
    assert moments(y, w)[0] == 2
    assert moments([a+b for a, b in zip(x, y)], w)[0] == F(5, 2) == 3*mean + 1
    assert moments([a*b for a, b in zip(x, y)], w)[0] == F(21, 2)
    assert mean * moments(y, w)[0] == 1
    assert moments([-2*v + 5 for v in x], w)[0] == 4
    assert moments([v + 2 for v in x], w)[0] == F(5, 2)
    assert moments([2, 2, 2], w) == (F(2), F(4))
    assert moments(x, [10, 20, 10]) == (mean, second)
    assert moments([99, 2], [0, 1]) == (F(2), F(4))
    for values, weights in (([], []), ([0], [0]), ([0, 1], [1, -1]), ([0, 1], [1]), ([0.5], [1])):
        try:
            moments(values, weights)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid exact weighted model accepted")
    print("E[X] =", mean, "; E[X²] =", second)
    print("Dependent Y = 2X+1: E[X+Y] = 5/2; E[XY] = 21/2, not E[X]E[Y] = 1")


if __name__ == "__main__":
    main()

"""Normalize a weight vector, then compute expectations as dot products."""
import numpy as np


def moments(values, weights):
    x, w = np.asarray(values, dtype=float), np.asarray(weights, dtype=float)
    if x.ndim != 1 or x.size == 0 or w.shape != x.shape or not np.all(np.isfinite(x)):
        raise ValueError("Require finite nonempty values and matching weights")
    if not np.all(np.isfinite(w)) or np.any(w < 0) or not np.isfinite(w.sum()) or w.sum() <= 0:
        raise ValueError("Require nonnegative finite weights with positive finite total")
    p = w / w.sum()
    return float(np.dot(p, x)), float(np.dot(p, x*x))


def main():
    x, w = np.array([-2., 0., 4.]), np.array([1., 2., 1.])
    mean, second = moments(x, w)
    np.testing.assert_array_equal([mean, second], [0.5, 5])
    y = 2*x + 1
    assert moments(y, w)[0] == 2
    assert moments(x + y, w)[0] == 2.5
    assert moments(x*y, w)[0] == 10.5 != mean * moments(y, w)[0]
    assert moments(-2*x + 5, w)[0] == 4
    assert moments(x + 2, w)[0] == 2.5
    assert moments(np.full(3, 2.), w) == (2., 4.)
    assert moments(x, 10*w) == (mean, second)
    assert moments([99, 2], [0, 1]) == (2., 4.)
    for values, weights in (([], []), ([0], [0]), ([0, 1], [1, -1]), ([np.nan], [1]), ([0], [np.inf])):
        try:
            moments(values, weights)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid weighted model accepted")
    print("Normalized weights:", w/w.sum(), "E[X] ≈", mean, "E[X²] ≈", second)
    print("Same-outcome Y = 2X+1:", y, "E[X+Y] ≈", moments(x+y, w)[0])


if __name__ == "__main__":
    main()

Explore

Increase the raw weight at $4$ and predict the center moving toward $4$. Multiply all weights by the same positive factor: normalization cancels that factor and the mean stays unchanged. Give one value zero weight and explain why it no longer contributes to the weighted sum.

Choose Shift by 2 to obtain transformed mean $5/2$. Choose Scale by −2 from the canonical model to obtain transformed mean $-1$. Compare those inspector rows with the source mean $1/2$. Choose Zero variance, which sets every source value to $2$: the source mean becomes $2$. Use affine scale $-2$ and shift $3$ on the canonical values to predict transformed mean $2$ before reading it.

Common mistakes and counterexamples

The mean need not be a possible value or the most probable one. Here the most probable value is $0$, while the mean is $1/2$. Rounding the mean to a support point changes the quantity rather than correcting it.

Squaring before averaging differs from squaring afterward: $\mathbb E[X^2]=5$ but $(\mathbb E[X])^2=1/4$. Also, independence is not a prerequisite for linearity. The explicit $Y=2X+1$ example is dependent and still obeys the sum rule. Its product calculation refutes a careless extension of that rule.

Numerical considerations

Finite rational values and weights allow exact expectations. Construct fractions from integers or decimal strings rather than first rounding them to binary floats. NumPy calculations are approximate in general; exact-looking canonical decimals do not imply every weighted input remains exact.

A sample mean averages observed data, whereas this stage computes a theoretical expectation from a specified model. Raw weights describe proportional probabilities here, not necessarily sample frequencies. Agreement between a short observed average and the model mean is not guaranteed; the sampling chapters study that distinction later.

Engineering and ML connections

Expected loss weights per-outcome losses by the model’s probabilities. Combining loss terms uses linearity even when they depend on the same example. Replacing theoretical probabilities with observed frequencies yields an empirical average; it estimates a model quantity under additional assumptions rather than becoming an exact expectation automatically.

Exercises

Use exact rational answers for the finite model. The explanations test the limits of linearity as well as its useful identities.

Summary and checkpoints

Expectation is a normalized probability-weighted sum. Check that you can obtain $1/2$ and $5$ as first and second moments, predict affine changes to the center, justify dependent linearity for $Y=2X+1$, and distinguish a theoretical mean from an observation or sample estimate.