What changes when two variables share one outcome?

Predict whether the upper-left cell has the product of its row and column masses. Then select the top row: which denominator should replace the complete table total?

Intuition: the table keeps the pairing

A marginal distribution describes one variable on its own. It does not say which values occur together. Two households can have the same distribution of daily electricity use and different relationships between morning and evening use. A joint table preserves those pairings.

Each cell represents one pair $(U,V)$. A row collects all pairs with a particular $U$; a column collects all pairs with a particular $V$. Summing a row removes the $V$ coordinate. Conditioning on that row instead keeps its internal proportions and rescales the retained mass to one.

Definition

For finite variables, the joint PMF is

The marginals sum across the unobserved coordinate:

If $p_U(u_i)>0$, the conditional PMF of $V$ given $U=u_i$ is

The analogous column formula divides by $p_V(v_j)>0$. A zero marginal makes this finite conditioning formula undefined; it does not produce an all-zero or uniform distribution.

Mechanics: one normalization, then a restriction

The stage uses rows and columns at latent values $-1,0,1$. Its canonical raw weights are $[1,1,0;1,2,1;0,1,1]$, with total $8$. Raw weights express relative masses; the model explicitly divides every cell by the complete positive total.

Select a cell to edit its nonnegative integer weight. Row and column selectors choose the conditioning event. Read joint cell masses, both marginal distributions, and the selected conditional distribution together. Zero cells are valid; making the entire table zero is invalid. Reset restores the nine canonical weights and the canonical selections.

The independent example uses outer-product weights $[1,2,1;2,4,2;1,2,1]$. Its total is $16$. Its marginal distributions match the canonical ones even though its joint masses differ.

Derivation: why the conditional row sums to one

For a positive-mass row, its conditional masses satisfy

Multiplying the conditional formula back by its denominator gives the product rule

Independence requires every joint cell to factor:

For positive conditioning rows, that equality means the conditional distribution equals the unconditional $V$ marginal. The cellwise test also handles zero rows: every cell there is zero, so both sides of the factorization are zero without dividing by zero.

Worked examples

Both canonical marginals are $[1/4,1/2,1/4]$. The corner joint mass is $1/8$, while the product of its marginals is $(1/4)(1/4)=1/16$. These variables are dependent.

Condition on $U=-1$. Retained row weights $[1,1,0]$ have total $2$, so

Conditioning on the middle row instead gives $[1/4,1/2,1/4]$, which happens to match the $V$ marginal. One matching row does not prove independence; the corner test already disproves it.

In the independent example the corner mass becomes $1/16$ and the center mass remains $1/4$. Every positive row now has conditional distribution $[1/4,1/2,1/4]$. Changing dependence need not change either marginal.

Executable reference code

The exact reference normalizes nine integer weights with Fraction, sums both marginals, builds conditional rows and columns, and tests every factorization exactly. Assertions cover canonical masses, the independent table, deterministic variables, proportional raw weights, and undefined conditioning.

The NumPy reference makes summation axes explicit: axis=1 removes columns and returns row marginals; axis=0 returns column marginals. It uses a documented floating tolerance for its independence cross-check, which is approximate evidence rather than an exact proof.

"""Exact joint, marginal, conditional masses from a 3x3 raw-weight table."""
from fractions import Fraction as F


def joint(weights):
    if len(weights) != 9 or any(type(w) is not int or not 0 <= w <= 1_000_000 for w in weights):
        raise ValueError("Require nine nonnegative integer weights, each at most 1000000")
    total = sum(weights)
    if total == 0:
        raise ValueError("The complete table needs positive weight")
    p = [[F(weights[3*i+j], total) for j in range(3)] for i in range(3)]
    rows = [sum(row, F(0)) for row in p]
    columns = [sum((p[i][j] for i in range(3)), F(0)) for j in range(3)]
    row_conditionals = [None if rows[i] == 0 else [v/rows[i] for v in p[i]] for i in range(3)]
    column_conditionals = [None if columns[j] == 0 else [p[i][j]/columns[j] for i in range(3)] for j in range(3)]
    independent = all(p[i][j] == rows[i]*columns[j] for i in range(3) for j in range(3))
    assert sum(rows) == sum(columns) == 1
    for conditional in (*row_conditionals, *column_conditionals):
        assert conditional is None or sum(conditional) == 1
    return p, rows, columns, row_conditionals, column_conditionals, independent


def main():
    p, rows, columns, row_given, col_given, independent = joint([1,1,0,1,2,1,0,1,1])
    assert p[0][0] == F(1,8) and p[1][1] == F(1,4)
    assert rows == columns == [F(1,4), F(1,2), F(1,4)]
    assert row_given[0] == [F(1,2), F(1,2), F(0)] and not independent
    assert col_given[1] == [F(1,4), F(1,2), F(1,4)]
    assert joint([1,2,1,2,4,2,1,2,1])[-1]
    empty_row = joint([0,0,0,1,0,0,0,0,0])
    assert empty_row[3][0] is None and empty_row[4][1] is None
    assert empty_row[-1]  # A deterministic pair factors, including its zero cells.
    for factor in (1, 2, 10):
        assert joint([factor*w for w in [1,1,0,1,2,1,0,1,1]]) == joint([1,1,0,1,2,1,0,1,1])
    for bad in ([0]*9, [-1]+[1]*8, [1]*8, [True]+[1]*8, [1.5]+[1]*8):
        try:
            joint(bad)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid table accepted")
    print("Joint mass:", p, "marginals:", rows, columns)
    print("V given U=-1:", row_given[0], "independent:", independent)
    print("Zero-mass conditioning is None, never an invented distribution.")


if __name__ == "__main__":
    main()

"""NumPy joint-table operations; float equality is an approximate cross-check."""
import numpy as np


def joint(weights):
    w = np.asarray(weights, dtype=float)
    if w.shape != (3,3) or not np.all(np.isfinite(w)) or np.any(w < 0) or not np.isfinite(w.sum()) or w.sum() <= 0:
        raise ValueError("Require a finite nonnegative 3x3 table of positive total")
    p = w/w.sum()
    rows, columns = p.sum(axis=1), p.sum(axis=0)
    row_given = [None if mass == 0 else p[i]/mass for i, mass in enumerate(rows)]
    col_given = [None if mass == 0 else p[:,j]/mass for j, mass in enumerate(columns)]
    factors = np.outer(rows, columns)
    independent = bool(np.allclose(p, factors, rtol=0, atol=1e-14))
    return p, rows, columns, row_given, col_given, independent


def main():
    w = np.array([[1,1,0],[1,2,1],[0,1,1]])
    p, rows, cols, rg, cg, independent = joint(w)
    np.testing.assert_allclose(rows, [.25,.5,.25], rtol=0, atol=1e-14)
    np.testing.assert_allclose(cols, [.25,.5,.25], rtol=0, atol=1e-14)
    np.testing.assert_allclose(rg[0], [.5,.5,0], rtol=0, atol=1e-14)
    np.testing.assert_allclose(cg[1], [.25,.5,.25], rtol=0, atol=1e-14)
    assert p[0,0] == .125 and not independent
    assert joint(np.outer([1,2,1],[1,2,1]))[-1]
    zero = joint([[0,0,0],[1,0,0],[0,0,0]])
    assert zero[3][0] is None and zero[4][1] is None and zero[-1]
    for bad in (np.zeros((3,3)), -w, [[1]], np.full((3,3), np.nan)):
        try:
            joint(bad)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid table accepted")
    print("Joint mass ≈", p, "row/column marginals ≈", rows, cols)
    print("Conditional row ≈", rg[0], "independent within atol=1e-14:", independent)
    print("Use exact fractions for a mathematical proof of factorization.")


if __name__ == "__main__":
    main()

Explore

Predict which marginal entries change when one corner weight increases. After normalization, even unchanged raw cells can receive different probabilities because the complete denominator changes. Verify the total mass and inspect both affected marginal families.

Use the empty-row or empty-column preset, leaving positive weight elsewhere. Explain why the overall distribution remains valid while that conditional distribution becomes undefined. Restore the row and compare its conditional probabilities with the unconditional column marginal. Multiply all weights by the same positive integer: which probabilities should remain exactly fixed?

Common mistakes and counterexamples

Summing and conditioning answer different questions. The canonical top-row marginal is $1/4$; its conditional distribution has total $1$. Keeping the global denominator in a conditional row would leave total $1/4$ and fail normalization.

Marginals do not determine the joint table. The canonical and independent tables have identical marginals but different corner masses. Also, checking only one row, one covariance, or one cell cannot establish independence across all cells. A mismatch in any cell does suffice to disprove it.

Numerical considerations

In this bounded table, integer weights and normalized fractions support exact sums and comparisons. An all-zero total is rejected; individual zero rows and columns are preserved and reported explicitly. Display rounding never decides whether a conditional denominator is zero or whether the table factors.

With floating data, very small positive marginals can amplify relative errors during conditional division. Distinguish an exact zero from a small positive quantity, and document any numerical comparison tolerance. Do not round a valid row to zero before computing its conditional probabilities.

Engineering and ML connections

A classification confusion matrix records joint behavior between a label and a prediction. Row and column normalization answer different conditional questions, while global normalization answers joint frequency questions. Always state which coordinate is conditioned on and which denominator is used.

Feature marginals alone cannot reveal all dependence between features. A generative model that replaces a joint distribution by the product of marginals makes an independence assumption; it needs justification beyond matching each feature histogram separately.

Exercises

Use exact fractions and state which normalization each answer uses.

Summary and checkpoints

A joint distribution preserves pairing. Marginalization adds away a coordinate; conditioning restricts and renormalizes positive retained mass. Check that you can recover both canonical marginals, normalize the top row, explain an empty condition, and test independence by every joint cell’s factorization.