Does knowing B change the chance of A?

The opening model uses Two fair coins: A means first coin H, and B means second coin H. Compare the joint mass $1/4$ with the marginal product. Then choose Fair die for the even faces and top three faces. Both still have mass $1/2$: predict whether that fact alone makes them independent, then inspect the changed shared mass.

Intuition: compare information, not diagram separation

If learning that B happened leaves the chance of A unchanged, the events provide no probability update about each other. This can happen even though they share outcomes. Two independent fair coins illustrate it: knowing the second coin is heads leaves the first coin’s chance of heads at $1/2$.

By contrast, mutually exclusive events forbid each other. Knowing a die roll is odd makes its chance of being even zero. Their drawn regions may be separate, but that separation encodes strong information.

Definition

Events A and B are independent exactly when

The probabilities of A and B alone are their marginal masses; the intersection is their joint mass. If $P(B)>0$, the product equality is equivalent to $P(A\mid B)=P(A)$. If $P(A)>0$, it is also equivalent to $P(B\mid A)=P(B)$.

The product definition includes zero-mass cases without division. Independence of two events is a statement about those events under a specified model. It does not imply that every pair of events in the same experiment is independent.

Mechanics: test the equality from one model

The stage sums original weights for A, B, and their intersection. Its comparison uses these derived masses. Change membership and read the joint term and product separately before looking at the classification. Equal-sized event drawings alone cannot determine it.

Replay the trace: collect marginal weights, collect shared weights, multiply the marginals, and compare. Where a conditioning mass is positive, relate the result to the conditional inspector. Where it is zero, use the product comparison and keep the corresponding conditional probability undefined.

Derivation: the conditional equivalence

With positive B mass,

The last condition is symmetric even though the original conditional expression is directional. Independence also extends to complements: subtract the joint mass from $P(A)$ to obtain

For disjoint events with positive marginals, joint mass is zero and the product is positive, so equality fails. If one marginal is zero, its intersection is a subset of a zero-mass event and also has zero mass; equality then holds.

Worked examples

For the canonical fair die, $A=\{2,4,6\}$ and $B=\{4,5,6\}$. The joint mass is $1/3$, while the product is $(1/2)(1/2)=1/4$. They are dependent; indeed $P(A\mid B)=2/3\ne1/2$.

For two independent fair coins, $\Omega=\{HH,HT,TH,TT\}$ with masses $1/4$. Let A mean first coin H and B mean second coin H. Then $A=\{HH,HT\}$, $B=\{HH,TH\}$, and $A\cap B=\{HH\}$. The joint mass $1/4$ equals the marginal product. Given B, $HH$ and $TH$ remain equally weighted, so the first coin still has heads probability $1/2$.

For a counterexample within the same coin model, let C mean “both coins H.” Then $P(C\mid A)=1/2$ but $P(C)=1/4$. Independent component tosses do not make arbitrary derived events independent.

Executable reference code

Python checks the exact product definition by summing Fraction weights over sets. It verifies the dependent die, independent coin events, mutually exclusive positive events, and a zero-mass event with undefined reverse conditioning. NumPy computes the same small event masses with masks and prints the numerical joint-minus-product difference; exact fractions remain the reference for classification.

"""Exact independence tests include zero-mass events without dividing by zero."""
from fractions import Fraction as F


def compare(weights, a, b):
    if not weights or any(not isinstance(p, F) or p < 0 for p in weights.values()):
        raise ValueError("Use a nonempty model of nonnegative Fraction masses")
    if sum(weights.values(), F(0)) != 1:
        raise ValueError("Masses must sum exactly to one")
    a, b = set(a), set(b)
    if not (a | b) <= weights.keys():
        raise ValueError("Events must be inside the sample space")
    pa = sum((weights[x] for x in a), F(0))
    pb = sum((weights[x] for x in b), F(0))
    joint = sum((weights[x] for x in a & b), F(0))
    return {"P(A)": pa, "P(B)": pb, "joint": joint, "product": pa * pb,
            "independent": joint == pa * pb,
            "P(A|B)": joint / pb if pb else None,
            "P(B|A)": joint / pa if pa else None}


def main():
    die = {face: F(1, 6) for face in range(1, 7)}
    a, b = {2, 4, 6}, {4, 5, 6}
    dependent = compare(die, a, b)
    assert dependent["joint"] == F(1, 3)
    assert dependent["product"] == F(1, 4)
    assert dependent["P(A|B)"] == F(2, 3)
    assert not dependent["independent"]
    coins = {outcome: F(1, 4) for outcome in ("HH", "HT", "TH", "TT")}
    first_h, second_h = {"HH", "HT"}, {"HH", "TH"}
    independent = compare(coins, first_h, second_h)
    assert independent["joint"] == independent["product"] == F(1, 4)
    assert independent["P(A|B)"] == F(1, 2) and independent["independent"]
    assert not compare(die, a, set(die) - a)["independent"]
    assert compare(die, a, set())["independent"]
    zero = compare(die, set(), b)
    assert zero["independent"] and zero["P(A|B)"] == 0 and zero["P(B|A)"] is None
    zero_label = {"x": F(1, 2), "y": F(1, 2), "z": F(0)}
    assert compare(zero_label, {"z"}, {"x"})["independent"]
    assert compare(die, a, set(die))["independent"]
    assert not compare(coins, first_h, {"HH"})["independent"]
    cells = (F(1, 6), F(1, 6), F(1, 3), F(1, 3))
    assert sum(cells) == 1 and cells[0] + cells[1] == F(1, 3)
    assert cells[0] + cells[2] == F(1, 2)
    # Pairwise independence does not force the triple product rule.
    matches = {"HH", "TT"}
    assert compare(coins, first_h, matches)["independent"]
    assert compare(coins, second_h, matches)["independent"]
    triple = sum((coins[x] for x in first_h & second_h & matches), F(0))
    assert triple == F(1, 4) != F(1, 8)
    for weights, event in ((die, {7}), ({}, set()),
                           ({1: F(-1), 2: F(2)}, {1}), ({1: F(1, 2)}, {1})):
        try:
            compare(weights, event, set())
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid model or event was accepted")
    print("Dependent fair-die events:", dependent)
    print("Independent fair-coin positions:", independent)
    print("Zero event: product definition holds; reverse conditioning is undefined:", zero)
    print("Pairwise coin events: triple mass 1/4, product 1/8 (not mutually independent)")


if __name__ == "__main__":
    main()

"""Numerical joint-product comparisons are cross-checks, not exact proofs.

The literal Fraction joint and product supply the mathematical classification.
Absolute error tolerance 1e-12, relative tolerance 0, checks these small arrays.
"""
from fractions import Fraction as F
import numpy as np

ATOL = 1e-12


def compare(weights, a, b):
    p = np.asarray(weights, dtype=float)
    a, b = np.asarray(a), np.asarray(b)
    if p.ndim != 1 or p.size == 0 or np.any(p < 0) or not np.all(np.isfinite(p)):
        raise ValueError("Use a nonempty finite nonnegative mass vector")
    if not np.isclose(p.sum(), 1.0, rtol=0, atol=ATOL):
        raise ValueError("Masses must sum to one within floating-point tolerance")
    if a.dtype != np.bool_ or b.dtype != np.bool_ or a.shape != p.shape or b.shape != p.shape:
        raise ValueError("Events need boolean masks matching the mass vector")
    pa, pb, joint = p[a].sum(), p[b].sum(), p[a & b].sum()
    return np.array([pa, pb, joint, pa * pb]), joint / pb if pb > 0 else None


def check_example(weights, a, b, exact, independent):
    masses, conditional = compare(weights, a, b)
    np.testing.assert_allclose(masses, [float(x) for x in exact], rtol=0, atol=ATOL)
    assert (exact[2] == exact[3]) == independent
    if independent:
        np.testing.assert_allclose(masses[2], masses[3], rtol=0, atol=ATOL)
    print("Marginals, joint, product ≈", masses, "; numerical gap ≈", masses[2] - masses[3])
    print("Exact joint/product:", exact[2], exact[3], "; independent:", independent)
    return conditional


def main():
    faces = np.arange(1, 7)
    p = np.full(6, 1 / 6)
    a, b = np.isin(faces, [2, 4, 6]), np.isin(faces, [4, 5, 6])
    die_conditional = check_example(p, a, b, [F(1, 2), F(1, 2), F(1, 3), F(1, 4)], False)
    np.testing.assert_allclose(die_conditional, float(F(2, 3)), rtol=0, atol=ATOL)
    coins = np.array(["HH", "HT", "TH", "TT"])
    first_h, second_h = np.isin(coins, ["HH", "HT"]), np.isin(coins, ["HH", "TH"])
    coin_conditional = check_example(np.full(4, 0.25), first_h, second_h,
                                     [F(1, 2), F(1, 2), F(1, 4), F(1, 4)], True)
    np.testing.assert_allclose(coin_conditional, float(F(1, 2)), rtol=0, atol=ATOL)
    check_example(p, a, ~a, [F(1, 2), F(1, 2), F(0), F(1, 4)], False)
    undefined = check_example(p, a, np.zeros(6, dtype=bool), [F(1, 2), F(0), F(0), F(0)], True)
    assert undefined is None
    for bad_p, bad_a, bad_b in (([-0.1, 1.1], [True, False], [False, True]),
                                 ([0.5, 0.4], [True, False], [False, True]),
                                 ([np.nan], [True], [True]),
                                 ([1.0], [1], [True]), ([1.0], [True], [True, False])):
        try:
            compare(bad_p, bad_a, bad_b)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid model or masks were accepted")
    print("The zero event is independent; conditioning on it is undefined")


if __name__ == "__main__":
    main()

Explore

Choose Fair die, then make B the whole die space: the joint mass equals $P(A)$ and the product equals $P(A)\times1$, so independence holds. Make B empty: both terms are zero, so independence again holds, while conditioning on B is undefined. Make A odd and B even: both marginal masses are $1/2$, but the joint mass is zero and independence fails. Reset in this chapter restores the independent two-coin model.

Use the coin reference to inspect the four joint cells. For independent A and B with masses $1/3$ and $1/2$, the cells A-and-B, A-and-not-B, not-A-and-B, and neither have masses $1/6,1/6,1/3,1/3$. Their row and column sums recover the marginals, and their total is one.

Common mistakes and counterexamples

Independence does not mean disjointness or equal probabilities. Events of mass zero are independent of every event under the product definition, yet conditioning on them remains undefined. An event is independent of itself only if its mass is zero or one, because $p=p^2$ then holds.

Do not infer independence from a short empirical run, apparent visual separation, or lack of an obvious causal link. Independence is a property of the specified joint probability model. Correlation, introduced later, summarizes a different feature of numerical variables and is not a general replacement for this definition.

Numerical considerations

The finite rational reference compares joint and product exactly. Decimal formatting cannot turn $1/3$ into $1/4$, and an exact small discrepancy remains dependence. NumPy’s absolute tolerance $10^{-12}$ with zero relative tolerance is appropriate for cross-checking these small hand-worked values; an approximate equality test alone cannot prove exact independence of arbitrary floating-point or measured inputs.

If a larger numerical model reports “approximately independent,” it needs an explicit error and modeling policy. The stage’s exact entered model and uncertainty in a physical data source are separate issues. A sampling fluctuation also need not reflect a defect in a model that assumes independence.

Engineering and ML connections

Independent draws support many simulation and learning algorithms, but repeated use of the same observation, shared sources, or coordinated selection can create dependence. Naive Bayes later assumes certain features are independent conditional on a class, a stronger modeling choice than merely observing small correlations. Test the assumption against the intended joint structure rather than the visual layout of a dataset.

Exercises

Summary and checkpoints

Independence means joint mass equals the product of marginal masses. Check that you can verify the coin example, reject the die example, distinguish independence from mutual exclusion, and apply the zero-mass definition without inventing an undefined conditional ratio.