Conditional probability as restricted sample space

Restrict, then renormalize

1Original P = 1/6Inside B: 0

2Original P = 1/6Inside B: 0

3Original P = 1/6Inside B: 0

4Original P = 1/6Inside B: 1/3

5Original P = 1/6Inside B: 1/3

6Original P = 1/6Inside B: 1/3

1= 1/6

2= 1/6

3= 1/6

4= 1/6

5= 1/6

6= 1/6

Sample space: 1, 2, 3, 4, 5, 6. Equal weights explicitly assume equiprobable outcomes. A = {2, 4, 6}; B = {4, 5, 6}. The denominator changes from 1 to P(B) = 0.5. Inside B, P(A|B) = 2/3.

Total probability: 1
Event A: 0.5
Event B: 0.5
Intersection: 1/30.333333…
Conditional probability: 2/30.666667…

= exact model quantity · ≈ numerical companion or observed estimate

Selected outcome weight

from 0 to 100, whole numbers only

The denominator changes from 1 to P(B) = 0.5. Inside B, P(A|B) = 2/3.

P(A)=\sum_{\omega\in A}p_\omega=0.5

P(A\mid B)=\frac{\frac{1}{3}}{0.5}=\frac{2}{3}

Start with total mass one

Equal weights explicitly assume equiprobable outcomes.

\sum_\omega p_\omega=1

1= 1/6

2= 1/6

3= 1/6

4= 1/6

5= 1/6

6= 1/6

SpeedStep 1 of 3

Select the conditioning event B and the target event A. Remove a face from B to change the retained denominator, then inspect the normalized mass of A within B; empty B to examine undefined conditioning.

With a fair die, take $A=\{2,4,6\}$ and $B=\{4,5,6\}$ . Given that B happened, the face must be 4, 5, or 6. Predict the chance of an even face before reading the equation.

Intuition: information changes the reference total

Before observing B, all six die faces are relevant. After learning B, only three can explain the information. Of those, two are even. The original chance of evenness is $1/2$ , the original shared mass is $1/3$ , and the conditional chance is $2/3$ . These are three different questions about the same experiment.

Restriction keeps only outcomes compatible with the evidence. Normalization rescales their total to one. It preserves relative weights among the survivors rather than inventing a new equal-weight model.

Definition

For events $A,B$ with $P(B)>0$ , conditional probability is

P(A\mid B)=\frac{P(A\cap B)}{P(B)}.

For an individual outcome in a finite space, the conditional mass is $p_\omega/P(B)$ when $\omega\in B$ , and zero otherwise. The event after the bar is the known condition and supplies the denominator. If $P(B)=0$ , this event-ratio definition is undefined.

Mechanics: retain, total, normalize

The stage separates original event mass from conditional mass. Changing B changes which outcomes survive and the retained total. Changing A selects which of those survivors answer the target question. The original weights continue to define the experiment; the conditional view is derived from them.

Follow the trace: find $A\cap B$ , add its original masses, find the original mass of B, and divide. The equation and inspector make the denominator visible. Replay after a membership change to explain why the same numerator can produce a different conditional answer.

Derivation: why divide by P(B)?

Suppose every surviving weight is multiplied by a common factor $c$ . We require a normalized conditional space, so

\sum_{\omega\in B}cp_\omega=cP(B)=1.

Thus $c=1/P(B)$ , which exists only for positive retained mass. Summing these normalized weights over the target survivors gives the conditional formula. Rearranging also gives

P(A\cap B)=P(B)P(A\mid B),\qquad P(B)>0.

This multiplication rule does not assume independence. It pairs the probability of the evidence with the chance of the target after that evidence.

Worked examples

For the canonical die, $P(A\cap B)=2/6=1/3$ and $P(B)=3/6=1/2$ . Therefore

P(A\mid B)=\frac{1/3}{1/2}=\frac23.

Within B, faces 4, 5, and 6 each have conditional mass $1/3$ . Two of them belong to A. Knowing B increases the chance of A from $1/2$ to $2/3$ , so these events are dependent.

Worked example: Restrict, then renormalize

1Original P = 1/6Inside B: 0

2Original P = 1/6Inside B: 0

3Original P = 1/6Inside B: 0

4Original P = 1/6Inside B: 1/3

5Original P = 1/6Inside B: 1/3

6Original P = 1/6Inside B: 1/3

1= 1/6

2= 1/6

3= 1/6

4= 1/6

5= 1/6

6= 1/6

Sample space: 1, 2, 3, 4, 5, 6. Equal weights explicitly assume equiprobable outcomes. A = {2, 4, 6}; B = {4, 5, 6}. The denominator changes from 1 to P(B) = 0.5. Inside B, P(A|B) = 2/3.

Total probability: 1
Event A: 0.5
Event B: 0.5
Intersection: 1/30.333333…
Conditional probability: 2/30.666667…

= exact model quantity · ≈ numerical companion or observed estimate

Selected outcome weight

from 0 to 100, whole numbers only

The denominator changes from 1 to P(B) = 0.5. Inside B, P(A|B) = 2/3.

P(A)=\sum_{\omega\in A}p_\omega=0.5

P(A\mid B)=\frac{\frac{1}{3}}{0.5}=\frac{2}{3}

Start with total mass one

Equal weights explicitly assume equiprobable outcomes.

\sum_\omega p_\omega=1

1= 1/6

2= 1/6

3= 1/6

4= 1/6

5= 1/6

6= 1/6

SpeedStep 1 of 3

Toggle face 5 out of the worked conditioning event B. Predict P(A | B) = 1 because all retained faces are even, then restore B = {4, 5, 6} and replay the 2/3 update.

For masses $p_x=1/2$ , $p_y=1/3$ , $p_z=1/6$ , condition on $B=\{y,z\}$ . Its mass is $1/2$ , so the new masses are $2/3$ and $1/3$ . Restricting to two outcomes does not make them equally likely.

Direction matters too. On a fair die, let $C=\{2\}$ and $D=\{2,4,6\}$ . Then $P(C\mid D)=1/3$ , but $P(D\mid C)=1$ . The shared numerator is the same; the two denominators differ.

Executable reference code

The Python program validates normalized weights and events, constructs the full conditional distribution with exact fractions, and reports None for zero evidence. It checks normalization and the die and unequal-weight examples. NumPy masks the original array before division and checks the denominator first; its decimals are approximations of exact hand-worked answers.

code/python/probability_conditional.py

"""Condition by restricting original masses, then normalizing their total.

None means undefined conditioning, not probability zero. Fraction keeps a tiny
positive evidence mass distinct from impossible evidence without an epsilon.
"""
from fractions import Fraction as F


def conditional_distribution(weights, evidence):
    if not weights or any(not isinstance(p, F) or p < 0 for p in weights.values()):
        raise ValueError("Use nonnegative Fraction masses in a nonempty model")
    if sum(weights.values(), F(0)) != 1:
        raise ValueError("Masses must sum exactly to one")
    evidence = set(evidence)
    if not evidence <= weights.keys():
        raise ValueError("Evidence must be inside the sample space")
    retained = sum((weights[x] for x in evidence), F(0))
    if retained == 0:
        return None
    return {x: p / retained if x in evidence else F(0) for x, p in weights.items()}


def conditional_probability(weights, target, evidence):
    target = set(target)
    if not target <= weights.keys():
        raise ValueError("Target must be inside the sample space")
    conditioned = conditional_distribution(weights, evidence)
    if conditioned is None:
        return None
    return sum((conditioned[x] for x in target), F(0))


def main():
    die = {face: F(1, 6) for face in range(1, 7)}
    a, b = {2, 4, 6}, {4, 5, 6}
    original = dict(die)
    given_b = conditional_distribution(die, b)
    assert given_b == {1: F(0), 2: F(0), 3: F(0), 4: F(1, 3), 5: F(1, 3), 6: F(1, 3)}
    assert sum(given_b.values(), F(0)) == 1
    assert conditional_probability(die, a, b) == F(2, 3)
    assert conditional_probability(die, a, {4, 6}) == 1
    assert conditional_probability(die, a, {5}) == 0
    assert conditional_probability(die, a, set(die)) == F(1, 2)
    assert conditional_probability(die, a, set()) is None
    assert die == original  # Conditioning must not overwrite the base experiment.
    weighted = {"x": F(1, 2), "y": F(1, 3), "z": F(1, 6)}
    assert conditional_distribution(weighted, {"y", "z"}) == {"x": F(0), "y": F(2, 3), "z": F(1, 3)}
    assert conditional_probability(die, {2}, a) == F(1, 3)
    assert conditional_probability(die, a, {2}) == 1
    zero_weight = {"x": F(1), "z": F(0)}
    assert conditional_distribution(zero_weight, {"z"}) is None
    tiny = F(1, 10 ** 100)
    assert conditional_probability({"rare": tiny, "other": 1 - tiny}, {"rare"}, {"rare"}) == 1
    for weights, target, evidence in ((die, {7}, b), (die, a, {7}),
                                      ({1: F(-1), 2: F(2)}, {1}, {2}),
                                      ({1: F(1, 2)}, {1}, {1}), ({}, set(), set())):
        try:
            conditional_probability(weights, target, evidence)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid model or event was accepted")
    print("Die given B = {4, 5, 6}: exact normalized masses", given_b)
    print("Exact P(A | B) =", conditional_probability(die, a, b))
    print("Unequal survivors y, z:", conditional_distribution(weighted, {"y", "z"}))
    print("P({2} | even) = 1/3; P(even | {2}) = 1")
    print("Empty or zero-mass evidence: undefined (None)")


if __name__ == "__main__":
    main()

code/python/probability_conditional_numpy.py

"""Mask and normalize; zero represented evidence returns None before division.

The exact Fraction reference determines mathematical impossibility. These small
examples use moderate representable weights; underflow in other inputs needs
scaling or exact arithmetic, not an arbitrary denominator epsilon.
"""
from fractions import Fraction as F
import numpy as np

ATOL = 1e-12


def condition(weights, evidence):
    p = np.asarray(weights, dtype=float)
    b = np.asarray(evidence)
    if p.ndim != 1 or p.size == 0 or np.any(p < 0) or not np.all(np.isfinite(p)):
        raise ValueError("Use a nonempty finite nonnegative mass vector")
    if not np.isclose(p.sum(), 1.0, rtol=0, atol=ATOL):
        raise ValueError("Masses must sum to one within floating-point tolerance")
    if b.dtype != np.bool_ or b.shape != p.shape:
        raise ValueError("Evidence needs a boolean mask with one entry per outcome")
    retained = np.where(b, p, 0.0)
    total = retained.sum()
    if total == 0.0:
        return None
    return retained / total


def main():
    faces = np.arange(1, 7)
    p = np.full(6, 1 / 6)
    original = p.copy()
    a, b = np.isin(faces, [2, 4, 6]), np.isin(faces, [4, 5, 6])
    posterior = condition(p, b)
    exact = [F(0), F(0), F(0), F(1, 3), F(1, 3), F(1, 3)]
    np.testing.assert_allclose(posterior, [float(x) for x in exact], rtol=0, atol=ATOL)
    np.testing.assert_allclose(posterior.sum(), 1.0, rtol=0, atol=ATOL)
    np.testing.assert_allclose(posterior[a].sum(), float(F(2, 3)), rtol=0, atol=ATOL)
    np.testing.assert_array_equal(p, original)
    weighted = condition([0.5, 1 / 3, 1 / 6], [False, True, True])
    np.testing.assert_allclose(weighted, [0.0, float(F(2, 3)), float(F(1, 3))], rtol=0, atol=ATOL)
    np.testing.assert_allclose(condition(p, a)[faces == 2].sum(), float(F(1, 3)), rtol=0, atol=ATOL)
    np.testing.assert_allclose(condition(p, faces == 2)[a].sum(), 1.0, rtol=0, atol=ATOL)
    assert condition(p, np.zeros(6, dtype=bool)) is None
    assert condition([1.0, 0.0], [False, True]) is None
    # Do not use ATOL to classify small positive evidence as impossible.
    np.testing.assert_allclose(condition([1e-100, 1.0], [True, False]), [1.0, 0.0], rtol=0, atol=ATOL)
    for bad_p, mask in (([], []), ([0.5, 0.4], [True, False]),
                        ([-0.1, 1.1], [True, False]), ([np.nan], [True]),
                        ([1.0], [1]), ([1.0], [True, False])):
        try:
            condition(bad_p, mask)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid masses or evidence mask were accepted")
    print("Die conditional masses ≈", posterior, "; exact reference", exact)
    print("P(A | B) ≈", posterior[a].sum(), "(exact 2/3)")
    print("Weighted conditional masses ≈", weighted, "(exact 0, 2/3, 1/3)")
    print("Zero-mass evidence: undefined (None); original weights remain unchanged")


if __name__ == "__main__":
    main()

Explore

Keep A fixed and shrink B to $\{4,6\}$ : the conditional probability becomes one. Change B to $\{5\}$ : it becomes zero. Set B to the whole space: conditioning changes nothing. Finally empty B: no conditional distribution exists.

Compare each case with the numerator and denominator separately. A zero numerator with positive denominator is a valid zero answer. A zero denominator prevents normalization entirely. You can perform these comparisons through membership controls and replay the derivation without waiting for random observations.

Common mistakes and counterexamples

Keeping the original denominator computes the joint event, not the conditional probability. Swapping the events after and before the bar changes the question. Neither conditioning nor changing a diagram establishes causation: learning an event occurred is not the same operation as intervening to force it.

Another mistake is overwriting the original mass vector with a conditional one and then treating it as the original model in later calculations. Retain the base experiment so marginal, joint, and conditional quantities stay comparable.

Numerical considerations

Exact fractions preserve the denominator-zero distinction. Small positive evidence is still valid evidence, not impossible evidence merely because its displayed decimal rounds to zero. However, division by a small evidence mass can amplify absolute input error, especially if the weights are measurements rather than exact specified values.

The NumPy reference checks small known answers within absolute tolerance $10^{-12}$ and zero relative tolerance. This is a reference-comparison tolerance, not a rule for replacing tiny positive evidence with zero. Larger inference systems may need scaled or logarithmic weights to avoid underflow; such numerical choices must keep impossible evidence distinct from numerical loss of representable mass.

Engineering and ML connections

Conditional summaries compare data inside an observed subgroup. Their denominator is subgroup size or subgroup weight, not full dataset size. Conditional models likewise describe predictions given inputs or context. A selection filter can change rates even when the original population is unchanged, so always name the conditioned population before interpreting a metric.

Exercises

Summary and checkpoints

Conditioning restricts to B and divides by its original mass. Check that you can explain the changed denominator, renormalize unequal survivors, distinguish joint from conditional probability, reverse a conditional question correctly, and report zero-probability evidence as undefined.

What changes when we know B happened?

Restrict, then renormalize

Intuition: information changes the reference total

Definition

Mechanics: retain, total, normalize

Derivation: why divide by P(B)?

Worked examples

Worked example: Restrict, then renormalize

Executable reference code

Explore

Common mistakes and counterexamples

Numerical considerations

Engineering and ML connections

Exercises

Change the denominator

Renormalize unequal weights

Condition on the smaller event

Refuse division by zero

Read a conditional trace

Summary and checkpoints

The map of the book

Mathematical Language

Linear Algebra

Calculus

Probability

Statistics

Optimization

Information Theory

Numerical Computing