IV.3 · Probability
Does changing the order make a different outcome?
Choose labels with or without replacement, then decide whether order matters. Counts become probabilities only after you justify the weights of the counted outcomes.
Count the right kind of outcome
First 12 selections at most · labels A through H
- A → B
1 ordering(s) · P = 1/12 - A → C
1 ordering(s) · P = 1/12 - A → D
1 ordering(s) · P = 1/12 - B → A
1 ordering(s) · P = 1/12 - B → C
1 ordering(s) · P = 1/12 - B → D
1 ordering(s) · P = 1/12 - C → A
1 ordering(s) · P = 1/12 - C → B
1 ordering(s) · P = 1/12 - C → D
1 ordering(s) · P = 1/12 - D → A
1 ordering(s) · P = 1/12 - D → B
1 ordering(s) · P = 1/12 - D → C
1 ordering(s) · P = 1/12
4 labeled items, 2 draws, order matters, without replacement. 12 selections. Under uniform draws, these displayed selections are equiprobable.
- Number of selections
- 12
- Underlying ordered outcomes
- 12
= exact model quantity · ≈ numerical companion or observed estimate
from 2 to 8, whole numbers only
from 0 to 4, whole numbers only
Specify what counts as one outcome
AB and BA are different; AA is forbidden.
Begin with two selections from four labels without replacement. Predict the ordered count, then switch to unordered. Find the pair of ordered outcomes represented by one unordered selection. The stage names its labels A through H; the examples and Python use numbers 1 through 8 for the same distinct items.
Intuition: count what the experiment distinguishes
Giving a first prize to label 1 and a second prize to label 2 differs from giving the prizes in the reverse order. Choosing a two-person team from those labels ignores that order. The same labels therefore support different outcome spaces depending on the experiment’s question.
Replacement asks a separate question: can a label be used again? Keeping these two choices separate prevents a correct counting formula from being used for the wrong experiment.
Definition
Let be the number of distinct labels and the number selected, both integers. For ordered selection without replacement, there are
outcomes when . Ignoring order gives the combination count . The factorial convention is .
With replacement, ordered sequences number . Unordered selections are multisets, which record how many times each label appears; for they number . These are counts, not automatically a uniform probability model.
Mechanics: classify before calculating
The stage’s state records , , order, and replacement. Its outcome representation and count follow those choices. Read the derivation trace from the same state: identify available labels for each slot, multiply choices, and account for ignored order when appropriate.
The interactive range is , . It includes without replacement so you can see an impossible selection. Changing a control does not authorize an impossible draw or a probability with zero total outcomes. Reset returns to , , without replacement.
Derivation: multiplication and grouping
Without replacement, the first slot has choices, the next has , and so on. Multiply because each partial selection branches into that many next choices. Every unordered set of distinct labels has exactly orderings. Dividing the ordered count by therefore gives combinations.
With replacement, every slot has choices, giving . To count unordered multisets, distribute indistinguishable marks among labelled boxes. A row of marks and separators encodes the counts. Choosing the mark positions gives .
Repeated labels make the number of orderings vary by multiset. If a multiset has counts summing to , its number of ordered preimages is . Under independent uniform draws its probability is that number divided by ; this is why “ignore order” does not automatically mean “make all unordered results uniform.”
Worked examples
For , , without replacement, the ordered total is . Each pair occurs in two orders, so there are unordered pairs. The event “select labels 1 and 2” has probability under uniform ordered draws; equivalently it is one of six equally likely unordered pairs.
Worked example: Count the right kind of outcome
First 12 selections at most · labels A through H
- A → B
1 ordering(s) · P = 1/12 - A → C
1 ordering(s) · P = 1/12 - A → D
1 ordering(s) · P = 1/12 - B → A
1 ordering(s) · P = 1/12 - B → C
1 ordering(s) · P = 1/12 - B → D
1 ordering(s) · P = 1/12 - C → A
1 ordering(s) · P = 1/12 - C → B
1 ordering(s) · P = 1/12 - C → D
1 ordering(s) · P = 1/12 - D → A
1 ordering(s) · P = 1/12 - D → B
1 ordering(s) · P = 1/12 - D → C
1 ordering(s) · P = 1/12
4 labeled items, 2 draws, order matters, without replacement. 12 selections. Under uniform draws, these displayed selections are equiprobable.
- Number of selections
- 12
- Underlying ordered outcomes
- 12
= exact model quantity · ≈ numerical companion or observed estimate
from 2 to 8, whole numbers only
from 0 to 4, whole numbers only
Specify what counts as one outcome
AB and BA are different; AA is forbidden.
With replacement, there are sequences and multisets. Four multisets repeat a label; six contain distinct labels. The result has one preimage and mass . The result has two preimages and mass . Assigning to each describes a different experiment: direct uniform selection of multisets.
Executable reference code
The Python program uses standard-library formulas and independently enumerates outcomes with itertools to check them throughout the stage’s small parameter range. It groups replacement sequences by sorted tuples and assigns exact Fraction masses from preimage counts. NumPy checks the same finite grouping using rows of sequences, row sorting, and occurrence counts; no random sampling is needed.
"""Count outcomes, then justify their masses; never assume multisets are uniform.
Enumeration is bounded to the stage's n = 2..8, k = 0..4. Formulas return exact
integers; probabilities below use Fraction, not floating-point count ratios.
"""
from collections import Counter
from fractions import Fraction as F
from itertools import combinations, combinations_with_replacement, permutations, product
from math import comb, perm
def validate(n, k):
if type(n) is not int or type(k) is not int or not 2 <= n <= 8 or not 0 <= k <= 4:
raise ValueError("This explorer requires integer n = 2..8 and k = 0..4")
def selection_count(n, k, ordered, replacement):
validate(n, k)
if replacement:
return n ** k if ordered else comb(n + k - 1, k)
if k > n:
return 0
return perm(n, k) if ordered else comb(n, k)
def enumerate_selections(n, k, ordered, replacement):
validate(n, k)
labels = range(1, n + 1)
if replacement:
return tuple(product(labels, repeat=k) if ordered else combinations_with_replacement(labels, k))
return tuple(permutations(labels, k) if ordered else combinations(labels, k))
def multiset_masses(n, k):
sequences = enumerate_selections(n, k, ordered=True, replacement=True)
preimages = Counter(tuple(sorted(row)) for row in sequences)
return {multiset: F(count, len(sequences)) for multiset, count in preimages.items()}
def main():
assert selection_count(4, 2, True, False) == 12
assert selection_count(4, 2, False, False) == 6
assert selection_count(4, 2, True, True) == 16
assert selection_count(4, 2, False, True) == 10
assert selection_count(2, 3, True, False) == 0
assert selection_count(2, 3, False, False) == 0
for n in range(2, 9):
for k in range(5):
for ordered, replacement in product((False, True), repeat=2):
outcomes = enumerate_selections(n, k, ordered, replacement)
assert len(outcomes) == selection_count(n, k, ordered, replacement)
if k == 0:
assert outcomes == ((),)
masses = multiset_masses(n, k)
assert sum(masses.values(), F(0)) == 1
for n, k in ((1, 2), (9, 2), (4, -1), (4, 5), (4.0, 2), (4, True)):
try:
selection_count(n, k, True, False)
except ValueError:
pass
else:
raise AssertionError("Out-of-range or noninteger input was accepted")
pairs = enumerate_selections(4, 2, True, False)
target_mass = F(sum(set(row) == {1, 2} for row in pairs), len(pairs))
assert target_mass == F(1, 6)
masses = multiset_masses(4, 2)
assert masses[(1, 1)] == F(1, 16)
assert masses[(1, 2)] == F(1, 8)
print("Two of four, without replacement: ordered 12, unordered 6; P({1,2}) =", target_mass)
print("With replacement: ordered 16, unordered 10; exact induced multiset masses:", masses)
print("k = 0: one empty outcome; k > n without replacement: no sampling space")
if __name__ == "__main__":
main()"""Group bounded ordered selections with NumPy, keeping multiplicities visible.
Counts in this n = 2..8, k = 0..4 explorer fit fixed-width integers. Derived
floating-point probabilities use absolute tolerance 1e-12, relative tolerance 0.
"""
from fractions import Fraction as F
from itertools import permutations, product
from math import comb, perm
import numpy as np
ATOL = 1e-12
def grouped_selections(n, k, replacement):
if type(n) is not int or type(k) is not int or not 2 <= n <= 8 or not 0 <= k <= 4:
raise ValueError("Use integer n = 2..8 and k = 0..4")
labels = range(1, n + 1)
sequences = tuple(product(labels, repeat=k) if replacement else permutations(labels, k))
rows = np.asarray(sequences, dtype=np.int64).reshape(len(sequences), k)
if len(rows) == 0:
return rows, np.empty((0, k), dtype=np.int64), np.empty(0, dtype=np.int64), None
# Sorting removes order; counts retain how many ordered rows each result had.
unordered, multiplicities = np.unique(np.sort(rows, axis=1), axis=0, return_counts=True)
masses = multiplicities / len(rows)
return rows, unordered, multiplicities, masses
def main():
for n in range(2, 9):
for k in range(5):
for replacement in (False, True):
rows, unordered, counts, masses = grouped_selections(n, k, replacement)
ordered_count = n ** k if replacement else (perm(n, k) if k <= n else 0)
unordered_count = comb(n + k - 1, k) if replacement else (comb(n, k) if k <= n else 0)
assert len(rows) == ordered_count and len(unordered) == unordered_count
if masses is not None:
assert counts.sum() == ordered_count
np.testing.assert_allclose(masses.sum(), 1.0, rtol=0, atol=ATOL)
if k == 0:
assert rows.shape == unordered.shape == (1, 0) and counts.tolist() == [1]
rows, pairs, counts, masses = grouped_selections(4, 2, False)
assert len(rows) == 12 and len(pairs) == 6 and np.all(counts == 2)
np.testing.assert_allclose(masses, float(F(1, 6)), rtol=0, atol=ATOL)
rows_r, multisets, counts_r, masses_r = grouped_selections(4, 2, True)
assert len(rows_r) == 16 and len(multisets) == 10
repeated = np.all(multisets == [1, 1], axis=1)
distinct = np.all(multisets == [1, 2], axis=1)
np.testing.assert_allclose(masses_r[repeated], float(F(1, 16)), rtol=0, atol=ATOL)
np.testing.assert_allclose(masses_r[distinct], float(F(1, 8)), rtol=0, atol=ATOL)
assert grouped_selections(2, 3, False)[3] is None
for n, k in ((1, 2), (9, 2), (4, -1), (4, 5), (4.0, 2)):
try:
grouped_selections(n, k, False)
except ValueError:
pass
else:
raise AssertionError("Invalid explorer inputs were accepted")
print("Without replacement: 12 sequences, 6 pairs; probabilities ≈", masses)
print("With replacement: 16 sequences, 10 multisets")
for row, count, mass in zip(multisets, counts_r, masses_r):
print(tuple(row), ": ordered preimages", count, "; probability ≈", mass)
print("Impossible without-replacement selection: no normalized masses (None)")
if __name__ == "__main__":
main()Explore
Set in all four modes. There is exactly one empty selection, including one empty ordered tuple. Under this valid experiment its probability is one. Then set , without replacement: there are zero selections, so a normalized uniform selection model cannot be formed. Turn replacement on and observe that ordered choices become possible again.
Predict counts for , without replacement: ordered and unordered. Return to two-of-four with replacement and inspect which unordered results represent more sequences.
Common mistakes and counterexamples
Dividing every ordered replacement count by fails when labels repeat. For two-of-four it gives , but there are ten multisets. Using without replacement also overcounts repeated-label sequences that the experiment excludes.
An impossible selection is not an empty selection. One has no outcome; the other has the single empty outcome. Do not evaluate a factorial with a negative argument to extend the without-replacement formula past ; report count zero and the unavailable sampling model.
Numerical considerations
Counts are exact integers and probabilities are exact rational ratios in the standard-library reference. The bounded explorer permits enumeration, but grows quickly outside that range. A finite display must never be mistaken for an exhaustive list in a much larger experiment.
Python integers grow as needed. NumPy integer arrays have fixed-width types, so a larger counting implementation must consider overflow before arithmetic. Here the bounded counts fit exactly; division into floating-point masses is approximate, checked with absolute tolerance and zero relative tolerance. The reference does not claim a rounded array sum is an exact proof of normalization.
Engineering and ML connections
Sampling a minibatch without replacement differs from independently choosing training examples with replacement. Sequence order matters for temporal input, whereas a selected subset may ignore it. Counting helps estimate search-space size and memory requirements, but probabilities still depend on the sampling rule. Grouping randomized outputs can introduce unequal category probabilities even when the underlying draws are uniform.
Exercises
Summary and checkpoints
Choose an outcome space by specifying order and replacement. Multiply successive choices; divide by equal-size ordering groups only when justified. Check that you can explain 12 versus 6, 16 versus 10, the nonuniform replacement multisets, and the difference between one empty outcome and no possible outcome.