What does a probability-weighted tilt reveal about dependence?

Predict how reflection of one coordinate changes the sign of association. Predict what remains defined when one scale becomes zero.

Intuition: signs of centered products

When both coordinates lie above their means, their deviations have the same sign and their product is positive. The same is true when both lie below their means. Opposite deviations contribute negative products. Covariance adds those products using probability weights.

The picture must preserve mass: a point of probability $1/4$ contributes twice the weight of a point of probability $1/8$. A scatterplot’s occupied locations alone are insufficient. Positive covariance suggests a shared linear direction; cancellation can hide a curved relationship.

Definition

For finite second moments, write $\mu_X=\mathbb E[X]$ and $\mu_Y=\mathbb E[Y]$. Covariance is

If both variances are positive, correlation is

Covariance has the product of the two variables’ units. Correlation is dimensionless. If either variable is constant on positive-mass outcomes, its variance is zero and correlation is undefined, even though covariance is well defined and zero.

Mechanics: distinguish latent cells from mapped points

The latent table uses $U,V\in\{-1,0,1\}$ and canonical weights $[1,1,0;1,2,1;0,1,1]$. The mapped variables are $X=aU+b$ and $Y=cV+d$. The stage keeps the table clearly labeled by latent $U,V$, while its scatterplot and association rows describe mapped $X,Y$.

Change a scale or shift through the controls. If multiple latent cells map to one scatter location, their probability masses are merged. A zero scale is valid and collapses one mapped variable to a constant; it does not erase or rewrite the latent table. Reset restores identity maps and canonical weights.

Derivation: centering removes shifts

Expanding the centered product gives

For affine transformations, $aU+b-\mathbb E[aU+b]=a(U-\mathbb E[U])$, hence

The two variances multiply by $a^2$ and $c^2$. When both scales are nonzero,

Shifts leave covariance and correlation unchanged. Positive changes of units preserve correlation. Reflecting exactly one axis flips its sign; reflecting both restores it. The correlation formula above does not extend to a zero scale.

Worked examples

Canonical means are zero and both variances are $1/2$. Only the two mass-$1/8$ diagonal corners contribute nonzero $UV$, so

For $X=-2U+3$ and $Y=3V-1$, means become $3,-1$, variances become $2,9/2$, and covariance becomes $-3/2$. Its correlation is $-1/2$.

A different table, $[0,0,1;0,1,0;0,0,1]$, assigns equal mass to $(-1,1),(0,0),(1,1)$. Here $V=U^2$. Its moments are

Symmetry gives $\mathbb E[UV]=0$, so covariance and correlation are zero. Yet $\mathbb P(V=0\mid U=0)=1$ differs from $\mathbb P(V=0)=1/3$. The dependence is nonlinear and exact.

Executable reference code

The exact reference computes centered weighted products with Fraction, checks the alternative moment identity, and merges mapped points. It tests mapped independence separately from latent dependence, including the constant-variable collapse. Correlation is approximate when its square-root denominator is irrational.

The NumPy reference explicitly weights the nine cell coordinates. Calling an unweighted sample covariance routine on nine cell labels would ignore their probabilities and include zero-mass locations as observations; that is a different calculation.

"""Centered exact covariance for mapped latent U,V; correlation may be undefined."""
from fractions import Fraction as F
from math import sqrt


def association(weights, a=F(1), b=F(0), c=F(1), d=F(0)):
    if len(weights) != 9 or any(type(w) is not int or not 0 <= w <= 1_000_000 for w in weights) or sum(weights) <= 0:
        raise ValueError("Require nine bounded nonnegative integer weights of positive total")
    if any(isinstance(s, bool) or not isinstance(s, (int,F)) or abs(s)>1_000_000 for s in (a,b,c,d)):
        raise ValueError("Use exact integer or Fraction affine parameters bounded by 1000000")
    values = [-1,0,1]
    points = [(a*u+b,c*v+d,F(weights[3*i+j],sum(weights))) for i,u in enumerate(values) for j,v in enumerate(values)]
    ex = sum((x*p for x,y,p in points), F(0))
    ey = sum((y*p for x,y,p in points), F(0))
    vx = sum((p*(x-ex)**2 for x,y,p in points), F(0))
    vy = sum((p*(y-ey)**2 for x,y,p in points), F(0))
    cov = sum((p*(x-ex)*(y-ey) for x,y,p in points), F(0))
    assert cov == sum((p*x*y for x,y,p in points),F(0))-ex*ey
    rho = None if vx == 0 or vy == 0 else float(cov)/sqrt(float(vx*vy))
    merged = {}
    for x,y,p in points:
        if p:
            merged[x,y] = merged.get((x,y),F(0))+p
    mx = {x: sum((p for (u,v),p in merged.items() if u==x), F(0)) for x,y in merged}
    my = {y: sum((p for (u,v),p in merged.items() if v==y), F(0)) for x,y in merged}
    independent = all(merged.get((x,y),F(0)) == px*py for x,px in mx.items() for y,py in my.items())
    return ex, ey, vx, vy, cov, rho, independent, merged


def main():
    w=[1,1,0,1,2,1,0,1,1]
    assert association(w)[:6] == (F(0),F(0),F(1,2),F(1,2),F(1,4),.5)
    assert association(w,-2,3,3,-1)[:6] == (F(3),F(-1),F(2),F(9,2),F(-3,2),-.5)
    collapsed=association(w,0,3)
    assert collapsed[2] == 0 and collapsed[5] is None and collapsed[6]
    assert len(collapsed[7]) == 3 and sum(collapsed[7].values()) == 1
    nonlinear=association([0,0,1,0,1,0,0,0,1])
    assert nonlinear[:6] == (F(0),F(2,3),F(2,3),F(2,9),F(0),0.) and not nonlinear[6]
    assert association([1,2,1,2,4,2,1,2,1])[6]
    for a in (-2,0,1,3):
        for c in (-3,0,1,2):
            result=association(w,a,7,c,-4)
            assert result[2:5] == (F(a*a,2),F(c*c,2),F(a*c,4))
            assert result[5] is None if a*c==0 else abs(result[5]) == .5
    print("Canonical moments, covariance, correlation:",association(w)[:6])
    print("Nonlinear zero correlation is still dependent:",nonlinear[:7])
    print("Constant correlation is undefined. Association alone is not causation.")


if __name__ == "__main__":
    main()

"""Weighted theoretical covariance, not NumPy's unweighted sample covariance."""
import numpy as np


def association(weights, a=1., b=0., c=1., d=0.):
    w=np.asarray(weights,dtype=float)
    if w.shape != (3,3) or not np.all(np.isfinite(w)) or np.any(w<0) or not np.isfinite(w.sum()) or w.sum()<=0:
        raise ValueError("Require a finite nonnegative 3x3 table of positive total")
    if not np.all(np.isfinite([a,b,c,d])) or np.any(np.abs([a,b,c,d])>1_000_000):
        raise ValueError("Affine parameters must be finite and bounded by 1000000")
    p=w/w.sum()
    u,v=np.meshgrid([-1.,0.,1.],[-1.,0.,1.],indexing='ij')
    x,y=a*u+b,c*v+d
    ex,ey=np.sum(p*x),np.sum(p*y)
    # Constancy is a support property; floating centering can leave a spurious tiny variance.
    active=p>0
    constant_x=np.all(x[active]==x[active][0])
    constant_y=np.all(y[active]==y[active][0])
    if constant_x:
        ex=x[active][0]
    if constant_y:
        ey=y[active][0]
    dx,dy=x-ex,y-ey
    vx,vy,cov=np.sum(p*dx*dx),np.sum(p*dy*dy),np.sum(p*dx*dy)
    if constant_x or constant_y:
        cov=0.
    if (not constant_x and vx==0) or (not constant_y and vy==0):
        raise ValueError("A nonconstant marginal variance underflowed; rescale the numerical inputs")
    rho=None if constant_x or constant_y else float(cov/np.sqrt(vx)/np.sqrt(vy))
    return float(ex),float(ey),float(vx),float(vy),float(cov),rho


def main():
    w=[[1,1,0],[1,2,1],[0,1,1]]
    np.testing.assert_allclose(association(w),[0,0,.5,.5,.25,.5],rtol=0,atol=1e-14)
    np.testing.assert_allclose(association(w,-2,3,3,-1),[3,-1,2,4.5,-1.5,-.5],rtol=0,atol=1e-14)
    collapsed=association(w,0,3)
    assert collapsed[2]==0 and collapsed[5] is None
    for shift in (.1,.3,-.7):
        result=association(w,0,shift)
        assert result[0]==shift and result[2]==0 and result[4]==0 and result[5] is None
    one_row=association([[0,0,0],[1,1,1],[0,0,0]],1,.3)
    assert one_row[2]==0 and one_row[5] is None
    try:
        association(w,1e-300)
    except ValueError:
        pass
    else:
        raise AssertionError("Nonconstant variance underflow was silently accepted")
    nonlinear=association([[0,0,1],[0,1,0],[0,0,1]])
    np.testing.assert_allclose(nonlinear,[0,2/3,2/3,2/9,0,0],rtol=0,atol=1e-14)
    for a in (-2,0,1,3):
        for c in (-3,0,1,2):
            result=association(w,a,7,c,-4)
            np.testing.assert_allclose(result[2:5],[a*a/2,c*c/2,a*c/4],rtol=0,atol=1e-14)
            if a*c==0:
                assert result[5] is None
            else:
                np.testing.assert_allclose(result[5],np.sign(a*c)*.5,rtol=0,atol=1e-14)
    for bad in (np.zeros((3,3)),[[-1,0,0],[0,0,0],[0,0,1]],np.full((3,3),np.inf)):
        try:
            association(bad)
        except ValueError:
            pass
        else:
            raise AssertionError("Invalid table accepted")
    print("Weighted theoretical association ≈",association(w))
    print("Do not use np.cov on nine cell labels as nine equally likely samples.")
    print("Zero correlation can conceal nonlinear dependence; it proves no causal direction.")


if __name__ == "__main__":
    main()

Explore

Shift both coordinates and verify that centered association remains fixed while the means move. Scale one axis by $-2$ and predict its variance, covariance, and correlation before inspecting the answer. Scale both axes negatively and explain why correlation’s sign returns.

Use the reversed preset for a sign change, the collapsed preset for a constant variable, or the diagonal preset for perfect positive association. Inspect the nonlinear example and identify which conditional probability disproves independence. Finally set $a=0$. Explain how constant $X=b$ is independent of $Y$ even if the latent $U,V$ remain dependent, and why this does not make correlation defined.

Common mistakes and counterexamples

Independence with finite second moments implies zero covariance: independence factors $\mathbb E[XY]$. The converse fails in the $V=U^2$ example. Zero correlation means no centered linear association; it does not mean no relationship.

Variance of a sum includes association:

For canonical $U,V$, the result is $3/2$, not $1$. Correlation also cannot establish causation: a common cause, selection rule, or measurement process can produce association without a direct causal effect.

Numerical considerations

The stage preserves exact fractions for bounded weights, moments, covariance, and affine parameters. Correlation may require an irrational square root and is then labeled approximately. An exact zero variance produces an explicit undefined result, never a fabricated correlation of zero.

Centered products help avoid subtracting large nearly equal floating moments. Near-zero variance can still make a correlation numerically sensitive; displayed rounding must not decide whether the denominator is zero. For floating cross-checks, record the tolerance and keep theoretical weighted moments separate from sample estimators.

Engineering and ML connections

Correlation can detect redundant linear feature directions and covariance can describe the spread of paired quantities. Changing measurement units rescales covariance, while positive unit conversion leaves correlation fixed. A zero-variance feature needs an explicit standardization policy.

Feature screening by correlation alone can discard a predictive nonlinear relationship. Observational associations also need a causal model or experimental design before they justify an intervention; a signed coefficient is not that justification.

Exercises

Use exact fractions for moments and covariance, and explain the denominator whenever correlation is undefined.

Summary and checkpoints

Covariance averages centered cross-products; correlation divides by positive standard deviations. Check the canonical $1/4$ and $1/2$, the signed affine scale laws, zero-scale collapse, the dependent zero-correlation example, and the limits of a causal interpretation.