Mathematical Foundations

II.2 · Linear Algebra

What does a negative scale do?

Change one scalar and watch both components respond. Length follows its absolute value; direction follows its sign, until zero removes direction entirely.

Scale a move; inspect its sign

The length is multiplied by |k|. The direction is reversed because k is negative.

original vector
(3, 2)
scalar
−1.5
scaled vector
(−4.5, −3)
original length
√133.605551…
scaled length
3√13/25.408327…
relative direction
reversed

Exact expressions use =. Rounded computations use ≈. All arrows use these same quantities.

The length is multiplied by |k|. The direction is reversed because k is negative.

kv=−1.5[32]=[−4.5−3]k\mathbf v=-1.5\begin{bmatrix}3\\2\end{bmatrix}=\begin{bmatrix}-4.5\\-3\end{bmatrix}
∥kv∥=∣k∣∥v∥=3132\norm{k\mathbf v}=|k|\norm{\mathbf v}=\frac{3\sqrt{13}}{2}

from −4 to 4, up to 2 decimal places

from −4 to 4, up to 2 decimal places

from −2 to 2, up to 2 decimal places

Change k with its field or step buttons; move v with pointer, keys, or numeric entry. The second arrow is kv, derived from the same components shown in the inspector.

Try k=1k=1, k=0k=0, and k=−1k=-1 before reading. Predict whether the result is longer, shorter, or reversed when k=−0.5k=-0.5. The original arrow stays available for comparison. The plane fits both arrows at one common scale, so compare their lengths within the current picture rather than across two differently fitted pictures.

Intuition: repeat or reverse a move

The vector (3,2)(3,2) says “three right, two up.” Twice that move is (6,4)(6,4). Half that move is (1.5,1)(1.5,1). Scaling changes how much of the instruction you execute. A negative scalar first reverses the instruction and then changes its amount: −2(3,2)=(−6,−4)-2(3,2)=(-6,-4).

The scalar is one number. It acts on both components together. If you stretch only the horizontal component, the direction generally changes; that is a different operation. Uniform scalar multiplication keeps a nonzero vector on the same line through the origin.

Definition

For k∈Rk\in\mathbb R and v∈R2\mathbf v\in\mathbb R^2,

kv=[kvxkvy].k\mathbf v=\begin{bmatrix}kv_x\\kv_y\end{bmatrix}.

If k>0k>0 and v≠0\mathbf v\ne\mathbf0, kvk\mathbf v points the same way. If k<0k<0, it points the opposite way. If k=0k=0 or v=0\mathbf v=\mathbf0, the result is the zero vector. Its direction is undefined, regardless of the other input.

Mechanics: sign and amount are separate

The scalar control accepts decimals between −2-2 and 22 with at most two decimal places. This range is an exploration window, not a restriction on the mathematical definition. Values outside it are rejected with a message. Arrow keys on the original vector move its components; arrow keys in the scalar field step kk. Both input paths update the same lesson state.

Watch the inspector’s two lengths. A negative kk can make the result longer, but it cannot make the length negative. At zero, the length remains a defined number while the direction row explicitly becomes undefined. Making v=0\mathbf v=\mathbf0 produces that same exception even if kk is positive.

Derivation: the norm uses an absolute value

Apply the Euclidean length formula from I.3 Vectors:

∥kv∥=(kvx)2+(kvy)2=k2(vx2+vy2)=∣k∣∥v∥.\norm{k\mathbf v} =\sqrt{(kv_x)^2+(kv_y)^2} =\sqrt{k^2(v_x^2+v_y^2)} =|k|\norm{\mathbf v}.

The last step uses k2=∣k∣\sqrt{k^2}=|k|, which holds for every real kk. Writing kk instead would fail whenever kk is negative. Squared length scales by k2k^2; length scales by ∣k∣|k|.

The component definition also gives three useful laws:

k(u+v)=ku+kv,(a+b)v=av+bv,a(bv)=(ab)v.k(\mathbf u+\mathbf v)=k\mathbf u+k\mathbf v,\qquad (a+b)\mathbf v=a\mathbf v+b\mathbf v,\qquad a(b\mathbf v)=(ab)\mathbf v.

For example, the first horizontal component is k(ux+vx)=kux+kvxk(u_x+v_x)=ku_x+kv_x. The vertical component follows the same real-number distributive law. Scaling a combined move therefore agrees with scaling the two moves first and then adding them.

Worked examples

Take v=(3,2)\mathbf v=(3,2) and k=−1.5k=-1.5. The output is (−4.5,−3)(-4.5,-3). The original length is 13\sqrt{13} and the new length is 1.5131.5\sqrt{13}. Both component signs reverse, while the length factor is positive.

Inspect the worked calculation

Multiply both components of v = (3, 2) by −1.5.

kv=−1.5[32]=[−4.5−3]k\mathbf v=-1.5\begin{bmatrix}3\\2\end{bmatrix}=\begin{bmatrix}-4.5\\-3\end{bmatrix}
original vector
(3, 2)
scalar
−1.5
scaled vector
(−4.5, −3)
original length
√133.605551…
scaled length
3√13/25.408327…
relative direction
reversed
Inspect the multiplication and the length law separately, then replay the calculation.

A different problem asks which scalar maps (2,−3)(2,-3) to (−1,1.5)(-1,1.5). The horizontal ratio is −1/2-1/2 and the vertical ratio is 1.5/(−3)=−1/21.5/(-3)=-1/2. Because both ratios agree, the target is a scalar multiple. A target such as (−1,2)(-1,2) gives inconsistent ratios and cannot be reached by scaling this vector alone.

Executable reference code

The Fraction implementation makes decimal scaling exact. Its demonstration checks both components and the norm law. NumPy expresses the practical array operation directly. In each case, a zero output must be checked before any later direction computation.

code/python/linear_scaling.py
from fractions import Fraction
from math import hypot, isclose


def scale(k, v):
    if len(v) != 2:
        raise ValueError('Expected two components')
    k = Fraction(str(k))
    return tuple(k * Fraction(str(x)) for x in v)


if __name__ == '__main__':
    result = scale('-1.5', (3, 2))
    assert result == (Fraction('-4.5'), -3)
    assert isclose(hypot(*result), 1.5 * hypot(3, 2), rel_tol=2e-15)
    assert scale(0, (3, 2)) == (0, 0)  # no direction at zero
code/python/linear_scaling_numpy.py
import numpy as np

v = np.array([3.0, 2.0])
k = -1.5
scaled = k * v
assert np.array_equal(scaled, [-4.5, -3])
assert np.isclose(np.hypot(*scaled), abs(k) * np.hypot(*v))
# k = 0 yields zero; do not attach a direction to that result.

Explore

Set v=(3,4)\mathbf v=(3,4) and predict the length for k=0.25,−0.25,1.5,−1.5k=0.25,-0.25,1.5,-1.5. Compare each pair with the same absolute value. Then make one component of v\mathbf v zero: the direction rule still works on an axis. Finally set both components to zero and explain why changing kk cannot restore a direction.

Try to reach (2,3)(2,3) by scaling (3,2)(3,2). The ratios do not agree. This failed construction is a preview of span: one vector supplies one line of possible moves, while a second independent vector supplies another direction.

Common mistakes

Multiplying only one component is an axis stretch. Adding kk to both components is a translation in component space. Neither implements kvk\mathbf v. Also distinguish “opposite” from “perpendicular”: multiplying by −1-1 changes the direction by a half-turn, not a quarter-turn. A negative length is never the explanation of a reversed arrow.

Numerical considerations

The stage uses exact decimal rationals for multiplication and an exact radical when expressing the norm. Rendered screen coordinates and rounded norm readouts are numerical views of those values. In ordinary floating point, a very large product can overflow and a very small product can underflow to zero. Once an output has underflowed to zero, an algorithm must not claim its stored components still specify a unit direction. The conceptual norm law remains true over real numbers even when its direct numerical implementation loses information. Exact entered decimals also differ from exact representations of a binary64 measurement; choose the input interpretation deliberately.

Engineering and ML connections

A gain multiplies a signal; a time interval scales a velocity into a displacement. Units must be tracked: seconds times metres per second produce metres. In parameter updates, a learning rate scales a direction before addition. Negative scaling reverses an update, so signs in an optimization formula determine whether the move follows or opposes a gradient. Later chapters will derive those directions rather than assume them.

Exercises

calculation

Negative scaling

Compute −2 times v = (3, −1).

calculation

Scale a length

v = (3, 4), k = −1.5. Give the length of kv.

construction

Choose the gain

v = (2, −3). Find k so that kv = (−1, 1.5).

prediction

Predict k = 0

v is nonzero and k becomes zero. What happens?

Choose one answer

explanation

A length assertion

Which assertion expresses the norm law for a negative k?

Choose one answer

Summary and checkpoints

One scalar multiplies every component. Its absolute value controls length, and its sign controls the direction of a nonzero result. Verify that you can derive the norm law, diagnose inconsistent component ratios, and handle the zero exception. II.3 Norms and normalization uses a positive scalar chosen from the vector’s own length to create a unit direction.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available