Mathematical Foundations

III.5 · Calculus

Why does a product's rate need both factors?

Grow a rectangle whose sides are two changing quantities and see the product rule appear as two strips and a vanishing corner. Then divide, and meet the points where a quotient — and its rule — stop existing.

Derive sum, product and quotient rules

Combine u and v by

w(x) = u(x) · v(x) and its tangent at x = 1

u(x) = x + 1 (dashed blue), v(x) = x² + 1 (dashed green) and w = u · v (solid). At x = 1: w = 4, tangent slope by the rule 6.

Product as an area: u·v and its increment

Rectangle with sides u = 2 and v = 2. After Δx = 0.5 the sides become 2.5 and 3.25. The added area 4.125 is vΔu = 1 (blue strip) + uΔv = 2.5 (green strip) + ΔuΔv = 0.625 (corner).

Point
1
First factor
2
Second factor
2
Rate of u
1
Rate of v
2
Combined value u · v
4
Derivative by the rule
6
Check: expand, then differentiate
6
Exact agreement
yes, exactly equal

Exact values use =; binary64 approximations use ≈. Undefined values state their reason.

Product rule: each factor's rate is weighted by the OTHER factor's value. Exact result 6, equal to differentiating the expanded product. The corner ΔuΔv is second-order: it vanishes from the rate as Δx → 0.

Δ(uv)=v Δu+u Δv+Δu Δv\Delta(uv)=v\,\Delta u+u\,\Delta v+\Delta u\,\Delta v
(uv)′(1)=u′v+uv′=1⋅2+2⋅2=6(uv)'(1)=u'v+uv'=1\cdot2+2\cdot2=6
4.125=1+2.5+0.625(Δx=0.5)4.125=1+2.5+0.625\quad(\Delta x=0.5)
ddx(x3+x2+x+1)=3x2+2x+1\frac{d}{dx}\left(x^{3}+x^{2}+x+1\right)=3x^{2}+2x+1
at x=1: 6(expand first)\text{at } x=1:\ 6\quad\text{(expand first)}

Evaluate the factors

u(1) = 2, v(1) = 2.

u(1)=2,v(1)=2u(1)=2,\quad v(1)=2

from −3 to 3, up to 2 decimal places

from −3 to 3, up to 2 decimal places

from −3 to 3, up to 2 decimal places

from −2.5 to 2.5, up to 2 decimal places

from −1 to 1, up to 2 decimal places

Adjust u(x) = a·x + b and v(x) = x² + c, move x, and switch between sum, product and quotient. Every rule result is checked exactly by an independent route.

The product plane shows u⋅vu\cdot v as the area of a rectangle with sides u(x)u(x) and v(x)v(x). When xx grows by Δx\Delta x, the area gains a blue strip v Δuv\,\Delta u, a green strip u Δvu\,\Delta v and a small corner Δu Δv\Delta u\,\Delta v. The inspector confirms that these three pieces add up exactly to the change of the product. Now choose Quotient with v(x) = 0: there is no value, no tangent and no rule result at that xx.

Intuition: a rate is a sensitivity

u′(x)u'(x) says how fast uu changes per unit of xx. In a sum u+vu+v, the two changes simply add. In a product, a change in uu matters more when the other factor vv is large: a 1-unit wider rectangle gains area equal to its height. Each factor’s rate is weighted by the other factor’s value.

Definition

For differentiable uu, vv and constants α\alpha, β\beta:

(αu+βv)′=αu′+βv′,(\alpha u+\beta v)'=\alpha u'+\beta v', (uv)′=u′v+uv′,(uv)'=u'v+uv',

and, wherever v(x)≠0v(x)\ne0,

(uv)′=u′v−uv′v2.\left(\frac uv\right)'=\frac{u'v-uv'}{v^2}.

Mechanics: what the explorer computes

The factors have rational coefficients, so every value is exact. The rule row applies the formula above. The check row reaches the same derivative independently: for sums and products by expanding the polynomial first and differentiating term by term; for a quotient q=u/vq=u/v by differentiating the identity u=q vu=q\,v with the product rule. The agreement row is an exact rational comparison. Where v(x)=0v(x)=0, the quotient is undefined and every row says so.

Derivation: the product rule from area

Let Δu=u(x+Δx)−u(x)\Delta u=u(x+\Delta x)-u(x) and Δv=v(x+Δx)−v(x)\Delta v=v(x+\Delta x)-v(x). Then exactly

Δ(uv)=v Δu+u Δv+Δu Δv.\Delta(uv)=v\,\Delta u+u\,\Delta v+\Delta u\,\Delta v .

Divide by Δx\Delta x:

Δ(uv)Δx=vΔuΔx+uΔvΔx+Δu ΔvΔx.\begin{aligned}\frac{\Delta(uv)}{\Delta x}={}&v\frac{\Delta u}{\Delta x}+u\frac{\Delta v}{\Delta x}\\ &+\Delta u\,\frac{\Delta v}{\Delta x}.\end{aligned}

As Δx→0\Delta x\to0, the two quotients tend to u′u' and v′v', while Δu→0\Delta u\to0 because a differentiable uu is continuous. The corner term disappears, leaving u′v+uv′u'v+uv'.

Quotient. If q=u/vq=u/v and v≠0v\ne0, then u=qvu=qv, so u′=q′v+qv′u'=q'v+qv' and

q′=u′−qv′v=u′v−uv′v2.q'=\frac{u'-qv'}{v}=\frac{u'v-uv'}{v^2}.

This derivation divides by vv; it is meaningless where v=0v=0.

Worked example

Take u=x+1u=x+1, v=x2+1v=x^2+1 at x=1x=1. Then u=2u=2, v=2v=2, u′=1u'=1, v′=2v'=2.

  • Product: (uv)′=1⋅2+2⋅2=6(uv)'=1\cdot2+2\cdot2=6. Expanding first, uv=x3+x2+x+1uv=x^3+x^2+x+1 has derivative 3x2+2x+1=63x^2+2x+1=6.
  • Quotient: (u/v)′=(1⋅2−2⋅2)/22=−1/2(u/v)'=(1\cdot2-2\cdot2)/2^2=-1/2.

With Δx=1/2\Delta x=1/2: uu grows to 2.52.5 and vv to 3.253.25. The area changes by 8.125−4=4.1258.125-4=4.125, which is vΔu=1v\Delta u=1 plus uΔv=2.5u\Delta v=2.5 plus ΔuΔv=0.625\Delta u\Delta v=0.625.

Replay the worked derivation

Evaluate the factors

u(1) = 2, v(1) = 2.

u(1)=2,v(1)=2u(1)=2,\quad v(1)=2
Replay: evaluate the factors, differentiate each, then combine with the rule and check.

Domains and failures

A quotient rule result exists only where the quotient itself exists. For v=x2−1v=x^2-1 the quotient u/vu/v is undefined at x=±1x=\pm1. At x=−1x=-1 with u=x+1u=x+1, cancelling x+1x+1 removes a hole, but the derivative at −1-1 still does not exist in the original function, because the original function has no value there. The explorer draws dashed caution lines where vv vanishes.

Executable reference code

The reference checks the product and quotient rules against expansion with Fraction and raises an error where the denominator is zero. The NumPy version differentiates polynomial products with numpy.polynomial and masks points where a quotient is undefined.

code/python/calculus_rules.py
"""Product and quotient rules checked against expansion, with exact Fractions (III.5)."""
from fractions import Fraction as F


def poly_eval(c, x):
    return sum(ck * x**k for k, ck in enumerate(c))


def poly_mul(a, b):
    out = [F(0)] * (len(a) + len(b) - 1)
    for i, x in enumerate(a):
        for j, y in enumerate(b):
            out[i + j] += x * y
    return out


def poly_der(c):
    return [k * ck for k, ck in enumerate(c)][1:]


u, v = [F(1), F(1)], [F(1), F(0), F(1)]          # u = x + 1, v = x**2 + 1
x = F(1)
product_rule = poly_eval(poly_der(u), x) * poly_eval(v, x) + poly_eval(u, x) * poly_eval(poly_der(v), x)
assert product_rule == poly_eval(poly_der(poly_mul(u, v)), x) == 6


def quotient_rule(u, v, x):
    vx = poly_eval(v, x)
    if vx == 0:
        raise ZeroDivisionError("v(x) = 0: u/v and its derivative are undefined")
    return (poly_eval(poly_der(u), x) * vx - poly_eval(u, x) * poly_eval(poly_der(v), x)) / vx**2


assert quotient_rule(u, v, x) == F(-1, 2)
try:
    quotient_rule(u, [F(-1), F(0), F(1)], F(1))     # v = x**2 - 1 vanishes at 1
except ZeroDivisionError:
    pass
else:
    raise AssertionError("the quotient rule must not apply where v = 0")
code/python/calculus_rules_numpy.py
"""Rules with numpy.polynomial: products differentiate like expansions; quotients are masked where v = 0 (III.5)."""
import numpy as np
from numpy.polynomial import Polynomial as P

u, v = P([1, 1]), P([1, 0, 1])
x = np.linspace(-2, 2, 9)
assert np.allclose((u * v).deriv()(x), u.deriv()(x) * v(x) + u(x) * v.deriv()(x))

w = P([-1, 0, 1])                                  # x**2 - 1: zero at +-1
valid = w(x) != 0
q_prime = (u.deriv()(x[valid]) * w(x[valid]) - u(x[valid]) * w.deriv()(x[valid])) / w(x[valid]) ** 2
assert valid.sum() == len(x) - 2 and np.all(np.isfinite(q_prime))

Explore

Make uu negative with Negative factor: the area picture still balances, with signed strips. Set a=0a=0 so uu is constant: the product rule reduces to (constant⋅v)′=constant⋅v′(\text{constant}\cdot v)'=\text{constant}\cdot v'. In quotient mode, move cc below zero and slide xx toward a root of vv.

Common mistakes

(uv)′≠u′v′(uv)'\ne u'v': try u=v=xu=v=x. In the quotient rule, the order u′v−uv′u'v-uv' matters. A rule result is not valid at points where the function is undefined, even if the formula’s numerator looks harmless.

Numerical considerations

These rules are exact identities; numerical issues appear only in evaluation. A quotient near a zero of vv divides by a tiny number, amplifying any error in uu or vv; the derivative u′v−uv′u'v-uv' over v2v^2 amplifies even more. Exact arithmetic sidesteps this here; floating-point code should check how close vv is to zero.

Engineering and ML connections

Automatic differentiation (III.10) applies exactly these local rules — sum, product, quotient — at every node of a computation. Normalisations such as softmax or layer normalisation contain quotients, and their derivatives inherit the same domain restrictions near a vanishing denominator.

Exercises

calculation

Apply the product rule

u = x + 1, v = x² + 1. Give (uv)′ at x = 1.

calculation

Apply the quotient rule

u = x + 1, v = x² + 1. Give (u/v)′ at x = 1.

calculation

Area increment

u = 2, v = 3 grow by Δu = 0.5, Δv = 0.25. Give vΔu, uΔv and ΔuΔv.

prediction

A tempting mistake

Is (uv)′ = u′v′?

Choose one answer

prediction

Where the quotient rule applies

u = x + 1, v = x² − 1. Can you use the quotient rule at x = 1?

Choose one answer

Summary and checkpoints

Derivatives are linear; a product’s rate weights each factor’s rate by the other factor’s value; a quotient’s rule follows from the product rule and holds only where the denominator is nonzero. Next, III.6 handles functions of functions: the chain rule.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available