Mathematical Foundations

III.13 · Calculus

How fast does accumulated area grow?

Let the upper limit of an integral move. The accumulated quantity becomes a function, and its rate of change at x is the height of the integrand at x — wherever that height is continuous.

Watch accumulated area change at the rate f(x)

f(t) = 0.25x³ − x with the accumulated area from a to x

f(t) = 0.25x³ − x. Green: area above the axis between a = −2 and x = 1; gold: area below (subtracted). The orange strip from x to x + Δx has signed area ΔF = −0.37109375, close to f(x)·Δx = −0.375.

Accumulated quantity F(x) = ∫ₐˣ f(t) dt

F starts at 0 at x = a = −2 and accumulates signed area. At x = 1, F = 0.5625; its slope there is −0.75, equal to f(x).

Lower limit (F(a) = 0)
−2
Upper limit
1
Accumulated signed area
0.5625
Integrand at x
−0.75
Change over [x, x + Δx]
−0.37109375
Rectangle estimate
−0.375
Average of f over the strip
−0.7421875
Slope of F (derived from its exact pieces)
−0.75

Exact values use =; binary64 approximations use ≈. Undefined values state their reason.

The strip from x to x + Δx adds ΔF = −0.37109375, close to f(x)·Δx = −0.375. As Δx → 0 the ratio ΔF/Δx tends to f(x) = −0.75: F′(x) = f(x).

F(x)=∫−2xf(t) dtF(x)=\int_{-2}^{x}f(t)\,dt
F(x)=0.0625x4−0.5x2+1F(x)=0.0625x^{4}-0.5x^{2}+1
F(1)=0.5625F(1)=0.5625
ΔF=∫xx+Δxf(t) dt=−0.37109375\Delta F=\int_x^{x+\Delta x}f(t)\,dt=-0.37109375
f(x) Δx=−0.375f(x)\,\Delta x=-0.375
F′(1)=lim⁡Δx→0ΔFΔx=−0.75=f(1)F'(1)=\lim_{\Delta x\to0}\frac{\Delta F}{\Delta x}=-0.75=f(1)

Start the accumulator at a

F(a) = ∫ from a to a = 0 by definition.

F(−2)=0F(-2)=0

from −3 to 3, up to 2 decimal places

from −3 to 3, up to 2 decimal places

from −1 to 1, up to 2 decimal places

Drag the upper limit x along the graph of f or the lower limit a on the axis. The orange strip is the area added between x and x + Δx; the right plane plots the accumulated F with its tangent.

Start at x=1x=1 with a=−2a=-2: the accumulated signed area is F(1)=9/16F(1)=9/16. The orange strip adds ΔF=−95/256\Delta F=-95/256 — negative, because ff is below the axis there — almost exactly f(1) Δx=−3/8f(1)\,\Delta x=-3/8. On the right, the slope of FF at x=1x=1 is −3/4-3/4, the height of ff. Now choose Jump in f: FF has a corner, and no single slope exists at the jump.

Intuition: the accumulator’s speedometer

Imagine a tank that fills at rate f(t)f(t) (negative means draining). The amount in the tank since time aa is F(x)=∫axf(t) dtF(x)=\int_a^x f(t)\,dt. How fast is the amount changing at time xx? Exactly at the current rate, f(x)f(x). Integration accumulates a rate; differentiation recovers it.

Definition

For ff integrable on an interval containing aa, the accumulation function is

F(x)=∫axf(t) dt.F(x)=\int_a^x f(t)\,dt .

Fundamental theorem (first form). If ff is continuous at xx, then FF is differentiable at xx and F′(x)=f(x)F'(x)=f(x).

Second form. If GG is any antiderivative of a continuous ff (G′=fG'=f), then ∫abf=G(b)−G(a)\int_a^b f=G(b)-G(a).

Mechanics: what the explorer computes

The integrands are piecewise polynomials, so FF is computed exactly as a piecewise polynomial that is continuous by construction. The strip’s area ΔF=F(x+Δx)−F(x)\Delta F=F(x+\Delta x)-F(x) is exact, as is the rectangle estimate f(x) Δxf(x)\,\Delta x. The slope row is not f(x)f(x) copied over: it is the derivative of FF‘s own exact pieces, including one-sided slopes at piece boundaries. Agreement with f(x)f(x) is what the theorem predicts — and the jump preset shows where that agreement fails.

Derivation: why F′ = f

For Δx>0\Delta x>0,

F(x+Δx)−F(x)Δx=1Δx∫xx+Δxf(t) dt,\frac{F(x+\Delta x)-F(x)}{\Delta x}=\frac1{\Delta x}\int_x^{x+\Delta x}f(t)\,dt ,

the average of ff over the strip. If ff is continuous at xx, every value of ff on a short enough strip is within any chosen tolerance of f(x)f(x), so the average is too. Letting Δx→0\Delta x\to0 (from either side) gives F′(x)=f(x)F'(x)=f(x). If ff jumps at xx, the averages from the left and right approach the two different one-sided values of ff, and FF has a corner.

Worked example

Take f(t)=t3/4−tf(t)=t^3/4-t and a=−2a=-2. Then

F(x)=x416−x22+1,F(x)=\frac{x^4}{16}-\frac{x^2}{2}+1 ,

so F(−2)=0F(-2)=0, F(1)=9/16F(1)=9/16 and F(2)=0F(2)=0. At x=2x=2, ff changes from negative to positive, and FF has a local minimum; at x=0x=0 it changes from positive to negative and FF has a local maximum F(0)=1F(0)=1. With Δx=1/2\Delta x=1/2 at x=1x=1, ΔF=F(1.5)−F(1)=−95/256\Delta F=F(1.5)-F(1)=-95/256 while f(1) Δx=−3/8=−96/256f(1)\,\Delta x=-3/8=-96/256.

Replay the worked derivation

Start the accumulator at a

F(a) = ∫ from a to a = 0 by definition.

F(−2)=0F(-2)=0
Replay: start at a, accumulate to x, add one strip, let it shrink.

Sign changes, moving limits and jumps

  • Where f>0f>0, FF increases; where f<0f<0, it decreases; where ff changes sign, FF has a local extremum.
  • Moving aa shifts FF by a constant (∫a′af\int_{a'}^{a}f). The slope F′(x)=f(x)F'(x)=f(x) does not depend on aa.
  • With x<ax<a, F(x)=−∫xafF(x)=-\int_x^a f: the accumulation runs backward.
  • For a jump in ff, FF is still continuous but has a corner; the first form of the theorem needs continuity at xx.

Executable reference code

The reference builds FF exactly with Fraction antiderivatives and checks F(x+h)−F(x)F(x+h)-F(x) against f(x) hf(x)\,h for shrinking hh. The NumPy version accumulates a sampled ff with the cumulative trapezoid rule and differentiates the result numerically, recovering ff up to discretisation error.

code/python/calculus_accumulation.py
"""The accumulation function F(x) = integral from a to x of f, built exactly; F'(x) = f(x) (III.13)."""
from fractions import Fraction as F

f = lambda t: t**3 / 4 - t
P = lambda t: t**4 / 16 - t**2 / 2          # an antiderivative of f
a = F(-2)
Fx = lambda x: P(x) - P(a)                  # F(a) = 0 by construction

assert (Fx(a), Fx(F(1)), Fx(F(2)), Fx(F(0))) == (0, F(9, 16), 0, 1)

# A thin strip adds nearly f(x)*h; the ratio tends to f(x) as h shrinks.
x = F(1)
gaps = [abs((Fx(x + h) - Fx(x)) / h - f(x)) for h in (F(1, 2**k) for k in range(4, 14))]   # small h: gap ~ |f'(x)| h/2
assert all(b < a_ for a_, b in zip(gaps, gaps[1:]))
assert Fx(x + F(1, 2)) - Fx(x) == F(-95, 256)

# A jump in f gives a corner in F: one-sided slopes 1 and -1 at t = 1.
step_F = lambda x: x if x < 1 else 2 - x    # integral from 0 of (1 for t < 1, -1 for t >= 1)
h = F(1, 1000)
assert (step_F(F(1)) - step_F(1 - h)) / h == 1 and (step_F(1 + h) - step_F(F(1))) / h == -1
code/python/calculus_accumulation_numpy.py
"""Cumulative trapezoid accumulation of a sampled f, then numerical differentiation recovers f (III.13)."""
import numpy as np

t = np.linspace(-2, 3, 2001)
f = t**3 / 4 - t
F = np.concatenate([[0.0], np.cumsum((f[1:] + f[:-1]) / 2 * np.diff(t))])   # F(-2) = 0
exact = t**4 / 16 - t**2 / 2 + 1
assert np.max(np.abs(F - exact)) < 1e-5
recovered = np.gradient(F, t)
assert np.max(np.abs(recovered[1:-1] - f[1:-1])) < 1e-4

Explore

Find the maximum and minimum of FF by dragging xx to the zeros of ff. Move aa and watch the whole curve FF shift vertically while its tangent at xx keeps the same slope. On the jump preset, set Δx\Delta x negative to see the left slope.

Common mistakes

Confusing F(x)F(x) (an accumulated amount) with f(x)f(x) (a rate). Thinking the lower limit affects F′F'. Applying F′=fF'=f at a discontinuity of ff. Forgetting that areas below the axis make FF decrease.

Numerical considerations

Cumulative numerical integration followed by numerical differentiation recovers ff only approximately: discretisation error from the first step and the step-size trade-off of III.14 in the second. Working with exact antiderivatives, when they exist, avoids both.

Engineering and ML connections

Cumulative quantities are everywhere: position from velocity, charge from current, a cumulative distribution function from a density, whose derivative recovers the density (IV.8). Prefix sums in computing are the discrete accumulation function, and their differences recover the original sequence.

Exercises

calculation

Evaluate an accumulation function

F(x) = ∫ from 0 to x of (2t + 1) dt. Give F(2) and F′(2).

calculation

One thin strip

With f(x) = 3 near x, estimate F(x + 0.01) − F(x).

calculation

Change the lower limit

G(x) = ∫ from 1 to x of 2t dt. Give G(1), G(3) and G′(3).

prediction

Where F peaks

f changes from positive to negative at x = 2. What does F(x) = ∫ₐˣ f do there?

Choose one answer

prediction

A jump in f

f jumps from 1 to −1 at x = 1. Is F differentiable at 1?

Choose one answer

Summary and checkpoints

The accumulation function F(x)=∫axfF(x)=\int_a^x f grows at the rate f(x)f(x): F′(x)=f(x)F'(x)=f(x) wherever ff is continuous. Changing aa shifts FF by a constant; a jump in ff makes a corner in FF. Next, III.14 turns to computing derivatives numerically, where finite precision shapes the answer.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available