Mathematical Foundations

III.12 · Calculus

How do thin pieces add up to a total?

Slice an interval into panels, multiply each height by its width, and add. Refine the slices and the sum settles on the integral — signed accumulation, which is not the same as area.

Accumulate signed area panel by panel

Height of each panel

left endpoints under f(x) = 0.25x³ − x

Graph of f(x) = 0.25x³ − x from a = −2 to b = 3 with 10 panels using left endpoints. Green panels add, gold panels subtract. The sum is 0.703125; the integral is 1.5625.

Lower limit
−2
Upper limit
3
Panels
10
Panel width (signed)
0.5
Sum with left endpoints
0.703125
Integral (signed accumulation)
1.5625
Error of the sum
−0.859375
Geometric area between graph and axis
3.5625
Area above the axis
2.5625
Signed area below the axis
−1

Exact values use =; binary64 approximations use ≈. Undefined values state their reason.

Exceptional case: The graph crosses the axis: the integral 1.5625 is SIGNED accumulation (above minus below), while the geometric area is 3.5625. The sum 0.703125 approximates the signed value.

Δx=b−an=3−(−2)10=0.5\Delta x=\frac{b-a}{n}=\frac{3-(-2)}{10}=0.5
Sn=∑i=0n−1f(ξi) ΔxS_n=\sum_{i=0}^{n-1}f(\xi_i)\,\Delta x
Sn=0.703125S_n=0.703125
∫−23f(x) dx=1.5625\int_{-2}^{3} f(x)\,dx=1.5625
Sn−∫=−0.859375S_n-\int=-0.859375
∫∣f∣=3.5625\int|f|=3.5625
∫f+=2.5625,∫f−=−1\int f^+=2.5625,\quad \int f^-=-1
left endpoints: the sum and its error as the partition is refined
nΔxsumsum − integral
150−1.5625
22.5−1.171875−2.734375
41.25−0.29296875−1.85546875
80.6250.5126953125−1.0498046875
160.31251.007080078125−0.555419921875
320.156251.27716064453125−0.28533935546875
640.0781251.4179229736328125−0.1445770263671875

Panel 1: [−2, −1.5]

The sampled height times the signed width Δx gives the contribution 0; a negative height or a negative Δx makes it negative.

panel 1: 0\text{panel } 1:\ 0

from −3 to 3, up to 2 decimal places

from −3 to 3, up to 2 decimal places

from 1 to 64, whole numbers only

Drag the limits a and b on the axis, change the number of panels n, and choose how each panel's height is sampled. Green panels add; gold panels subtract.

The opening function x3/4−xx^3/4-x is positive on part of [−2,3][-2,3] and negative on another part. With ten left-endpoint panels the sum is 0.7031250.703125, while the integral is 25/16=1.562525/16=1.5625. The geometric area between the graph and the axis is larger still, 57/1657/16: the integral subtracts what lies below the axis. Increase nn and watch the table: the sums approach 25/1625/16.

Intuition: rate times time, piece by piece

If a speed f(t)f(t) were constant, distance would be speed × time. When it changes, chop time into short intervals on which it is nearly constant, use one sampled speed per interval, and add the products. A negative speed (moving backward) subtracts distance. The total is a signed accumulation; refining the chopping makes it exact in the limit.

Definition

Partition [a,b][a,b] into nn panels of width Δx=(b−a)/n\Delta x=(b-a)/n with points xi=a+i Δxx_i=a+i\,\Delta x. A Riemann sum chooses a sample ξi\xi_i in each panel:

Sn=∑i=0n−1f(ξi) Δx.S_n=\sum_{i=0}^{n-1}f(\xi_i)\,\Delta x .

The left, right and midpoint sums use ξi=xi\xi_i=x_i, xi+1x_{i+1} or the centre; the trapezoid sum averages the two end heights. For continuous ff, all of them converge to the same limit, the definite integral

∫abf(x) dx=lim⁡n→∞Sn.\int_a^b f(x)\,dx=\lim_{n\to\infty}S_n .

By the same formula, reversing the limits reverses the sign of Δx\Delta x: ∫baf=−∫abf\int_b^a f=-\int_a^b f, and ∫aaf=0\int_a^a f=0.

Mechanics: what the explorer computes

For the polynomial integrands every sum is computed exactly with rational arithmetic, and the integral comes from an exact antiderivative; their difference is the exact error. The positive part, the negative part and the geometric area ∫∣f∣\int|f| are obtained by splitting at the exact rational roots. For sin⁡x\sin x the values are binary64 and marked ≈\approx. Each drawn panel is the panel used in the sum, with the same height and width.

Derivation: signed area and geometric area

Split [a,b][a,b] at the points where ff changes sign. On a subinterval where f≥0f\ge0, the integral is the area under the graph; where f≤0f\le0, it is minus the area above the graph. Hence

∫abf=(area above)−(area below),\int_a^b f=\text{(area above)}-\text{(area below)}, ∫ab∣f∣=(area above)+(area below).\int_a^b|f|=\text{(area above)}+\text{(area below)}.

For x3/4−xx^3/4-x on [−2,3][-2,3] the roots are −2-2, 00 and 22: the area above is 1+25/16=41/161+25/16=41/16, the area below is 11, so the integral is 25/1625/16 and the geometric area 57/1657/16.

Worked example

f(x)=x3/4−xf(x)=x^3/4-x on [−2,3][-2,3] with n=10n=10 left panels: Δx=1/2\Delta x=1/2, samples −2,−1.5,…,2.5-2,-1.5,\dots,2.5. The exact sum is 45/6445/64. The right sum is 165/64165/64, the midpoint sum 195/128195/128 and the trapezoid sum 105/64105/64. With n=64n=64 the midpoint and trapezoid sums are within 0.0020.002 of 25/1625/16, while the left and right sums are still off by about 0.150.15: the sample point matters for the rate.

Replay the worked derivation

Panel 1: [−2, −1.5]

The sampled height times the signed width Δx gives the contribution 0; a negative height or a negative Δx makes it negative.

panel 1: 0\text{panel } 1:\ 0
Replay: each panel's signed contribution, then the total compared with the integral.

Exceptional cases

  • Exact cancellation. For x−1x-1 on [0,2][0,2] the integral is exactly 00 although the area is 11.
  • Reversed limits. With b<ab<a, every Δx\Delta x is negative and the sum changes sign.
  • Non-integrable behaviour. A function with a pole inside the interval, such as 1/x1/x on [−1,1][-1,1], has no Riemann integral at all; sample points can land arbitrarily close to the pole and the sums do not settle. The explorer’s integrands are bounded on the whole window.

Executable reference code

The reference computes exact left, right, midpoint and trapezoid sums with Fraction and compares them with the antiderivative. The NumPy version vectorises the sums for increasing nn and shows the errors shrinking at the expected rates.

code/python/calculus_riemann.py
"""Exact Riemann and trapezoid sums with Fractions; signed integral versus geometric area (III.12)."""
from fractions import Fraction as F

f = lambda x: x**3 / 4 - x
antiderivative = lambda x: x**4 / 16 - x**2 / 2


def riemann(f, a, b, n, rule):
    dx = (b - a) / n
    xs = [a + i * dx for i in range(n + 1)]
    if rule == "left":
        return sum(f(x) for x in xs[:-1]) * dx
    if rule == "right":
        return sum(f(x) for x in xs[1:]) * dx
    if rule == "midpoint":
        return sum(f((x0 + x1) / 2) for x0, x1 in zip(xs, xs[1:])) * dx
    return sum((f(x0) + f(x1)) / 2 for x0, x1 in zip(xs, xs[1:])) * dx      # trapezoid


a, b = F(-2), F(3)
exact = antiderivative(b) - antiderivative(a)
assert exact == F(25, 16)
assert [riemann(f, a, b, 10, r) for r in ("left", "right", "midpoint", "trapezoid")] == [F(45, 64), F(165, 64), F(195, 128), F(105, 64)]

# Geometric area: split at the roots -2, 0, 2 and add absolute values.
pieces = [(F(-2), F(0)), (F(0), F(2)), (F(2), F(3))]
area = sum(abs(antiderivative(q) - antiderivative(p)) for p, q in pieces)
assert area == F(57, 16) and area != exact

# Reversing the limits reverses the sign (negative dx).
assert riemann(f, b, a, 10, "left") == -riemann(f, a, b, 10, "right")
code/python/calculus_riemann_numpy.py
"""Vectorised Riemann sums: left/right errors shrink like 1/n, midpoint/trapezoid like 1/n**2 (III.12)."""
import numpy as np

f = lambda x: x**3 / 4 - x
a, b, exact = -2.0, 3.0, 25 / 16
errors = {}
for n in (16, 32, 64, 128):
    x = np.linspace(a, b, n + 1)
    dx = (b - a) / n
    errors[n] = {
        "left": abs(np.sum(f(x[:-1])) * dx - exact),
        "midpoint": abs(np.sum(f((x[:-1] + x[1:]) / 2)) * dx - exact),
        "trapezoid": abs(np.trapezoid(f(x), x) - exact),
    }
assert 1.8 < errors[64]["left"] / errors[128]["left"] < 2.2
assert 3.8 < errors[64]["trapezoid"] / errors[128]["trapezoid"] < 4.2
assert 3.8 < errors[64]["midpoint"] / errors[128]["midpoint"] < 4.2

Explore

On Positive: 4 − x², the integral and the area agree. Switch rules on the same nn: left and right sums bracket the integral when ff is monotone on each panel. Drag aa past bb and read the sign change. On sin x, compare midpoint and trapezoid errors.

Common mistakes

Treating the integral as area when the graph dips below the axis. Forgetting that Δx\Delta x is negative when b<ab<a. Using nn panels but n+1n+1 heights in a left or right sum (the trapezoid uses n+1n+1 points, weighted).

Numerical considerations

Floating-point sums of many small terms accumulate rounding error; summing in exact arithmetic avoids it, and in binary64 the order of summation (or compensated summation) matters for very large nn. Riemann sums converge slowly — the left and right rules have error proportional to Δx\Delta x — which is why III.15 studies better rules.

Engineering and ML connections

Integrals compute totals from rates: energy from power, distance from velocity, probability from a density (Part IV), expected values as weighted sums. Discretised integrals appear in every simulation and in numerical estimates of areas under ROC curves.

Exercises

calculation

A left Riemann sum

f(x) = x² on [0, 2] with n = 2 panels. Give the left-endpoint sum.

calculation

Signed accumulation

Compute ∫ from −1 to 2 of x dx and the geometric area between y = x and the axis on [−1, 2].

calculation

Reverse the limits

∫ from 0 to 3 of f dx = 5. What is ∫ from 3 to 0 of f dx?

prediction

Zero integral, nonzero area

∫ from 0 to 2π of sin x dx = 0. Does the graph enclose no area?

Choose one answer

prediction

More panels

For a continuous f on [a, b], what happens to Riemann sums as n → ∞?

Choose one answer

Summary and checkpoints

A Riemann sum multiplies sampled heights by panel widths and adds; its limit is the definite integral, a signed accumulation. Area above the axis adds, area below subtracts; reversing the limits reverses the sign. Next, III.13 lets the upper limit move and discovers that the rate of accumulation is the integrand itself.

The map of the book

97 of 97 chapters written · eight Parts

demo Foundation demo Try the live vector example Open as a page

Part I

Mathematical Language

The notation and mental objects everything else uses.

9 of 9 chapters written

  1. I.1 Scalars, quantities, units, powers and roots available
  2. I.2 Coordinates and points available
  3. I.3 Vectors: components, magnitude, and direction reference chapter available
  4. I.4 Algebraic expressions, equations and inequalities available
  5. I.5 Functions as input → transformation → output available
  6. I.6 Essential function families available
  7. I.7 Sets, intervals, and relations available
  8. I.8 Indexed quantities and summation available
  9. I.9 The notation map for ML available

Part II

Linear Algebra

Matrices as transformations: the strongest foundation in the book.

19 of 19 chapters written

  1. II.1 Vector addition and subtraction available
  2. II.2 Scalar multiplication and vector scaling available
  3. II.3 Norms, magnitude, and normalization available
  4. II.4 Dot product, angle, and orthogonality available
  5. II.5 Projection and distance to a direction available
  6. II.6 Linear combinations and span available
  7. II.7 Linear independence, basis, and coordinates available
  8. II.8 Matrices as linear transformations available
  9. II.9 Matrix × vector available
  10. II.10 Matrix × matrix as composition available
  11. II.11 Transpose and orthogonality available
  12. II.12 Determinant: orientation and area/volume scaling available
  13. II.13 Linear systems as intersecting constraints available
  14. II.14 Gaussian elimination and row operations available
  15. II.15 Rank and invertibility available
  16. II.16 Inverse matrices available
  17. II.17 Eigenvalues and eigenvectors available
  18. II.18 Orthogonal bases and least-squares geometry available
  19. II.19 Singular Value Decomposition available

Part III

Calculus

Change, local approximation, and sensitivity made visible.

15 of 15 chapters written

  1. III.1 Limits through approach available
  2. III.2 Continuity and local behavior available
  3. III.3 Derivative as secant → tangent available
  4. III.4 Derivative as rate of change and local linear model available
  5. III.5 Derivative rules available
  6. III.6 Chain rule available
  7. III.7 Multivariable functions and partial derivatives available
  8. III.8 Gradient and directional derivatives available
  9. III.9 Jacobian available
  10. III.10 Computational graphs and automatic differentiation available
  11. III.11 Hessian and curvature available
  12. III.12 Integrals as accumulation available
  13. III.13 Fundamental derivative/integral connection available
  14. III.14 Numerical differentiation available
  15. III.15 Numerical integration available

Part IV

Probability

Uncertainty through sample spaces, evidence, and repetition.

16 of 16 chapters written

  1. IV.1 Random experiments, outcomes, and sample spaces available
  2. IV.2 Events and probability mass available
  3. IV.3 Counting and finite probability where needed available
  4. IV.4 Conditional probability as restricted sample space available
  5. IV.5 Independence available
  6. IV.6 Bayes' theorem as probability-mass update available
  7. IV.7 Random variables available
  8. IV.8 PMF, PDF, and CDF available
  9. IV.9 Bernoulli and Binomial distributions available
  10. IV.10 Uniform and Gaussian distributions available
  11. IV.11 Expectation available
  12. IV.12 Variance and standard deviation available
  13. IV.13 Joint, marginal, and conditional distributions available
  14. IV.14 Covariance and correlation available
  15. IV.15 Law of Large Numbers available
  16. IV.16 Central Limit Theorem available

Part V

Statistics

From observed data to estimates, intervals, and likelihood.

10 of 10 chapters written

  1. V.1 Data summaries: mean, median, mode available
  2. V.2 Spread: variance, standard deviation, robust interpretation available
  3. V.3 Population, sample, and sampling variability available
  4. V.4 Estimators and estimation available
  5. V.5 Confidence intervals through repeated sampling available
  6. V.6 Covariance matrices and multivariate spread available
  7. V.7 Regression fundamentals available
  8. V.8 Likelihood available
  9. V.9 Maximum Likelihood Estimation available
  10. V.10 Maximum A Posteriori estimation available

Part VI

Optimization

Learning as repeated parameter updates governed by an objective.

10 of 10 chapters written

  1. VI.1 Objective functions and optimization landscapes available
  2. VI.2 Local vs global optima and stationary points available
  3. VI.3 Convexity available
  4. VI.4 Gradient descent flagship chapter available
  5. VI.5 Learning rate and convergence behavior available
  6. VI.6 Momentum available
  7. VI.7 Stochastic and mini-batch gradient descent available
  8. VI.8 Adam intuition available
  9. VI.9 Constraints and projected/penalized intuition available
  10. VI.10 Numerical stopping and optimization diagnostics available

Part VII

Information Theory

Logarithms, entropy, and the loss behind token prediction.

8 of 8 chapters written

  1. VII.1 Logarithms and log probabilities in computation available
  2. VII.2 Self-information available
  3. VII.3 Entropy available
  4. VII.4 Cross entropy and negative log-likelihood available
  5. VII.5 KL divergence available
  6. VII.6 Logits and softmax available
  7. VII.7 Numerical stability: log-sum-exp and stable softmax available
  8. VII.8 From token logits to language-model loss available

Part VIII

Numerical Computing

Why equivalent expressions behave differently on real machines.

10 of 10 chapters written

  1. VIII.1 Floating-point representation available
  2. VIII.2 Rounding and representation error available
  3. VIII.3 Error propagation and conditioning available
  4. VIII.4 Catastrophic cancellation available
  5. VIII.5 Overflow and underflow available
  6. VIII.6 Stable algorithms available
  7. VIII.7 Numerical linear algebra realities available
  8. VIII.8 Sparse matrices and sparse structure available
  9. VIII.9 Vectorized computation available
  10. VIII.10 Reproducibility, random seeds, and nondeterminism available