What does a function do with one input?

Try it before reading on. Leave the rule at “subtract 2, then square” and move the input to $4$: the boxes read $4$, then $2$, then $4$. Now move the input to $0$. The output is the same, $4$. Two different inputs gave one output, and that is allowed. Then pick the rule $\sqrt{x+a}$ and drag the input to the left of $-3$: the machine stops at the square root and the point leaves the graph. By the end of the chapter you will know what a function is, how to find the inputs it accepts and the outputs it can give, how to chain two functions, and when a function can be run backwards.

Intuition: a machine with one slot

Think of a vending machine that takes one coin and gives one item. It does not matter how it works inside; what matters is the behaviour: put in the same coin and you get the same item, and there is never a coin that gives two items at once. A function is the mathematical version. It takes one input, performs a sequence of steps, and gives one output.

The picture of a pipeline is more than a metaphor. The rule $f(x)=(x-2)^{2}$ is not a mysterious symbol; it is two instructions in order: subtract $2$, then square. Evaluating $f(4)$ means carrying them out on $4$: $4-2=2$, then $2^{2}=4$. Reading a formula as a list of steps is the habit that this chapter builds, because every question about functions is a question about a step.

• What can go in? Some steps refuse some values: you cannot divide by $0$ or take the square root of a negative number. The inputs the whole machine accepts form its domain.
• What can come out? The outputs that really occur form the image.
• Which inputs give this output? Asking this runs the machine backwards. Sometimes there is one answer, sometimes two, sometimes none.
• What if I wire two machines together? The output of the first becomes the input of the second. The order matters.

This chapter uses the equations of the previous chapter (to find which inputs give an output you solve an equation, and to run a machine backwards you undo its steps in reverse order), the coordinate plane of the second chapter (the graph), and the squares and roots of the first.

Definition: function, domain, image, graph

Function. A function $f$ from a set $A$ to a set $B$ is a rule that assigns to every element $x$ of $A$ exactly one element $f(x)$ of $B$. We write $f\colon A\to B$ and $x\mapsto f(x)$, and read $f(x)$ as ”$f$ of $x$” (the output of $f$ for the input $x$). It is not the product of $f$ and $x$.

Domain and codomain. $A$ is the domain: the inputs the function is defined on. $B$ is the codomain: the set the outputs are drawn from. A function is not just a formula; the formula together with its domain is what makes it a function. When no domain is stated, we take the largest set of real numbers for which every step can be carried out.

Image. The image of $f$ is the set of outputs that actually occur, $\{f(x)\mid x\in A\}$. It can be smaller than the codomain. For $f(x)=x^{2}$ with codomain $\R$ the image is $[0,\infty)$, because no square is negative.

Graph. The graph of $f$ is the set of points $(x,f(x))$ in the plane. A curve is the graph of a function exactly when every vertical line meets it at most once (the vertical line test): a second point above the same $x$ would be a second output.

Preimage. For a number $h$, the inputs $x$ with $f(x)=h$ are the preimages of $h$. There can be none, one, or several. Finding them means solving the equation $f(x)=h$.

One-to-one and onto. $f$ is one-to-one (injective) when different inputs always give different outputs: no output has two preimages. $f$ is onto its codomain when every element of the codomain is an output: every output has at least one preimage. On a graph, “one-to-one” is the horizontal line test: every horizontal line meets the curve at most once.

Composition. For $g\colon A\to B$ and $f\colon B\to C$, the composite $f\circ g$ is the function $x\mapsto f(g(x))$: apply $g$ first, then $f$. Read it right to left.

Inverse. If $f$ is one-to-one with image $B$, its inverse $f^{-1}\colon B\to A$ sends each output back to the one input that produced it, so $f^{-1}(f(x))=x$ and $f(f^{-1}(y))=y$. The inverse exists exactly when $f$ is one-to-one onto its image. The notation $f^{-1}$ is not $1/f$.

Reading a rule as a pipeline

The stage at the top offers six rules, each one or two steps:

For $a=0$ the multiplier rules become the constant $0$ with domain $\R$ and image $\{0\}$.

The stage keeps every number exact: $\sqrt{2}$ stays $\sqrt{2}$ and $1/3$ stays $1/3$, and the inspector says whether a value is exact or a rounded decimal. One flat state, the rule, the number $a$ and the input $x$, produces everything you see: the boxes, the point, the dashed lines, the table and the verdict. Dragging the point changes only $x$, so the curve itself does not move; changing $a$ or the rule redraws it.

When a step refuses, the pipeline stops there. The boxes after the refusal are marked “not reached”, the point drops to the $x$-axis, and the verdict names the step and the reason. A function has no output at such an input; it is not “zero” and not “infinity”.

The shape of a shift

Compare $x^{2}$ with $(x-2)^{2}$. The second is the first with the input moved before squaring: the machine subtracts $2$, so the input must be $2$ larger to reach the same value. The graph moves to the right by $2$, which feels backwards at first because of the minus sign. Test with the output $0$: $(x-2)^{2}=0$ when $x=2$.

Domain: where the machine refuses

A step refuses an input when it has no answer. There are two refusals in this chapter:

• the reciprocal of $0$ (no number multiplied by $0$ gives $1$), and
• the square root of a negative number (no real number squared is negative).

The domain of the whole machine is the set of inputs that no step refuses. Two ways to find it:

1. Forwards: try an input and watch the pipeline, as the first stage does.
2. Backwards: start from the last step and ask what it needs. $\sqrt{x+3}$ needs $x+3\ge0$, that is $x\ge-3$; the set $[-3,\infty)$ contains $-3$ because $\sqrt{0}=0$. $\dfrac{1}{x-2}$ needs $x-2\neq0$, that is $x\neq2$. For a machine with two steps, pull the requirement of the last step back through the first.

The domain is the input side of the graph, the set of $x$ values that have a curve above or below them. At the edge of $\sqrt{x+3}$ the curve starts with a filled dot at $(-3,0)$; at $x=2$ the curve of $\dfrac{1}{x-2}$ has a gap, with a vertical dashed line (an asymptote) showing where it blows up.

A formula is not yet a function

The formulas $\dfrac{x^{2}-4}{x-2}$ and $x+2$ agree at every $x\neq2$, and at $x=2$ the first divides by $0$ while the second gives $4$. They are different functions because their domains differ. Cancelling the common factor $(x-2)$ is the same step that lost a root in the previous chapter: it is legal only where the factor is not $0$.

Domain, image and the inputs that give an output

Fix a height $h$ and ask: which inputs $x$ give $f(x)=h$? On the graph, draw the horizontal line at height $h$ and read the $x$ where it meets the curve.

With $(x-2)^{2}$ the line at $h=4$ meets the curve twice, at $x=0$ and $x=4$: both satisfy $(x-2)^{2}=4$, because $x-2=\pm2$. At $h=0$ it touches once, at $x=2$. At $h=-1$ it misses, since a square is never negative: the equation has no solution. The preimages come from solving, not from reading the picture: the model solves $(x-2)^{2}=h$ exactly as $x=2\pm\sqrt{h}$ for $h>0$, and the picture confirms it.

Three consequences follow directly.

• The image is the set of heights that the line meets. For $(x-2)^{2}$ it is $[0,\infty)$, the bar on the vertical axis.
• One-to-one fails when some height meets the curve twice. $(x-2)^{2}$ and $\lvert x\rvert$ are not one-to-one; $ax$ (with $a\neq0$), $\sqrt{x+a}$ and $\dfrac{1}{x-a}$ are.
• Onto $\R$ fails when some height is missed. The line $y=-1$ misses every curve whose image is $[0,\infty)$. The curve $\dfrac{1}{x-2}$ misses $y=0$ as well: $1$ divided by any number is never $0$, so the equation $\dfrac{1}{x-2}=0$ has no solution, and the image is $\R$ without $0$.

An important distinction: a function may send two inputs to one output (the horizontal line meets twice), but it may never send one input to two outputs (the vertical line would meet twice). The first is the case of $(x-2)^{2}$; the second is not a function at all.

Composition: one machine after another

Let $f(x)=x^{2}$ and $g(x)=2x-1$. The composite $f\circ g$ means first $g$, then $f$: $f(g(x))=(2x-1)^{2}$. The composite $g\circ f$ means first $f$, then $g$: $g(f(x))=2x^{2}-1$. At $x=3$ the first gives $g(3)=5$ and $f(5)=25$; the second gives $f(3)=9$ and $g(9)=17$. Composition is not commutative: the order is part of the answer. The stage starts at $x=0$, where $f(g(0))=(-1)^{2}=1$ but $g(f(0))=2\cdot0-1=-1$. The graph shows heights from $-6$ to $6$, so at $x=3$ the heights $25$ and $17$ are beyond its top edge: the stage then says so and gives the exact numbers beside the graph instead of drawing a dot.

At $x=1$ the two orders happen to agree (both give $1$). Setting $(2x-1)^{2}=2x^{2}-1$ and simplifying gives $2x^{2}-4x+2=0$, that is $2(x-1)^{2}=0$, so $x=1$ is the only input where they agree: agreement at one input does not make the two functions equal.

The domain of a composite

A composite can accept fewer inputs than its inner function. Take $f(x)=\sqrt{x}$ and $g(x)=2x-1$. The function $g$ accepts every real number, but $f\circ g$ needs the output of $g$ to be allowed in $f$: $2x-1\ge0$, so $x\ge\tfrac12$. The input $x=0$ is accepted by $g$, gives $-1$, and the square root refuses it. In the stage the composite $f\circ g$ at $0$ is marked undefined while $g\circ f$ is fine. The domain of a composite is the part of the inner function’s domain whose outputs the outer function accepts.

A second trap: $\sqrt{x^{2}}$ is not $x$. At $x=-3$ the square gives $9$ and the root gives $3$, so $\sqrt{x^{2}}=\lvert x\rvert$. Choose $f=\sqrt{x}$ and $g=x^{2}$ and set the input to $-3$ to see it.

Running a machine backwards: the inverse

To find the inverse of $y=2x-1$, solve for $x$: add $1$ to both sides, then divide by $2$. The inverse is $x=\dfrac{y+1}{2}$. Its steps are the original steps undone in the opposite order (the last step is undone first), and each undoing step is a balanced step from the previous chapter.

The graph of the inverse is the mirror image of the graph of $f$ in the line $y=x$. Swapping the roles of input and output swaps the coordinates of every point: if $(3,5)$ is on the graph of $2x-1$, then $(5,3)$ is on the graph of its inverse. In the stage the filled dot is $(x,y)$ and the hollow dot is its mirror image $(y,x)$.

Not every function has an inverse. On all of $\R$ the square sends $3$ and $-3$ to the same output $9$. A backward machine fed with $9$ would have to return both, and a function returns only one value. The mirror image of the parabola is a parabola on its side, which fails the vertical line test. The fix is to restrict the domain: if only $x\ge0$ is allowed, $3$ is the single preimage of $9$ and the inverse is the square root. A function has an inverse exactly when it is one-to-one (on the domain you chose), and the domain of the inverse is the image of the original. That is why restricting the inputs is not cheating: it changes the function.

Also look at $\sqrt{x}$ and $\dfrac{1}{x}$. The inverse of $\sqrt{x}$ is the square $y^{2}$ for $y\ge0$. The reciprocal is its own inverse: $\dfrac{1}{1/x}=x$, so its graph is its own mirror image in $y=x$.

Worked examples

Five complete solutions, each one a list of steps with the reason for each. They use only what this chapter defines, and each number you see is computed by the same model as the stages.

Executable reference code

Run the Python examples below to follow the calculations step by step. First, functions as values: evaluate, compose, tabulate and invert.

# A function is a rule that sends every input to exactly one output. In Python a function is also a value: it can be passed
# to another function, stored in a list, or returned.
def shifted_square(x):
    return (x - 2) ** 2  # subtract 2, then square


def square(x):
    return x**2


def line(x):
    return 2 * x - 1


def compose(outer, inner):
    """outer after inner: the output of `inner` becomes the input of `outer`."""
    return lambda x: outer(inner(x))


print(shifted_square(-1), shifted_square(5))  # 9 9: two different inputs may share one output
print(compose(square, line)(3), compose(line, square)(3))  # 25 17: the order of the machines matters

inputs = list(range(-3, 4))
table = [(x, shifted_square(x)) for x in inputs]
print(table)  # [(-3, 25), (-2, 16), (-1, 9), (0, 4), (1, 1), (2, 0), (3, 1)]
outputs = {y for _, y in table}
print(len(outputs), len(inputs))  # 6 7: fewer outputs than inputs, so the rule is not one-to-one and cannot be undone


# 2x - 1 multiplies by 2 and then subtracts 1. Its inverse undoes the steps in the opposite order: add 1, then halve.
def undo_line(y):
    return (y + 1) / 2


print(undo_line(line(3)))  # 3.0: the round trip returns the input

Then the domain. Python raises an exception exactly where the mathematics has no value, and NumPy returns nan instead, so the domain has to be written down.

import math

import numpy as np


# The domain of a function is the set of inputs for which the machine can finish. Python tells you the moment it cannot:
# a division by zero or a square root of a negative number raises an exception instead of returning a number.
def reciprocal_shift(x):
    return 1 / (x - 2)  # undefined where x - 2 = 0


def root_shift(x):
    return math.sqrt(x + 3)  # undefined where x + 3 < 0


def defined(f, x):
    try:
        f(x)
    except (ZeroDivisionError, ValueError):
        return False
    return True


print([defined(reciprocal_shift, x) for x in (0, 1, 2, 3)])  # [True, True, False, True]: only x = 2 is refused
print([defined(root_shift, x) for x in (-4, -3, 0)])  # [False, True, True]: the edge x = -3 belongs to the domain

try:
    math.sqrt(-1)
except ValueError as error:
    print(error)  # math domain error

# The domain of a composite is smaller than the domain of the inner function: sqrt(2x - 1) needs 2x - 1 >= 0, so x >= 1/2.
print([defined(lambda x: math.sqrt(2 * x - 1), x) for x in (0, 0.5, 1)])  # [False, True, True]

# NumPy applies the rule to a whole array. Invalid inputs become nan (not a number) rather than raising, so state the
# domain yourself with a mask, and say what to do outside it.
x = np.array([-4.0, -3.0, 0.0, 5.0])
inside = x + 3 >= 0
y = np.where(inside, np.sqrt(np.where(inside, x + 3, 0.0)), np.nan)
print(y.tolist())  # [nan, 0.0, 1.7320508075688772, 2.8284271247461903]
print(inside.tolist())  # [False, True, True, True]

Finally the machine-learning picture: a model is a function of its input and of its parameters, the loss is a function of the parameters, and a layer is a composition.

# A model is a function of an input x that also depends on parameters: here f(x; w, b) = w*x + b. Training chooses the parameters,
# so the LOSS is a second function, this time of (w, b), with the data held fixed.
def model(x, w, b):
    return w * x + b


data = [(0, 1), (1, 3), (2, 5), (3, 7)]  # these points lie exactly on y = 2x + 1


def loss(w, b):
    """Mean squared error of the model on the data, as a function of the parameters."""
    return sum((model(x, w, b) - y) ** 2 for x, y in data) / len(data)


print(loss(2, 1))  # 0.0: the right parameters
print(loss(1, 1))  # 3.5: errors 0, -1, -2, -3 give (0 + 1 + 4 + 9) / 4
print(loss(2, 0))  # 1.0: every prediction is 1 too low


# A neural-network layer is a composition: an affine map followed by an activation function.
def relu(z):
    return max(z, 0.0)


def make_layer(w, b):
    return lambda x: relu(model(x, w, b))


layer = make_layer(2.0, 1.0)
print(layer(-1.0), layer(2.0))  # 0.0 5.0: 2*(-1) + 1 = -1 is clipped to 0, and 2*2 + 1 = 5 passes through
print(relu(-3.0), relu(-1.0))  # 0.0 0.0: two inputs, one output, so relu cannot be undone

Two remarks on the first file. compose(outer, inner) returns a new function: in Python a function is an ordinary value, which is what makes composition a one-line definition. And len(outputs) < len(inputs) is a finite version of “not one-to-one”: if the number of distinct outputs is smaller than the number of inputs, two inputs must share an output.

Explore: change the parameters

These questions have definite answers; check them against the stages.

• On the first stage choose $(x-a)^{2}$ with $a=2$. Find two inputs between $-3$ and $7$ that give the output $9$. (They are $-1$ and $5$.)
• On the first stage choose $\sqrt{x+a}$ with $a=3$. At which step does the machine stop for $x=-4$, and what is the largest whole-number input it refuses?
• On the first stage choose $\dfrac{1}{x-a}$ with $a=2$. Which input is refused, and what happens to the output as the input gets close to $2$?
• On the height stage choose $(x-2)^{2}$. For which heights $h$ does the line meet the curve twice, once, and not at all?
• On the height stage choose $\lvert x-a\rvert$. Does changing $a$ change how many inputs a given positive height has?
• On the composition stage choose $f=x^{2}$ and $g=2x-1$. Slide the input and look for an input other than $1$ at which the two orders give the same value. Is there one? (Solve $(2x-1)^{2}=2x^{2}-1$ to check.)
• On the composition stage choose $f=\sqrt{x}$ and $g=x^{2}$ and set $x=-3$. What does the composite $f\circ g$ give, and which familiar function is it?
• On the inverse stage choose $x^{2}$ with every real input allowed and the input $2$. Which other input has the same output, and where is its dot on the graph? Then restrict the inputs to $x\ge0$ and see what the mirror image becomes.

Common mistakes

Numerical considerations

The mathematics above is exact. A program computes with doubles, and a composition of exact inverses is not always the identity in doubles.

Squaring a square root. $\bigl(\sqrt{x}\bigr)^{2}$ is exactly $x$ for every non-negative $x$. In doubles, math.sqrt(2) ** 2 is 2.0000000000000004, because the root is rounded to the nearest double before it is squared. The reciprocal of a reciprocal is rounded twice: 1 / (1 / 49) is 49.00000000000001. And near a pole the cancellation in $x-2$ turns a tiny rounding error into a huge relative error. The table feeds the machine $2+10^{-k}$ and prints the exact answer, which is $10^{k}$, next to the double the program holds.

The domain does not announce itself in NumPy. math.sqrt(-1) raises a ValueError, but np.sqrt(-1.0) returns nan with only a warning, and nan then flows through every later step. The reference file masks the invalid inputs explicitly and says what to put there. A nan in a loss is nearly always a domain error upstream: a sqrt, a log or a division that received an input outside its domain.

Inverses are only as exact as the data. Undoing a transformation with its inverse, such as unstandardising a feature, can differ from the original in the last digit; compare with a tolerance (math.isclose), not ==.

Connections to engineering and machine learning

• A model is a function of its input and its parameters. A linear model $f(x;w,b)=wx+b$ takes an input $x$ and has parameters $w$ and $b$. Training does not change $x$; it chooses $(w,b)$. The reference file computes the loss as a second function, now of the parameters, with the data held fixed: the loss is $0$ at $(2,1)$, $3.5$ at $(1,1)$ and $1$ at $(2,0)$.
• A network is a composition. A layer applies an affine map and then an activation function, and a deep network composes many layers. The order matters for the reason in this chapter: $g\circ f$ and $f\circ g$ are different functions.
• Activations and invertibility. The ReLU function, $\max(z,0)$, sends all negative numbers to $0$, so it is not one-to-one and cannot be undone: information is lost. The reference file shows relu(-3.0) and relu(-1.0) both giving 0.0.
• Domains in code. The functions $\sqrt{x}$, $\log x$ and $1/x$ all have restricted domains. Machine learning code guards them (np.maximum(x, eps), masks, clipped probabilities) because a single input outside the domain turns into nan or inf.
• Undoing a transformation. Standardising a feature is an invertible function of the data. Its inverse exists when the scale is not $0$, which is the restriction that made the earlier multiplication by $0$ impossible to undo. Likewise a probability can be mapped to an unbounded score and back by a pair of inverse functions, on the restricted domain where both are defined.
• Looking ahead. The next chapter surveys the families that appear again and again (lines, powers, exponentials, logarithms and more), and each of them has a domain, an image and, sometimes, an inverse.

Exercises

Answer each question and press Check. A wrong answer tells you which mistake it looks like. After one attempt you may open a worked solution.

Summary and checkpoints

• A function assigns to every input in its domain exactly one output. A formula without a domain is not yet a function; the domain is the set of inputs for which every step can be carried out.
• Read a rule as a pipeline of steps. The two refusals are the reciprocal of $0$ and the square root of a negative number; an input that is refused has no output.
• The image is the set of outputs that occur. To find the inputs that give an output $h$, solve $f(x)=h$: there may be none, one or several.
• One input never gives two outputs; two inputs may share one. One-to-one means no output is shared (horizontal line test).
• In $f\circ g$ the function $g$ acts first. Composition is not commutative, and the domain of a composite can be smaller than the domain of the inner function.
• An inverse undoes the steps in the opposite order. It exists exactly when the function is one-to-one; restricting the domain can create one (the square root for $x^{2}$ on $x\ge0$). Its graph is the mirror image in $y=x$. It is not $1/f$.
• In code: write the domain down, do not compare a computed value with a pole using ==, and remember that NumPy turns domain errors into nan silently.

Checkpoints. You are ready to move on when you can, without looking:

1. Evaluate $(x-3)^{2}$ at $x=-1$ step by step, and say what the rule does to the input first.
2. State the domain of $\sqrt{x+3}$ and of $\dfrac{1}{x-2}$, and say which step refuses.
3. List the inputs that give the output $9$ under $(x-2)^{2}$, and the image of $-2x^{2}$.
4. Compute $f(g(3))$ and $g(f(3))$ for $f=x^{2}$ and $g=2x-1$, and say which acts first in each.
5. Explain why $x^{2}$ has no inverse on all of $\R$ and what restriction gives it one.