How does the next-token prediction become one scalar loss?

Intuition

At each position a model assigns scores to possible next tokens. The actual target selects one probability from that vocabulary distribution. Its negative log probability is the position’s loss. A sequence loss averages those actual losses, counting only the positions explicitly included.

Definition

Use vocabulary $V=\{\mathrm{the},\mathrm{cat},\mathrm{sat},\mathrm{eos}\}$. Position t has supplied logits $z_{t,i}$ and observed target index $y_t$. Its NLL is $L_t=-\log\operatorname{softmax}(\mathbf z_t)_{y_t}=\operatorname{LSE}(\mathbf z_t)-z_{t,y_t}$. If U is the set of unmasked positions, $\overline L=(1/|U|)\sum_{t\in U}L_t$ for $|U|>0$. When U is empty, mean loss is undefined.

Mechanics

The explicit vocabulary is the, cat, sat and <eos>. Position buttons select which actual vector and target to inspect. Numeric score controls and Target token update that selected position. Ignore selected position changes its explicit mask; the sequence table shows all individual losses and inclusion flags. The aggregate count, total and mean derive from those same rows.

Derivation

Stable log-softmax gives $\log p_{t,i}=(z_{t,i}-m_t)-\log\sum_j e^{z_{t,j}-m_t}$. Negating the target entry yields NLL directly from scores. A one-hot target has exactly one nonzero coefficient, so $-\sum_i\mathbf1\{i=y_t\}\log p_{t,i}=-\log p_{t,y_t}$. The table shows those actual one-hot terms. For the gradient bridge, differentiating $\log\sum_i e^{z_i}-z_y$ gives $\partial L/\partial z_j=p_j-\mathbf1\{j=y\}$; the chain rule connects logits to model parameters.

Worked examples

With four equal logits, every token has probability 1/4 and target NLL is $\log4\approx1.386294$ nats, or 2 bits. For logits (1000,0,−1000,−1000) and target <eos>, probability underflows to zero but NLL remains 2000 nats. If three position losses are (2,4,8) and the middle is masked, count=2 and mean=(2+8)/2=5 nats. An all-masked sequence has total zero and undefined mean.

Executable reference code

The standalone pedagogical implementation exposes the operations. Its NumPy companion uses array reductions and explicit domain/mask guards. Both include assertions for the hand-worked examples.

"""Four supplied vocabulary logits per position; explicit observed target and ignore mask."""
from math import exp,log,fsum,isfinite,inf
VOCABULARY=('the','cat','sat','<eos>')
def mean_loss(positions):
    if not 1<=len(positions)<=4:raise ValueError('Use 1–4 positions')
    losses=[];contributing=[]
    for logits,target,ignored in positions:
        if len(logits)!=4 or type(target) is not int or not 0<=target<4 or type(ignored) is not bool or any((not isfinite(z) and z!=-inf) or (isfinite(z) and abs(z)>1000) for z in logits):raise ValueError('Invalid position')
        m=max(logits)
        if m==-inf:raise ValueError('No normalizable scores')
        shifted=[z-m for z in logits]
        log_denominator=log(fsum(exp(z) for z in shifted))
        loss=log_denominator-shifted[target]
        losses.append(loss)
        if not ignored:contributing.append(loss)
    count=len(contributing);total=fsum(contributing)
    return losses,count,total/count if count else None
if __name__=='__main__':
    p=[([0,0,0,0],2,False),([1000,0,-1000,-1000],3,True)]
    assert mean_loss(p)==([log(4),2000],1,log(4))
    assert mean_loss([(z,y,True) for z,y,_ in p])[2] is None
    print(mean_loss(p))

"""Array token NLL from centered log-softmax; select contributing rows before summing."""
import numpy as np
VOCABULARY=('the','cat','sat','<eos>')
def mean_loss(logits,targets,ignored):
    z=np.asarray(logits,dtype=float);y=np.asarray(targets);mask=np.asarray(ignored)
    if z.ndim!=2 or z.shape[1]!=4 or not 1<=z.shape[0]<=4 or y.shape!=(z.shape[0],) or mask.shape!=y.shape or mask.dtype!=bool or not np.issubdtype(y.dtype,np.integer) or np.any((y<0)|(y>=4)) or np.any(np.isnan(z)) or np.any(z==np.inf) or np.any(np.isfinite(z)&(np.abs(z)>1000)):raise ValueError('Invalid token batch')
    maxima=z.max(axis=1,keepdims=True)
    if np.any(maxima==-np.inf):raise ValueError('All-excluded position')
    with np.errstate(over='ignore',under='ignore'):shifted=z-maxima;e=np.exp(shifted)
    logp=shifted-np.log(e.sum(axis=1,keepdims=True));losses=-logp[np.arange(y.size),y]
    used=losses[~mask];count=used.size
    return losses,count,float(used.sum()/count) if count else None
if __name__=='__main__':
    losses,count,mean=mean_loss([[0,0,0,0],[1000,0,-1000,-1000]],[2,3],[False,True])
    assert np.array_equal(losses,[np.log(4),2000]) and count==1 and mean==np.log(4)
    assert mean_loss([[0,0,0,0]],[0],[True])[2] is None
    print(losses,count,mean)

Explore

Change position and identify its different logits and target. Switch the observed target without changing the vector and inspect the changed one-hot term. Choose Tiny target probability, finite loss and compare probability zero with finite NLL. Ignore that position and verify the denominator decreases by exactly one. Mask every position, then include them again.

Common mistakes and counterexamples

The target need not be the token with highest predicted probability. Never take log of an underflowed probability when stable log-softmax is available. Masks are explicit; a large loss does not automatically disappear. Mean denominator is the contributing count, not sequence length or vocabulary size. An undefined empty mean is not zero loss.

Numerical considerations

This toy has four named vocabulary entries and one to four supplied positions, with explicit target indices and boolean masks. Invalid targets, vector shapes and undefined predictions are rejected even in ignored positions. Finite scores are bounded to [−1000,1000], with explicit −∞ exclusions allowed by the model. Scores and loss are approximate binary64 quantities. Individual unmasked infinite NLL propagates to the aggregate; masked infinities are skipped before summation rather than multiplied by zero. Stable logs are computed directly from logits, with independent raw Decimal400 references. This is a supplied-score explorer: it has no transformer, tokenizer, external API or network training engine.

Engineering and ML connections

Language-model training evaluates a target token at each contributing sequence position and reduces to a scalar. Given a differentiable score-producing model, the chain rule turns that scalar into parameter gradients. The accepted Optimization update then applies a learning rate to those gradients. This chapter exposes the loss mechanism without implementing a model or optimizer.

Connect to the gradient-descent update.

Exercises

Predict intermediate quantities, calculate small cases and explain numerical or mask boundaries.

Summary and checkpoints

At each position a model assigns scores to possible next tokens. The actual target selects one probability from that vocabulary distribution. Its negative log probability is the position’s loss. A sequence loss averages those actual losses, counting only the positions explicitly included. Read the actual pipeline, preserve the log information and state which inputs contribute.