In plain words
In AI, entropy usually refers to information entropy. It measures how much uncertainty remains before an outcome is known. For a fixed set of possible outcomes, entropy is highest when all are equally likely and lowest when one is certain.
A closer look
Entropy averages the surprise of possible outcomes, weighted by their probabilities. With base-2 logarithms it is measured in bits: a fair coin has 1 bit of entropy, while a coin guaranteed to land heads has 0. A rare outcome carries more information when it happens, but entropy describes the whole distribution.
For a language model, next-token entropy describes how spread out its predicted probabilities are at a particular step. Several similarly likely continuations mean higher entropy; one dominant continuation means lower entropy. This measures uncertainty within that prediction, not whether the resulting answer is true.
Cross-entropy is related but different: it measures average surprise for outcomes from one distribution using probabilities assigned by another. As a training loss, it penalizes a model for assigning low probability to the observed target. It is not simply the entropy of the model’s own predictions.
In practice
An image classifier assigns “cat” and “dog” probabilities of 50% each: entropy is 1 bit. At 99% cat and 1% dog, entropy is about 0.08 bits. The second prediction is more concentrated, but it can still be wrong.
A useful distinction
Low entropy does not guarantee accuracy, and high entropy does not automatically mean poor performance: several outcomes may be valid. Here the term refers to information theory, rather than directly to heat or physical disorder.