In plain words
Training compute is the amount of computation spent adjusting a model’s parameters during training. It is often estimated in floating-point operations, or FLOPs.
A closer look
Compute depends on factors such as model size, the amount of training data processed, and the training method. Hardware speed and utilization affect how long that computation takes.
A compute budget can be allocated in different ways. More computation can support better performance, but data quality, architecture, and training choices also matter.
In practice
Two training runs can perform similar amounts of computation but take different lengths of time because they use different hardware or use it with different efficiency.
A useful distinction
Training compute is not the same as parameter count, electricity use, or price. FLOPs measure total operations; FLOPs per second measure computational speed. Inference compute is spent running the trained model.