B14
Statistical physics and learning
Under what assumptions can a learning problem be expressed as a statistical-mechanical model?

IV.11 / AI
At a fixed training budget, how does allocating computation between examples and parameters affect loss?
Scaling-law studies fit relationships between model performance and resources over measured ranges. Kaplan and colleagues and the later Chinchilla study provide distinct experiments and compute-allocation conclusions. Their comparison is useful because the resource budget is not a single number: parameters, training tokens, optimization, and evaluation distributions all matter. An empirical curve should retain the conditions under which it was fitted.
Imagine a fixed training budget divided between a larger model with fewer token exposures and a smaller model trained on more data. Plot a hypothetical loss for each allocation and label the measured region separately from any extrapolation. The decision depends on the fitted relationship and the intended evaluation, not parameter count alone.
Recreate a small comparison table of the two papers' experimental assumptions and identify which changed assumption matters most for their allocation result.
Jared Kaplan and coauthors · Empirical language-model loss scaling under specified experiments.
Jordan Hoffmann and coauthors · Training-compute allocation across parameter and token counts.
B14
Under what assumptions can a learning problem be expressed as a statistical-mechanical model?