B29
Compression, memory, and low-resource intelligence
At the same accuracy, how much memory and energy does each representation require?

IV.16 / AI
What happens to accuracy, memory use and response time when a model is quantized?
Efficient inference and learning involve concrete tradeoffs among memory, latency, energy, accuracy, and engineering complexity. GPTQ offers one studied example of post-training weight quantization. The relevant question is whether a change improves the intended workload on the actual hardware, including loading and transfer costs, while preserving the behavior that matters for that workload.
A billion weights stored at 16 bits require about two billion bytes for the weights alone. An idealized four-bit representation reduces that component to about half a billion bytes, before scales, metadata, and runtime state. Compare this estimate with measured process memory and latency; a storage reduction need not translate proportionally into faster execution.
Benchmark one efficiency change against an unchanged baseline with fixed prompts, hardware, and quality criteria, then report the full resource tradeoff.
Elias Frantar and coauthors · Post-training quantization as a memory and computation technique with evaluation tradeoffs.
B29
At the same accuracy, how much memory and energy does each representation require?