Shannon Scaling Law: LLMs Hit a Noisy Channel Ceiling
The Shannon Scaling Law, grounded in the Shannon-Hartley theorem, unifies monotonic and non-monotonic scaling phenomena. It predicts a hard ceiling on model performance from noisy channel capacity, directly challenging the compute-optimal scaling assumptions of leading AI labs.











