arXiv cs.CLPaper
Scaling Laws for Looped Mixture of Experts
This matters for anyone building or selecting foundation models under resource limits. Looping and sparsity are complementary, and having predictive laws for their interaction means smarter model-size decisions. If you're training at scale, this changes how you think about the depth-versus-breadth tradeoff.