Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection

## 📄 2604.22753v1 **作者**: Sijie Li, Shanda Li, Haowei Lin, Weiwei Sun, Ameet Talwalkar **分类**: cs.LG **发表**: 2026-04-24 ### 摘要 Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, assembling a sufficiently informative set of pilot experiments is already a major budget-allocation problem rather than a routine preprocessing step. We formulate scaling-law fitting as budget-aware sequential experimental design: given a finite pool of runnable experiments with heterogeneous costs, choose which runs to execute so as to maximize extrapolation accuracy in a high-cost target region. We then propose an uncertainty-aware method for sequentially allocating experimental budget toward the runs most useful for target-region extrapolation. Across a diverse benchmark of scaling-law tasks, our method consistently outperforms classical design-based baselines, and often approaches the performance of fitting on the full experimental set while using only about 10% of the total training budget. Our code is available at https://github.com/PlanarG/active-sl. 🔗 arXiv 论文页面 --- Scaling Law 一直是烧钱大赛，但拟合它本身也要花大钱？这篇论文说，用不确定性感知的方法从海量候选实验里精准挑出最有价值的10%，就能逼近全量实验的拟合效果——预算砍掉90%，精度不掉。这对大厂炼丹党和做高效实验设计的团队来说，绝对是值得放进工具箱的思路。

<< 返回首页