Overview:
Scaling laws have emerged as a central paradigm in understanding the behavior of large-scale machine learning models. Foundational work has shown that performance metrics such as loss, accuracy, or perplexity often follow predictable power-law relationships with respect to compute, dataset size, and model parameters. These results have played a crucial role in guiding the design and training of modern large language models (LLMs).
However, most existing scaling law research focuses on pre-training. Post-training methods — such as supervised fine-tuning, reinforcement learning from human feedback (RLHF), preference optimization, and instruction tuning — are far less understood from a scaling perspective. In practice, post-training is critical: it shapes model alignment, reasoning capabilities, and task specialization. Yet, unlike pre-training, the data regimes are smaller, compute efficiency differs, and objective functions are less standardized.
A systematic exploration of post-training scaling laws could therefore bridge a critical knowledge gap: How do alignment, generalization, and reasoning performance scale with post-training compute and data budgets? How do these laws differ from the well-studied pre-training regimes? Research Questions:
Potential research questions for theses lie at the intersection of scaling law theory, optimization, and alignment:
1) How do scaling laws differ between pre-training and post-training? For example, do loss improvements saturate faster in post-training due to smaller datasets?
2) What are the trade-offs between parameter count, dataset size, and optimization steps in post-training regimes such as RLHF or Direct Preference Optimization (DPO)?
3) How can token-based FLOP accounting be extended to post-training pipelines (rollouts, updates, LoRA adapters) to produce compute-optimal scaling curves? To what extent do different reward functions (e.g., correctness vs. preference-based) alter scaling exponents?
4) Can scaling law predictions help anticipate performance limits of alignment methods at larger model scales?
Prerequisites:
Work in this research area requires very good knowledge of deep learning and large language models. Familiarity with reinforcement learning, optimization, and applied statistics is recommended. Expect access to either High Performing Cluster or local GPUs. Expect good supervision from motivated PhD Students.
Start: Immediately
Contact: Patrick Wilhelm (patrick.wilhelm ∂ tu-berlin.de)
References: Youtube Quick Notes:
“Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws” https://www.youtube.com/watch?v=QJMUg5Uvb74
Stanford Lecture (1h) : https://www.youtube.com/watch?v=6Q-ESEmDf4Q
Lex Fridman and Dario Amodei (Antrophic AI) - 15min https://www.youtube.com/watch?v=GrloGdp5wdc
Base Paper:
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, et al., “Scaling Laws for Neural Language Models,” arXiv:2001.08361, 2020.
T. Henighan, J. Kaplan, M. Katz, M. Chen, et al., “Scaling Laws for Autoregressive Generative Modeling,” arXiv:2010.14701, 2020.
J. Hoffmann, S. Borgeaud, A. Mensch, et al., “Training Compute-Optimal Large Language Models,” arXiv:2203.15556, 2022.
E. Alabdulmohsin, K. Lee, E. B. Khalil, et al., “Scaling Laws for Fine-Tuning Language Models,” arXiv:2202.01169, 2022.
S. Wei, Y. Tay, R. Bommasani, et al., “Emergent Abilities of Large Language Models,” arXiv:2206.07682, 2022.
Y. Zhang, H. Bai, Y. Kong, et al., “Language Model Alignment with Scaling Laws,” arXiv:2309.12284, 2023.
R. Sorscher, D. Ganguli, J. Kaplan, S. McCandlish, et al., “Beyond Neural Scaling Laws: Beating Power Law Scaling via Data Pruning,” arXiv:2206.14486, 2022.
L. Gao, J. Schulman, J. Hilton, et al., “Scaling Laws for Reward Model Overoptimization,” arXiv:2303.15343, 2023.