The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

agents162 research126 infra70 claude69 capital49 labs43 culture39 safety38 openai33 regulation30 anthropic24 security24 reasoning19 evaluation18 robotics13 frontier-models12

arXiv cs.CLPaper15h ago

Training-Free On-Policy Distillation

Clean and potentially high-impact for teams managing multiple fine-tuned models. If you have separate specialists for math and code, you can now fold them without gradient steps. The catch is that rollout quality matters more than the solver, so garbage specialist behavior spreads to the merged model. Test this on your specialist stack; if it works, you cut serving costs and latency. If it doesn't, the reason is instructive.

training distillation model-merging

arXiv cs.CLPaper12d ago

Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective

The mechanism matters more than the headline. OPD doesn't expand student capabilities; it reshapes the reward landscape, which can backfire into reward hacking when the teacher rewards behaviors it rarely exhibits. If you're using OPD in production post-training, the real lever is not the distillation itself but cleaning your implicit reward signal. Masking bad outputs during training or using SFT initialization are concrete fixes.

research post-training distillation reward-modeling