PoEM: Predicting RL Outcomes from Existing Policies
If this holds, it's huge: skip the RL loop and interpolate in log-policy space. The paper shows it works when rewards are linearly related and often when they're not. The implication is immediate for anyone training multiple reward models or iterating on alignment objectives. Verify this works on your actual reward setup before trusting it, but the signal is strong enough to test immediately.