arXiv cs.AIPaper
TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts
This one matters. TasteVal isolates experimental design skill from raw coding ability by having the model play researcher while a fixed coder executes. It measures how much compute a model needs to reach expert-level conclusions. That's a direct proxy for predicting AI research productivity, which is what everyone should care about in forecasting. The benchmark has eight frontier-grade tasks that are actually hard.