arXiv cs.CLPaperClaude Watch
SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction
The coding-agent benchmark gap just closed. Real task horizons, real multi-turn interaction, and user personas mined from actual logs reveal what existing benchmarks hide: coding assistants work well for architects but poorly for non-technical users. If you're shipping Claude Code or similar tools, this exposes the interaction patterns that matter. The weak-to-strong synthesis pipeline for task generation is reusable.