Agent framework choice shifts scores 27 points; long-horizon failures persist

A Stanford-led benchmark called DuMateBench found that using the same underlying model, Opus-4.8, different agent frameworks produced scores ranging from 0.5821 to 0.8548, a 27.27 percentage-point gap, showing that framework selection has a large independent effect on performance. Across multiple separate benchmarks, frontier models fail most long-horizon tasks: the Long-Horizon-Terminal-Bench found 17 frontier models averaged a 6.4% pass rate across 46 multi-step terminal tasks, with 10 of 17 solving zero under strict grading and the best reaching 28.3%. A paper testing eight leading models on year-long interconnected decisions with delayed feedback found the best setup, Qwen3.7-Max with Hermes, reached 27.3% of average human performance. WeaveBench, covering 114 hybrid GUI-CLI tasks, found the best pass rate at 41.2%. Qwen's E-Commerce Bench, covering a 365-day simulated online store management task, found almost no model learns to improve its strategies over the full period. Meta's ADeptS-Bench found no tested model consistently stayed above 80% task success while keeping attack success below 30%; all 7 tested models processed a $25K checkout without stopping. A paper found agent execution traces compress to finite-state machines of 7 to 43 states. Princeton's reliability research found that rising capability scores on long-horizon benchmarks have produced only small actual reliability improvements.