FRI: Experts Underestimated AI Progress, but Methodology Favors Finding Underestimates

The Forecasting Research Institute reviewed AI progress forecasts from its own studies spanning mid-2022 to August 2026 and found that experts and superforecasters consistently underestimated AI capability gains and lab revenues, while overestimating biorisk uplift from AI. FRI drew on multiple studies including the XPT, an AI adversarial collaboration, an LLM-biorisk study, an AI-cyber risk pilot, and its Longitudinal Expert AI Panel (LEAP). On specific cases: superforecasters assigned only a 1.7% probability to a Millennium Prize being solved as soon as it was; experts gave 4.5% and had a median prediction of 2040. On revenue, AI 2027 had predicted the top AI lab to have a $2 trillion valuation and $38 billion per year in revenue by late 2026; one analysis notes Anthropic's IPO is said to be valued at $2 trillion and its annualized revenue topped $65 billion by late July 2026, with a separate report suggesting it has already passed $100 billion. Self-driving cars are a partial counterexample, representing roughly 0.6% of ride-share trips, slower diffusion than many had forecast. FRI itself acknowledged that its methodology is structurally biased toward surfacing underestimates: a median forecast is revealed as too low the moment real-world performance crosses it, whereas an overestimate can only be confirmed after the full resolution date passes. FRI also noted that the vast majority of its AI progress forecasts remain unresolved, meaning the overall accuracy picture could shift substantially over time.