WebVoyager, WebArena, Mind2Web: Triangulating Agent Reality
WebVoyager top-agent 85%, WebArena 65%, Mind2Web 60-70%. Spread is informative: WebVoyager favors short-task skills, WebArena tests long reasoning, Mind2Web tests cross-site generalization. Triangulating gives the most honest capability read.
The three most-cited browser-agent benchmarks in 2026 are WebVoyager, WebArena, and Mind2Web. The headline scores for top agents differ meaningfully: WebVoyager top scores reach 85%+, WebArena 65-70%, Mind2Web 60-70%. The spread is not noise — it reflects that each benchmark measures structurally different capabilities. A buyer reading any one in isolation gets a distorted picture. Reading all three together produces the most honest capability assessment available.
The triangulation matters because vendor marketing predictably cites whichever benchmark produces the most favorable score. An agent’s “85%” claim usually comes from WebVoyager; the same agent on WebArena might score 60%. Neither number is wrong; they measure different things. The buyer who triangulates avoids being misled by selective benchmark citation.
What each benchmark measures
WebVoyager evaluates agents on real-world web tasks across ~15 popular sites (Amazon, Apple, BBC News, Google Maps, etc.). Tasks are typically short (3-5 steps), single-site, and goal-oriented (“find the cheapest 16GB iPhone case under $20 on Amazon”). The benchmark rewards strong screenshot-to-action mapping and good single-task focus. Top-agent scores hit 85%+ because the task distribution plays to current agents’ strengths.
WebArena tests agents in a containerized environment that simulates a fully-featured shopping site, social network, and CMS. Tasks are longer (5-15 steps), require state management across multiple page transitions, and often involve form-filling, multi-criteria filtering, and resource modification. Top scores at 65-70% reflect the harder cognitive load — agents that ace WebVoyager regularly fail WebArena’s long-horizon tasks.
Mind2Web evaluates agents on a curated set of ~2,000 tasks across 137 real websites. The diversity emphasizes generalization — an agent that learned to handle Amazon’s checkout has to generalize to checkout on 50+ other e-commerce sites. Tasks vary in length and complexity. Top scores at 60-70% reflect the cross-site generalization difficulty.
The three together cover most of the capability dimensions that matter for production agent deployment: short-task focus (WebVoyager), long-horizon reasoning (WebArena), and cross-site generalization (Mind2Web).
What the spread reveals
When the same agent gets dramatically different scores on the three benchmarks, the spread is informative about the agent’s actual capability profile.
Agent A scoring 85% / 50% / 70% has strong single-task execution but weak long-horizon reasoning. Useful for short browse-and-extract workflows; risky for multi-step workflows.
Agent B scoring 75% / 70% / 55% has balanced capability but weak cross-site generalization. Useful when the production target set is small and well-known; risky for broad-target deployments.
Agent C scoring 70% / 65% / 75% generalizes well but lacks raw execution capability. Useful as a research/exploration agent; less useful for production extraction workflows.
The capability-profile diagnosis is more useful than the single score. Vendors that publish all three scores are doing the buyer a service; vendors that publish only their best score are obscuring information.
The cost-tier interaction
Each benchmark behaves differently when the agent is constrained to cheap models.
WebVoyager scores tolerate cheap models well. Flash-tier and small reasoning models score within 5-10 points of their reasoning-tier equivalents on WebVoyager. The short task length means the agent does not need to maintain extensive working state, which is what differentiates reasoning-tier from Flash-tier.
WebArena scores drop sharply on cheap models. The long-horizon nature of WebArena tasks rewards reasoning-tier capability. Switching from Sonnet 4.6 to Haiku 4.5 typically drops WebArena scores 15-20 points. The benchmark is harder to game with cheap models.
Mind2Web scores drop moderately on cheap models. The cross-site generalization rewards model capability for new-site adaptation but does not depend heavily on long-horizon reasoning. Drop is typically 8-12 points.
For production deployment cost modeling, the relevant scores are the cheap-model variants on each benchmark. A vendor citing 85% WebVoyager on the reasoning-tier configuration is telling the buyer about the upper-bound capability; the production-relevant number is the same agent on Flash-tier, which might be 75% on WebVoyager but only 50% on WebArena.
The benchmark-vs-production gap by benchmark
The 20-point benchmark-to-production gap applies differently across the three benchmarks.
WebVoyager has the largest gap. Production reliability typically lands 25-35 points below WebVoyager headline scores. The benchmark’s choice of well-defended real-world sites does include some adversarial conditions, but the task distribution is much narrower than production workloads. WebVoyager-leading agents commonly hit 50-55% production reliability.
WebArena has the smallest gap. The containerized environment is more sterile than real-world conditions, but the long-horizon task structure is closer to production workflow shapes. The benchmark-to-production gap is closer to 10-15 points. WebArena-leading agents at 70% typically hit 55-60% in production.
Mind2Web has a moderate gap. The cross-site nature of the benchmark approximates production diversity, but the curated task set excludes the messiest real-world conditions. Gap is typically 15-20 points.
The implication is that WebVoyager scores are the most-cited but least-predictive of production. WebArena scores are less-cited and more-predictive. Mind2Web sits between.
What it means for vendor evaluation
For buyers comparing browser-agent vendors in 2026, the triangulation approach produces better procurement decisions than relying on any single benchmark.
Demand scores on all three benchmarks. Vendors who publish only one score should be asked for the others. Vendors who decline to publish all three are signaling that the unpublished scores are unflattering.
Read the capability profile, not the average. An agent that scores 85/55/65 is different from one scoring 75/70/65. Same approximate average, very different production behavior. The procurement decision should match the capability profile to the production workload shape.
Weight WebArena and Mind2Web more heavily than WebVoyager. For most production workloads, long-horizon reliability and cross-site generalization matter more than short-task speed. The WebVoyager-heavy marketing of many vendors is selling capability that production deployments don’t fully use.
Verify with production-trial data. No combination of benchmark scores fully predicts production performance. A 4-6 week production trial on representative target sites provides better signal than any benchmark. The benchmarks are useful for vendor short-listing; the trial is useful for vendor selection.
For Apify Store publishers building actors that operate in adjacent territory to browser-agents, the benchmark landscape is also competitive context. The Apify MCP server, exposed to LLM agents, competes against direct browser-agent execution. The benchmark scores tell the buyer whether the agent-and-direct-execution path is good enough, which determines whether the Apify-actor-as-tool path captures share. The numbers say: for short single-target tasks, direct agent execution is competitive; for long-horizon or cross-site tasks, Apify actors with curated extraction logic still win.
Sources