Output: experiments/{benchmark}/{split}/{agent}/{model_id}/{timestamp}/

<p align="center">
  <img src="docs/logo/blue.svg" alt="Browseruse-Bench" width="600">
</p>

<p align="center">
  <a href="https://lexmount.github.io/browseruse-agent-bench/">Landing Page</a> •
  <a href="https://github.com/lexmount/browseruse-agent-bench/issues">Issues</a> •
  <a href="https://github.com/lexmount/browseruse-agent-bench/discussions">Discussions</a> •
  <a href="#leaderboard">Leaderboard</a> •
  <a href="https://docs.bubench.lexmount.io/">Documentation</a> •
  <a href="https://huggingface.co/datasets/Lexmount/LexBench-Browser">Dataset</a>
</p>

<p align="center">
  English | <a href="./README_ZH.md">简体中文</a>
</p>

## Why browseruse-agent-bench

**browseruse-agent-bench** is a reproducible evaluation framework for browser agents.
**LexBench-Browser** is the built-in public dataset used by the default benchmark workflow.
Together they make external results easy to run, compare, cite, and submit back.

| What you can do | Why it matters |
|-----------------|----------------|
| Run **LexBench-Browser: 210 public tasks across 107 real websites** | Test browser agents on long-tail multilingual workflows beyond toy pages |
| Compare **Agent × Model × Browser × Eval** | Separate agent quality from model choice, browser backend, and judge strategy |
| Inspect leaderboard, cost, latency, token usage, and trajectories | Debug failures instead of only reporting a final score |
| Submit agents, dataset tasks, and reproducible results | Turn forks and PRs into visible benchmark contributions |

## Description


**browseruse-agent-bench** is an all-in-one evaluation framework for AI browser agents, designed to benchmark *multiple agents across multiple datasets, browser backends, and models* under controlled and reproducible settings. The Python package/CLI is published as **browseruse-bench** and `bubench`. It supports both local and cloud browsers, integrates LLM-as-Judge for automated evaluation, and provides a built-in local leaderboard along with efficiency and cost metrics such as agent steps, end-to-end latency, and token usage.

**Supported Datasets**

- [x] **LexBench-Browser** — Browser-agent dataset covering e-commerce, social, academic, financial, and other mainstream Chinese/English websites (v1.0, 2026-04-30)
  - `All` (210, no login required)
  - `lexmount` (118, mainland-accessible websites) / `global` (92, international websites)
  - Hugging Face: [Lexmount/LexBench-Browser](https://huggingface.co/datasets/Lexmount/LexBench-Browser)
- [x] **Online-Mind2Web** — Real website interaction tasks
  - `All` (300) / `Hard` (hard subset)
- [x] **BrowseComp** — Browser operation competition tasks, no login required
  - `All` (1266)
- [ ] More benchmarks

> Details: [Benchmarks overview](https://docs.bubench.lexmount.io/en/benchmarks/overview).

**Supported Agents & Browsers**

| Agent | Supported Browsers |
|-------|-------------------|
| [browser-use](https://github.com/browser-use/browser-use) | `Chrome-Local`, `lexmount`, `browser-use-cloud`, `agentbay`, `browserbase`, `browserless`, `steel` |
| [skyvern](https://github.com/Skyvern-AI/Skyvern/) | `local`, `lexmount`, `skyvern-cloud`, `agentbay`, `browserbase`, `browserless`, `steel` |
| [Agent-TARS](https://github.com/bytedance/UI-TARS-desktop) | Built-in browser |
| More agents | — |

> Details: [Agents overview](https://docs.bubench.lexmount.io/en/agents/overview).

## News

- **[2026.04.30]** 🎉 **browseruse-agent-bench v1.0** — initial open-source release. The LexBench-Browser dataset v1.0 ships 210 public tasks across 107 distinct websites with a 6-category × 16-tag robustness label system; reference integrations cover browser-use, skyvern, Agent-TARS and deepbrowse.

## Quickstart


**1. Clone the repository**

```bash
git clone https://github.com/lexmount/browseruse-agent-bench.git
cd browseruse-agent-bench
```

**2. Install dependencies (Python>=3.11)**

Requires [uv](https://docs.astral.sh/uv/) (recommended). Select the section for your agent.

> **Note**: `browser-use` and `skyvern` have conflicting dependencies and cannot be installed together. If you plan to run multiple agents in parallel, refer to the [Environment Isolation](https://docs.bubench.lexmount.io/en/quickstart#running-multiple-agents-in-parallel) section in the documentation.

**browser-use**

```bash
uv sync --extra browser-use
source .venv/bin/activate          # macOS / Linux
.venv\Scripts\Activate.ps1         # Windows PowerShell
```

**skyvern**

```bash
uv sync --extra skyvern
source .venv/bin/activate          # macOS / Linux
.venv\Scripts\Activate.ps1         # Windows PowerShell
```

**Agent-TARS** (requires Node.js 18+)

```bash
uv sync
npm install -g @agent-tars/cli@0.3.0
source .venv/bin/activate          # macOS / Linux
.venv\Scripts\Activate.ps1         # Windows PowerShell
```

**CLI agents: claude-code / codex / cursor / openclaw** (requires Node.js 18+; **openclaw requires Node.js 22.19+**)

These agents share the main venv (`uv sync`, no extra) and drive

... (truncated)