Install dependencies

<h1 align="center">WildClawBench</h1>

<p align="center">
  <img src="assets/lobster_battle.png" alt="WildClawBench Lobster" width="480">
</p>

<div align="center">

[![Leaderboard](https://img.shields.io/badge/🏆_Leaderboard-WildClawBench-8c2416)](https://internlm.github.io/WildClawBench/)
[![HuggingFace](https://img.shields.io/badge/🤗_HuggingFace-Dataset-yellow)](https://huggingface.co/datasets/internlm/WildClawBench)
[![Tasks](https://img.shields.io/badge/Tasks-60-blue)]()
[![Models](https://img.shields.io/badge/Models-10-green)]()

</div>


> **Hard, practical, end-to-end evaluation for AI agents — in the wild.**

---

**WildClawBench** is an agent benchmark that tests what actually matters: can an AI agent do real work, end-to-end, without hand-holding?

We drop agents into a live [OpenClaw](https://github.com/openclaw/openclaw) environment — the same open-source personal AI assistant that real users rely on daily — and throw **60 original tasks** at them: clipping goal highlights from a football match, negotiating meeting times over multi-round emails, hunting down contradictions in search results, writing inference scripts for undocumented codebases, catching privacy leaks before they happen. Useful things. Hard things.

Hard enough that **every frontier model today scores below 0.6**. That makes scores mean something.

### Why WildClawBench?

Most agent benchmarks test isolated capabilities — calling a function, parsing JSON, following a single instruction. WildClawBench tests the full picture:

| | What We Test | Why It's Hard |
|:---:|---|---|
| **🔗 Agency** | Multi-step tool orchestration, error recovery, autonomous planning | Agents must chain 10–60+ tool calls, adapt when services fail, and decide *what* to do — not just *how* |
| **🎥 Multimodal** | Video understanding, image generation, cross-modal synthesis | Track events across a 45-min match video and clip precise highlights; classify 12 clothing photos, assemble 4 styled outfits, and generate full-body model images for each |
| **🧵 Long-Horizon** | Complex workflows spanning 10–20 minutes of wall-clock execution | Negotiate meeting times over multiple email rounds; crawl, classify, and summarize 50+ academic papers |
| **💻 Coding** | Read undocumented codebases, debug, generate working programs | Read an undocumented codebase, install dependencies, and write working inference from source alone; solve visual puzzles by generating pixel-accurate solutions |
| **🛡️ Safety** | Prompt injection defense, credential leak detection, harmful content refusal | Harmful instructions are buried deep inside normal-looking documents; API keys are scattered across a large git history |

### What Sets Us Apart

- **Real environment, not mocks.** Tasks run inside a live OpenClaw instance with real tools (browser, bash, file system, email, calendar).
- **60 original tasks, built by hand.** Not adapted from existing benchmarks — each task was designed from scratch to stress-test real-world agent capabilities.
- **Reproducible & isolated.** Each task runs in its own Docker container. Same image, same data, same grading code. Ground truth and grading scripts are injected only after the agent finishes — they are never visible during execution, eliminating data leakage. Scores are reproducible across machines.

---

## Leaderboard

> Results as of 2026-03-24. Full interactive leaderboard at [internlm.github.io/WildClawBench](https://internlm.github.io/WildClawBench/).

| Rank | Model | Org | Overall Score | Avg Time | Avg Cost |
|:----:|-------|-----|:-------------:|:--------:|:--------:|
| 🥇 | **Claude Opus 4.6** | Anthropic | **51.1%** | 508 min | $80.85 |
| 🥈 | GPT-5.4 | OpenAI | 48.5% | 350 min | $20.08 |
| 🥉 | MiMo V2 Pro | Xiaomi | 40.6% | 459 min | $26.47 |
| 4 | Gemini 3.1 Pro | Google DeepMind | 38.4% | 240 min | $18.22 |
| 5 | Qwen3.5 397B | Alibaba Cloud | 33.5% | 459 min | $22.33 |
| 6 | GLM 5 Turbo | Zhipu AI | 33.4% | 499 min | $14.80 |
| 7 | MiniMax M2.7 | MiniMax | 33.0% | 551 min | $7.47 |
| 8 | Kimi K2.5 | Moonshot AI | 28.7% | 406 min | $6.73 |
| 9 | Step 3.5 Flash | StepFun | 27.7% | 430 min | $6.63 |
| 10 | Grok 4.20 Beta | xAI | 19.5% | 94 min | $9.63 |

---

## Tasks

**60 tasks** across **6 categories**, spanning English and Chinese:

| Category | # | Example Tasks | Core Challenges |
|:---------|:-:|---------------|-----------------|
| **Productivity Flow** | 10 | ArXiv paper digest, PDF batch classification, calendar scheduling, Wikipedia biography, LaTeX table extraction | Information synthesis, multi-source aggregation, structured output |
| **Code Intelligence** | 12 | SAM3 inference from source, visual puzzle solving (jigsaw, connect-the-dots, link-a-pix), benchmark reproduction, academic homepage generation | Undocumented codebase comprehension, pixel-level visual reasoning, end-to-end code generation |
| **Social Interaction** | 6 | Multi-round meeting negotiation, chat action extraction, escalation routing, cross-departme

... (truncated)