ExploitHunter.app

<div align="center">

<img src="./screenshots/readme-mockups/agent-security-social-preview.png" alt="ExploitHunter.app social preview" width="100%" />

# ExploitHunter.app

**Open-source offensive-security harness. Real findings across frontier, cheap API, and local models.**

[![CI](https://github.com/justsml/ExploitHunter.app/actions/workflows/ci.yml/badge.svg)](https://github.com/justsml/ExploitHunter.app/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Node.js](https://img.shields.io/badge/node-%3E%3D24-brightgreen)](https://nodejs.org/)
[![DataExplorer](https://img.shields.io/badge/DataExplorer-📊-blue)](https://justsml.github.io/ExploitHunter.app/)

</div>

---

## Can complex AI-agent security work get 100x or 200x cheaper?

Sometimes, yes - but not by pretending one tiny model can replace every frontier-model decision.

The data-backed play is a conductor pattern: route the work to the model that belongs on that stage. Use cheap or local subagents for broad recon, inventory, tool-readiness checks, and first-pass evidence gathering; spend frontier-model budget only where deeper reasoning, validation, or report synthesis actually pays for itself.

That is the bet behind ExploitHunter.app. In the Hard Juice Shop sweep below, `qwen-3.6-flash` found **7 evidence-backed bug categories for $0.007**. Frontier Claude rows in the same table cost roughly **$2.00-$2.35** per run, so the right lane can be **280x+ cheaper** while still producing useful evidence. Local LM Studio and self-hosted Vast.ai/vLLM routes add another mode: useful signals with **$0 candidate API cost** when the target, privacy constraints, or run volume make hosted inference the bottleneck.

The answer is only partially "use cheaper models." The real answer is orchestration: a security-research conductor that keeps target scope, memory, approvals, tools, and artifacts stable while sending each slice of work to the right model route.

**What the results show:**

- Frontier models are still worth paying for on deeper investigation lanes and harder validation.
- Cheap hosted models can produce broad, evidence-backed web findings for cents when the harness gives them the right rails.
- Local models can run offline with no API key and still find useful signals.
- Model behavior varies by lane: broad web sweeps, disciplined tool use, Docker network attack paths, and slower high-score investigations do not all favor the same route.

[Data Explorer 📊](https://justsml.github.io/ExploitHunter.app/)

Current checked-in aggregate snapshot, generated from `evals/results/**`:

| Eval rows | Model profiles | Eval categories | Total tokens | Tool calls / budget | Cost recorded in rows |
| --------: | -------------: | --------------: | -----------: | ------------------: | --------------------: |
|     4,835 |            173 |               8 |  412,362,178 |      22,990/250,436 |              $212.813 |

### Eval commands and lab types

Every `pnpm eval:*` script below runs through the same `pnpm eval:labs` runner, just scoped to a different lab type and target set.

Before starting lab environments, the eval runner automatically stops running containers whose names start with `exploit-hunter-` and that were created at least four hours ago. Set `EXPLOIT_HUNTER_EVAL_CONTAINER_MAX_AGE_HOURS` to change the cutoff. The cleanup stops containers without removing them or their volumes.

- `pnpm eval:labs`: the single lab runner. It expands dataset rows, resolves each Benchmark Task, stages candidate-visible inputs, runs through the real browser/app path, records cost/tokens/tool budgets and strict LLM evidence, then tears the target down.
- `webapp-lab`: authorized HTTP targets. `pnpm eval:webapp` runs both broad anonymous attack vectors (`scope: attack-vectors`) and evidence-first Juice Shop analysis (`scope: juice-shop`) from [`evals/manifests/webapp.json`](./evals/manifests/webapp.json). Every row starts and stops the Benchmark Task declared by that manifest; no separate target-launch command is needed.
- `docker-lab`: Compose-backed services and networks. `pnpm eval:network-labs` covers service inventory, HTTP/network boundaries, controlled abuse cases, evidence, and clean teardown (`scope: docker-labs`).
- `offline-lab`: staged, network-independent artifacts. `pnpm eval:offline-labs` runs adapted Cybench for forensics, cryptanalysis, binary reversing, captures, archives, and air-gap-style analysis (`scope: offline/<categories>`).

`pnpm eval:webapp`, `pnpm eval:network-labs`, and `pnpm eval:security-bench` are selectors over that same pathway. `eval:webapp` includes both broad anonymous attack vectors and evidence-first Juice Shop analysis. Prompt improvement, model-tool behavior, skill recall, benchmark audits, and publishing checks remain supporting quality gates rather than additional target families.

**If you're newer to appsec, the shorthand is:**

- **Vulnerabilit

... (truncated)