duet-agent

# duet-agent

![duet-agent](assets/cover.png)

An opinionated, full-stack agent turn runner. Native multimodal memory. Native interrupts. Multi-agent by default. Serverless-friendly: every turn rehydrates from on-disk state, so a session can pause for minutes or months and resume in a fresh sandbox.

## Get started with one login

```bash
bun add --global @duetso/agent
duet login
duet "build a REST API with Express"
```

`duet login` opens your browser, signs you in, and writes a single `DUET_API_KEY` to `~/.duet/.env`. That one key unlocks:

- **Frontier language models** — Claude Opus / Sonnet / Haiku, GPT-5, Gemini, and friends routed through the Duet AI Gateway. No separate Anthropic, OpenAI, or Google billing to set up.
- **Image and video generation** — GPT Image 2, Seedance, and the other media models the gateway exposes.
- **Web scraping and research** — Firecrawl-powered scraping, search, and browser automation come bundled as default skills.
- **The latest Duet skills** — auto-synced into `~/.duet/skills` on first login and refreshed in the background on every subsequent run.

One login, every frontier model, every default skill, kept fresh. If you would rather wire your own provider keys, see [CLI Env Setup](#cli-env-setup).

## Why another agent framework?

Existing agent turn runners treat tools and memories as pluggable modules. This makes them flexible but fundamentally disconnected — memory is an afterthought.

duet-agent takes the opposite approach: **memory is woven into the core architecture.** Observations are recorded as the agent works, persisted across processes, and reflected when context grows; the runner cannot run without them. Interrupts are handled by the underlying pi agent runtime, so the turn runner does not need its own interrupt bus.

## Architecture

The diagram below walks through a realistic agent-routed relay: outbound conference
outreach. The user prompt enters the `TurnRunner`, the runner agent picks the next state based on
prompt, history, and available state definitions, and the relay drives the business process
until it hits a terminal state. The same definition can start in the middle — for example, the
runner can skip straight to `wait_for_reply` if the user says “I already emailed them”. Internally
these relays are state machines; “relays” is the user-facing name in the UI and docs — each state
hands off (relay-style) to the next.

```mermaid
stateDiagram-v2
  direction TB

  [*] --> Classify : user prompt
  Classify --> AgentMode : one-off task
  Classify --> Outreach : matches a relay

  AgentMode --> [*] : answer / edits

  state Outreach {
    direction TB
    [*] --> research_prospect

    research_prospect : research_prospect (agent)\nweb + notes lookup
    draft_email       : draft_email (agent)\nwrite first-touch email
    send_email        : send_email (script)\nbash: gmail send
    wait_for_reply    : wait_for_reply (poll)\nevery 6h: check inbox
    schedule_meeting  : schedule_meeting (script)\nbash: calendly create
    meeting_booked    : meeting_booked (terminal: completed)
    not_interested    : not_interested (terminal: completed)
    no_response       : no_response (terminal: cancelled)

    research_prospect --> draft_email     : enough signal
    research_prospect --> not_interested  : disqualified
    draft_email       --> send_email
    send_email        --> wait_for_reply
    wait_for_reply    --> schedule_meeting : positive reply
    wait_for_reply    --> not_interested   : declined
    wait_for_reply    --> no_response      : 14d timeout
    schedule_meeting  --> meeting_booked
  }

  Outreach --> [*]
```

Each state is one of the four kinds the runner understands:

- **agent** states run a sub-agent with a prompt, optional system prompt, and optional skill allowlist.
- **script** states shell out (`bash`, `curl`, CLIs) for anything with an API.
- **poll** states wait on an external signal by running one script check per interval, or by using a timer poll for pure delays.
- **terminal** states record a business outcome (`completed`, `cancelled`, `failed`).

Observational memory, pi coding tools, and guardrails sit underneath every state transition; they
are not states themselves.

## Key Differentiators

### Native Multimodal Memory

Memory is first-class. The default `MemoryStore` is in-memory and emits observation events; optional PGlite storage hydrates and persists durable observations outside the turn runner session.

The memory model follows observational memory: turn runner session messages are observed into durable text observations, and a reflector condenses observations when they grow too large. Every observation is tagged with the session id that produced it and a `kind` (`observation` or `reflection`).

Memory rendered into the prompt prefix splits into two layers:

- **Long-term memory** (cross-session) ranks every other session's rows by `priority × recencyDecay × kindBias` (7-day half-life, reflections weighted 1.3×) and pa

... (truncated)