WKAppBot — Computer Use, App Use, AppBot

# WKAppBot — Computer Use, App Use, AppBot

[![build-launcher](https://github.com/kiexpert/wkappbot-sdk/actions/workflows/build.yml/badge.svg)](https://github.com/kiexpert/wkappbot-sdk/actions/workflows/build.yml)
[![extended-smoke](https://github.com/kiexpert/wkappbot-sdk/actions/workflows/extended-smoke.yml/badge.svg)](https://github.com/kiexpert/wkappbot-sdk/actions/workflows/extended-smoke.yml)
[![Latest Release](https://img.shields.io/github/v/release/kiexpert/wkappbot-sdk?label=download)](https://github.com/kiexpert/wkappbot-sdk/releases/latest)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![.NET](https://img.shields.io/badge/.NET-8.0-blue)](https://dot.net)
[![Platform](https://img.shields.io/badge/platform-Windows%2010%2B-lightgrey)](https://github.com/kiexpert/wkappbot-sdk/blob/main/docs/INSTALL.md)
[![Docs](https://img.shields.io/badge/docs-kiexpert.github.io-blue)](https://kiexpert.github.io/wkappbot-sdk/)
[![Sponsor](https://img.shields.io/badge/sponsor-PayPal-00457C?logo=paypal)](https://paypal.me/kiexpert)

**Windows + Android UI automation for Claude, GPT, Gemini, Copilot, and any AI agent.**  
Focusless. Self-healing. AI-native. The open-source bridge between LLMs and the apps humans already use.

## Why this exists

Computer Use lets an AI click, type, and read the screen. **App Use** is the layer above: agents that drive *specific* applications the way a power user does — focuslessly, alongside the human, without seizing the screen.

WKAppBot gives AI agents:

- **Eyes** — read any UI, extract text, recognize controls
- **Hands** — click, type, scroll, invoke — without stealing focus

No app rewrite. No vendor API. If a human can use it, WKAppBot can automate it.

> **Free tier covers all base automation.** CDP browser automation, multi-AI delegation (`ask triad`), and `--sudo` admin access are paid Pro tiers. See [PRICING.md](./PRICING.md).

---

## Why WKAppBot?

Most automation tools steal focus, break on owner-drawn controls, and go silent when the DOM changes. WKAppBot is built around three hard-won principles:

**Focusless-first** -- UIA Invoke/Value/Toggle/Select never steal focus. Win32 PostMessage handles legacy MFC. SendInput is the last resort, used only when nothing else works. The user keeps working while the AI operates in the background.

**Self-healing** -- When UIA fails on owner-drawn controls (MFC/HTS), CCA segmentation + OCR triple cross-validation + Gemini Vision inference recover element identity automatically and cache results in an Experience DB for next time.

**AI-native** -- The same binary that runs automation also delegates to GPT, Gemini, and Claude in parallel, streams prompts into live browser AI sessions via CDP, and manages its own context handoff when token budgets run low.

---

## OS Support

| OS | Status |
|----|--------|
| Windows 11 | ✅ Fully supported |
| Windows 10 22H2+ | ✅ Fully supported |
| Windows 10 < 22H2 | ⚠️ Untested |
| Windows Server 2019+ | ✅ Headless mode (no GUI a11y) |
| macOS / Linux | ❌ Not supported |

---

## What It Automates

| Target | Method |
|--------|--------|
| Modern Windows apps (WPF, UWP, Electron) | UIA Invoke / Value / Toggle / Select patterns |
| Legacy MFC / HTS trading terminals | Win32 PostMessage, WM_CHAR, CMaskEditEx path |
| Web apps (Chrome / Edge) | CDP -- click, type, eval JS, read DOM text |
| Browser AI (Claude, GPT, Gemini) | CDP prompt pump, cross-prompt chunking, attachment lock |
| Android apps | ADB + Accessibility tree (`adb://device/...` grap) |
| Owner-drawn controls with no UIA | CCA segmentation -> OCR -> Vision API fallback chain |

---

## Core Features

### grap (Grab Accessible Pattern) -- Universal Element Address

> *Human sees windows; AI points with grap.*

Every UI element gets a single address that works across Win32, UIA, web, and Android:

```
{proc:'chrome', domain:'claude.ai'}#main textarea   # CSS inside a browser window
heroes#realtime-account                             # UIA scope inside a window grap
adb://Fold5/*heromts*#balance                       # Android element
hwnd:0x010B084A                                     # Direct Win32 handle
*notepad*;*calc*                                    # OR pattern
```

`a11y find <grap>` prints a verified `# TARGET "hwnd:0x..."` line -- copy-paste ready for the next command.

### Auto-Pipeline on Every Action
Every `a11y` action runs a smart pre-flight before executing:

```
blocker dismiss -> minimize restore -> tab activate
  -> zoom/magnifier -> execute (3-tier) -> result feedback -> fade
```

No manual "wait for window" boilerplate. Blocking dialogs, minimized windows, and wrong-tab states are handled automatically.

### 5-Tier Element Search
UIA -> Vision Cache -> Simple OCR -> Vision API (Claude) -> Coordinate-based.  
Each tier auto-logs hits to an Experience DB; repeat runs skip expensive tiers.

### AppBot Eye -- Always-On Daemon
A single background process combining:
- **Slack daemon** (So

... (truncated)