Allternit Computer Interface (ACI)

Status: Implementation IMPLEMENTED × Exposure PUBLIC PREVIEW. Routes verified against cmd/allternit-api/src/aci_routes.rs; batch and code routes added (they were undocumented). Batch and code grants are enforced by aci_batch.rs, aci_code.rs and aci_safety.rs.

Overview

The Allternit Computer Interface (ACI) is the platform layer that lets an agent observe and control a computer — typically a browser — through a live screenshot stream, mouse/keyboard actions, and human-in-the-loop approvals. ACI routes requests through the Allternit Computer Use (ACU) gateway and surfaces the run state to web, desktop, and mobile clients.

ACI runs on infrastructure you control. Code, screenshots, and credentials stay on the host.

ACI vs managed sandboxes

	ACI / self-hosted	Managed cloud sandbox
Hosting	Your machine, a private VPS, or a self-managed VM	Vendor-managed cloud account
Data residency	Stays on the host	Leaves the host by default
Network access	Same network as the host	Typically limited to public endpoints
Approval model	Pauses for human approval before destructive actions	Depends on the provider

Endpoints

Method	Path	Description
POST	`/api/aci/run`	Start a browser automation run
GET	`/api/aci/stream/:id`	SSE stream of run events
POST	`/api/aci/stop/:id`	Cancel a run
POST	`/api/aci/approve/:id`	Approve a pending action
POST	`/api/aci/approve/:id?deny=true`	Deny a pending action
GET	`/api/aci/approvals/:id`	Status of a pending approval.
GET	`/api/aci/policy/audit`	Policy audit listing.
GET	`/api/aci/handoff/:id`	Status of a sensitive-action handoff.
POST	`/api/aci/handoff/:id/approve`	Approve a handoff.
POST	`/api/aci/handoff/:id/deny`	Deny a handoff.
POST	`/api/aci/batch`	Submit an ordered batch of whitelisted browser actions under one hash-bound, single-use grant. Halts at first failure; the batch receipt is written before execution.
POST	`/api/aci/code`	Code-mode execution under its own single-use grant. Credential patterns, destructive calls, undeclared network targets and host-path access are refused with a 400. Never the default.

Start a run

curl -X POST http://127.0.0.1:8013/api/aci/run \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "goal": "Find the latest release notes on github.com/allternit/allternit",
    "model": "gizzi-pro",
    "allowedSites": ["github.com"],
    "openLinksInBrowser": true,
    "autoVerify": false,
    "sessionPersistence": "dont-keep"
  }'

Request body

Field	Type	Required	Description
`goal`	string	yes	Natural-language objective for the run
`model`	string	no	Model identifier to use for planning
`allowedSites`	array / any	no	Sites the run is allowed to interact with
`openLinksInBrowser`	boolean	no	Whether links should open in the browser
`autoVerify`	boolean	no	Skip human approvals for read-only runs
`sessionPersistence`	any	no	Passed through to the ACU gateway verbatim

Response:

{
  "sessionId": "550e8400-e29b-41d4-a716-446655440000",
  "adapterId": "browser"
}

The returned sessionId is used to attach to the event stream and to stop or approve the run.

Stream events

Run progress is delivered as Server-Sent Events at /api/aci/stream/{sessionId}.

curl -N http://127.0.0.1:8013/api/aci/stream/550e8400-e29b-41d4-a716-446655440000 \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer <token>"

Each event is a JSON object wrapped in a data: line:

data: {"type":"trace","data":{"message":"Navigating to github.com","event_type":"browser.navigate","data":{}},"ts":1712345678901}

data: {"type":"state","data":{"url":"https://github.com/allternit/allternit"},"ts":1712345679200}

data: {"type":"done","data":{"status":"completed"},"ts":1712345681000}

Event types

Type	Meaning
`trace`	Human-readable progress message from the ACU gateway
`state`	Structured state update from the planning loop
`done`	Run completed or was cancelled

The API keeps an in-process replay buffer for each run started in the current process, so clients can attach late and still see the full history. Runs created by another process fall back to proxying ACU’s single-consumer queue.

Stop a run

curl -X POST http://127.0.0.1:8013/api/aci/stop/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer <token>"

Response:

{
  "run_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "cancelled"
}

Approve or deny an action

When autoVerify is false, the ACU gateway pauses for human approval before destructive actions. Use the approve endpoint to continue or reject the action. Approve:

curl -X POST http://127.0.0.1:8013/api/aci/approve/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer <token>"

Deny:

curl -X POST "http://127.0.0.1:8013/api/aci/approve/550e8400-e29b-41d4-a716-446655440000?deny=true" \
  -H "Authorization: Bearer <token>"

Response:

{
  "run_id": "550e8400-e29b-41d4-a716-446655440000",
  "decision": "approve"
}

The `computer` tool

Agents use a computer through the computer toolset. It has one contract (allternit.computer.v1 and allternit.browser.v1, the same members as Anthropic’s computer_toolset_20260801 and browser_toolset_20260801) and one executor, POST /api/v1/computers/:id/toolset. Every model family has an adapter: Claude-native, OpenAI, Gemini, and JSON function tools for everything else. The executor is the only place that:

scales coordinates between the model’s screenshot frame and the screen. Screenshots are downscaled to a long edge of 1568 px or less and 1.15 MP or less.
checks the control lease (423) and asks for approval (409 approval_required, with a single-use grant tied to the action’s hash).
writes the audit row before the action runs.
emits computer.action events, which the agent-cursor overlay draws.

The SDK `computer` tool

The SDK’s ComputerUseCapability (@allternit/sdk/ai-runtime/capabilities/computer-use) still exposes the older single computer tool. It forwards actions to the ACU gateway. New integrations should call the toolset executor instead.

import { ComputerUseCapability } from '@allternit/sdk/ai-runtime/capabilities/computer-use';

const computer = new ComputerUseCapability({
  displayWidthPx: 1280,
  displayHeightPx: 720,
});

const tool = computer.getTool();
const screenshot = await tool.execute!({ action: 'screenshot' }, {});

Approvals in the SDK tool

The tool’s preExecute hook classifies every action with the same three classes the API uses (aci_safety): reversible (screenshot, cursor_position, mouse_move, scroll, wait, zoom, typing without a newline, keys other than Enter), risky (clicks, drags, button down/up, typing that contains a newline, Enter/Return, anything unknown) and irreversible. Reversible actions run. Risky and irreversible actions run only when the approve callback says yes. With no approver configured they are refused, so the tool never acts without a person’s decision.

const computer = new ComputerUseCapability({
  approve: async ({ action, actionClass, input }) => askThePerson(action, actionClass, input),
});

The callback may return true/false or { approved, reason }. classifyComputerAction(input) is exported if you want to show the class in your own UI.

Vision coordinates

ACI uses absolute pixel coordinates for mouse actions. The capability advertises the display size in tool metadata so the model can reason about coordinates. In the Python engine’s planning loop, the coordinate space is the real size of each screenshot (read from the PNG or JPEG header), or the size the session reports. It falls back to 1920x1080, the size of cloud computers. It is no longer fixed at 1280x720.

Supported actions

Action	Needs coordinate	Needs text
`mouse_move`, `left_click`, `right_click`, `double_click`, `triple_click`, `left_click_drag`	yes	no
`scroll`	yes	no
`type`, `key`, `hold_key`	no	yes
`screenshot`, `cursor_position`, `wait`	no	no

gizzi computer tools

GIZZI_ENABLE_COMPUTER_TOOL turns on two gizzi tools. The toolset computer tool sees the screen and uses the mouse and keyboard. The computers tool lists, creates, starts and stops computers and reads or writes files on them; it runs allternit-api’s computer_* tools. Their target is GIZZI_COMPUTER_ID, and the default is this device. On this device, take control in the Computer view first, so the agent can use the mouse and keyboard. (The older desktop tool and its GIZZI_ENABLE_DESKTOP_TOOL flag were removed on 2026-10-09.)

Control lease on this Mac

Only one controller drives a computer at a time. On cloud computers, input without the control lease is refused with 423 computer_controlled_elsewhere. The Mac running Allternit follows the same rule:

A person’s mouse and keyboard input (/api/v1/computers/:id/mouse and /keyboard) needs that person to hold control. Without it the API answers 409 take_control_first. If someone else holds it, the answer is 423.
An agent acting on this Mac (computer:local steps in agent runs) takes the lease as an agent before any click or keystroke. If a person has taken over, the agent’s input is refused with 423 computer_controlled_elsewhere and never reaches the computer. Screenshots and other read-only steps still work. The person keeps control until they hand it back.

Cua Driver and the agent cursor

Allternit Desktop embeds Cua Driver 0.34.0 for macOS (universal), Linux x86_64 and Windows x86_64. It is pinned by SHA-256 in surfaces/allternit-desktop/resources/computer-use/VERSION.json and staged by npm run prepare:cua-driver, which also records the staged binary’s own hash. That way a stale binary is never reused under a newer version. If /Applications/CuaDriver.app is installed on a Mac, Desktop uses that app’s daemon instead. When the engine starts a Cua session it calls the driver’s start_session with a cursor motion style. The default is signature_arc. To pick another, set ALLTERNIT_CUA_CURSOR_MOTION to spring_settle, magnetic, comet_swoop, adaptive or classic. Reduced motion is left to the driver’s auto theme, which follows the operating system’s setting and always wins. ALLTERNIT_CUA_REDUCED_MOTION=on|off overrides it. Drivers older than 0.34 don’t have start_session, and actions still run on their implicit session.

Error handling

HTTP	Code / Message	Meaning
200	—	Run started, streamed, stopped, or approved successfully
400	`goal is required`	Missing or empty `goal` in the run request
502	`acu_unavailable`	The ACU gateway could not be reached
409	`take_control_first`	Input on this Mac without holding its control lease
423	`computer_controlled_elsewhere`	Someone else (a person who took over, or another agent) holds control
502	`acu_error`	The ACU gateway returned a non-success status

​Overview

​ACI vs managed sandboxes

​Endpoints

​Start a run

​Request body

​Stream events

​Event types

​Stop a run

​Approve or deny an action

​The computer tool

​The SDK computer tool

​Approvals in the SDK tool

​Vision coordinates

​Supported actions

​gizzi computer tools

​Control lease on this Mac

​Cua Driver and the agent cursor

​Error handling

​Related pages

Overview

ACI vs managed sandboxes

Endpoints

Start a run

Request body

Stream events

Event types

Stop a run

Approve or deny an action

The `computer` tool

The SDK `computer` tool

Approvals in the SDK tool

Vision coordinates

Supported actions

gizzi computer tools

Control lease on this Mac

Cua Driver and the agent cursor

Error handling

Related pages