Status: Implementation IMPLEMENTED × Exposure PUBLIC PREVIEW. Contract: contracts/computer-toolset/ (allternit.computer.v2, allternit.browser.v1). Executor: cmd/allternit-api/src/computer_toolset.rs + computer_v2.rs. gizzi adapters: cmd/gizzi-code/src/runtime/tools/computer-toolset/.

​
Overview

The computer toolset is how every agent in Allternit uses a screen. There is one contract, one executor and one adapter per model family:
  1. Contract. allternit.computer.v2 (23 members: the 17 pixel members of allternit.computer.v1 — byte-identical names and input fields to Anthropic’s computer_toolset_20260801 — plus 6 structured members backed by the Allternit Driver) and allternit.browser.v1 (31 members, identical to Anthropic’s browser_toolset_20260801). v2 is additive: every v1 call is a valid v2 call.
  2. Executor. POST /api/v1/computers/:id/toolset in allternit-api. It validates the call, checks the control lease and approvals, writes the audit row, scales coordinates and dispatches the action.
  3. Adapters. gizzi turns each model’s tool calls into contract calls. gizzi is the only agent loop. No adapter runs a loop of its own.
:id is a computer id or this-device. In gizzi, set the target with GIZZI_COMPUTER_ID. The default is this-device.

​
Members

​
Computer (allternit.computer.v1)

​
Computer v2 (allternit.computer.v2)

The structured members. They read and drive the UI through the Allternit Driver sidecar — no screenshots for reads — and every model family gets them as the computer_v2 function tool (Claude gets them next to its native toolset). Prompt guidance in both tools steers models to prefer read_ui + run_batch over screenshot + pixel loops. run_batch runs ordered steps in one window. Each step may check wait_for (precondition) and expect (postcondition); the batch stops at the first failed check and returns one observe read (the fresh element map). verify checks bounded conditions — role, name, text, gone, value, enabled — on a fresh read. request_human pauses the lease, opens the live view with input enabled, and resumes the same session when the person signals done (POST /api/v1/computers/:id/human-done) or the timeout elapses. use_credential types a vault secret or a fresh TOTP code into the focused field; the value never enters the model context, the prompts or the logs. See Computer credentials. run_subtask hands a bounded step sequence (fill a form, search and pick a result, toggle settings) to the decision loop instead of the planner model. Each step reads the element map, picks one option from the visible elements × allowed ops with a typed decision (POST /v1/decisions, about 100–150 ms on a local scorer), runs it through the same lease → policy → audit path as every other call, and checks success with verify. It returns a step trace (decision, backend, latency, confidence per step) and a status: done, escalated (with the reason and a compact current screen, so the planner continues from there) or failed. A decision under its confidence threshold hands control back instead of guessing. Irreversible elements (the safety layer’s shared classifier) are never options unless constraints.allow_irreversible is set, and even then they end the subtask with needs_confirmation for the planner to run with the person’s approval. Secrets never go in inputs: use use_credential. With api_options, an MCP tool or API action the planner holds is offered ahead of the GUI options on the first step; picking it ends the subtask with status use_api and the planner makes that call. See How computer use decides for the loop and measured numbers. A subtask that has run before replays from the replay cache with no decisions: same goal, app and window, same input names, new values. A step whose element moved is found again or re-inferred with one decision, and the recording is rewritten. save_as keeps a recording as a named skill that run_skill runs with new input values; skills lists them. The result’s cache block says what happened (hit, healed, miss, diverged, uncached, off). See Replay cache and skills. run_parallel runs independent subtasks on separate computers at the same time, each under its own computer’s control lease, and returns every result in order. run_subtask with best_of: N runs N rollouts on N sandbox computers and a judge decision picks one by the rollouts’ step narratives; it is refused on the person’s own machine. See Parallel subtasks and Best-of-N.
Structured members need the Allternit Driver on the computer: this-device (macOS, Linux) and cloud guests (Linux, Windows — the driver ships in the guest image and the schema endpoint reports when it isn’t answering).

​
Browser (allternit.browser.v1)

Targets are either { "type": "coordinate", "x": 640, "y": 380 } or { "type": "ref", "ref": "ref_2" }. The executor reports what a target can run (see Schema). A member that a target can’t run returns is_error with a plain reason. It is never silently skipped.

​
Executor

model_frame is the size of the last screenshot the model saw. Leave it out and the executor uses the default frame for the screen. coordinate_space is pixels (the default) or normalized_1000. The checks run in this order:
  1. Contract validation. The member must exist and its input must match the member schema. The batch rule applies (see below). A bad request returns 400.
  2. Control lease. On a cloud computer the agent takes or renews the lease, so viewers can see who is driving and a person can take over. If someone else holds control, the call returns 423 computer_controlled_elsewhere. On this device the owner must have taken control first, otherwise the call returns 423 take_control_first.
  3. Policy and approval. A declarative policy denial returns 403 policy_denied. A member that needs approval returns 409 approval_required with an approval_id. Approve it with POST /api/aci/handoff/:approval_id/approve, then resend the same call with approval_grant set to that id. A grant works once, and only for the exact action it was issued for.
  4. Safety. Evaluated between the policy check and the approval check of step 3, so its confirmations ride the same approval. The safety layer assesses the step: app/domain lists, host policy and credential binding can deny it (403 safety_denied); the per-step monitor can pause it (safety_paused); irreversible classes, watch mode and monitor confirmations turn it into 409 approval_required on every target, with a safety object explaining why. On cloud computers a rollback snapshot is taken before the first risky step of a run.
  5. Audit row. A row is written to the policy audit log and fsynced before the action runs. If the row can’t be written, the action doesn’t run.
  6. Dispatch. The action is sent to the Allternit Driver on this device, the guest or the browser gateway. The v2 structured members answer from the driver’s element map (or the human gate, or the credential backends): no screenshot is taken, no coordinates are scaled, and the result is a text block with the driver’s JSON (element tree, batch result or verify rows).
Every call returns this result shape:
Refusals add an error code. Text results that carry screen or page content arrive spotlighted as untrusted data, and screenshots arrive with personal data blacked out and a short text note after the image.

​
Human hand-off

request_human opens a human window on the computer: computer.human_requested and computer.human_resumed events on the events stream carry { reason, live_view: true, input_enabled: true } for the UI, and on cloud computers the agent’s lease is released for the window (on this-device the person already is the controller). The person signals done with:
which wakes the waiting call; it re-takes the lease (a person still driving keeps control — agents never preempt a person) and returns “The session has control again.”

​
Browser results

Every dispatched browser result, success or failure, carries browser_state:
  • Tabs. Every page in the session is a tab with a stable tab_N id. Exactly one tab is active, and a session always has at least one. Closing the active tab makes the most recent remaining tab active. Closing the last tab opens a blank one.
  • state_changes lists what this call changed, in order: tab_opened, tab_closed, active_tab_changed and navigated (a tab’s URL changed). Popups and target=_blank links show up as tab_opened.
  • tab_id input. Members that take tab_id act on that tab without switching to it. Leave it out to act on the active tab.
  • Element refs. read_page and find return elements with refs such as ref_7. Every member that takes a target accepts { "type": "ref", "ref": "ref_7" } as well as a coordinate. A ref belongs to its tab. It goes stale after a navigation or a re-render that removes the element. A stale ref returns is_error and asks for a fresh read_page or find.
  • find matches the query’s words against each visible element’s role, accessible name, placeholder, name and id, and returns the 20 best matches with their refs and centre points. It doesn’t call a model.

​
URL policy

  • Only http, https and about:blank can be opened. file:, data:, javascript:, chrome: and every other scheme are refused on every target.
  • A browser that runs for a cloud computer refuses hosts that are loopback, private, CGNAT or mesh (100.64.0.0/10), link-local (including the 169.254.169.254 metadata address), metadata.google.internal, *.internal, *.local or localhost, or a host that resolves to any of those. The executor checks navigate first. The gateway then checks every request the page makes, including redirects and subresources.
  • A browser that runs on your own machine (this device, not a cloud computer) can open local and private addresses.
  • The workspace host policy (aci_safety) still applies on top.

​
Optional members

file_upload, read_console, read_network and javascript_exec are off by default. A call to one of them returns 403 member_disabled unless the request lists it in enable:
GET …/toolset/schema?toolset=browser&enable=read_console,read_network reports those members as enabled. @allternit/computer-driver and allternit-computer-driver send enable for the member being called, because the Anthropic SDK only calls the members its toolset config turned on. javascript_exec and file_upload still need an approval grant on every call.
  • read_console and read_network return the last 150 console messages or responses recorded on the tab. Recording starts the first time the tab is used through the toolset.
  • javascript_exec evaluates the text in the page and returns the result as JSON.
  • file_upload sets files on an <input type=file> ref. paths are on the browser’s computer. A cloud browser only uploads files under its upload folder (ALLTERNIT_BROWSER_UPLOAD_ROOT, default <tmp>/allternit-browser-uploads). document_ids aren’t supported.

​
Batch rule

The calls in one turn (turn_id) run in call_index order. After a call fails, or while it waits on an approval nobody has given, each later call in the turn returns is_error with exactly:
  • computer: Not executed: an earlier computer action in this turn failed.
  • browser: Not executed: an earlier action in this turn failed.
A retry of the failed call itself, for example with an approval grant, is allowed.

​
Model frame and scaling

Screenshots are downscaled so the long edge is at most 1568 px and the area is at most 1.15 MP. They are never upscaled, and the aspect ratio is kept. The model sees the screen in that frame and answers in it. The executor maps coordinates back to screen pixels, and the executor is the only place this happens. On this device the screen size is the native screenshot size in pixels, because Cua Driver 0.34’s desktop scope takes native screenshot pixels on a Retina Mac too. cursor_position converts Cua Driver’s points to pixels. zoom crops the region at full resolution and fits it to the frame. Grid-trained GUI models such as UI-TARS and Qwen-VL send coordinate_space: "normalized_1000", a 0–1000 grid over the whole screen.

​
Schema

This returns the target (this_device, guest_linux, guest_windows or browser_gateway), whether it is sandboxed, and every member with enabled, a reason when it is disabled, risk, needs_approval, a description and its input schema.

​
Events

Every action that runs emits this event:
x and y are screen pixels. The event goes out on the computer’s events websocket (/computers/:id/events) and on GET /api/v1/computers/:id/toolset/events, a server-sent event stream that works for every target, including this device. The agent-cursor overlay uses these events.

​
Adapters

The function tools are one per toolset, named computer (pixel members), computer_v2 (structured members) or browser. Each takes an action (the member name) plus that member’s fields. The fields have the same names and types as the contract. A tool only lists members the target reports as enabled. Both computer tools carry the steering guidance: prefer read_ui + run_batch over screenshot + pixel loops, give each bounded step sequence to run_subtask, keeping the planner’s own calls for planning and for what a subtask hands back, call an MCP tool or API instead of the GUI when one does the job, and put independent subtasks on separate computers into one run_parallel call. When the executor answers approval_required, gizzi asks the person through its normal permission prompt, records the approval and retries once with the grant.

​
What it replaced

  • gizzi browser tool (ACU gateway, goal-driven execute). It is replaced by the toolset browser tool under the same GIZZI_DISABLE_BROWSER_TOOL switch. Goal-driven browsing was a second agent loop, so it is gone.
  • gizzi desktop tool. Removed on 2026-10-09. Screen input is the computer tool; provisioning and files are the computers tool, which calls allternit-api’s computer_* tools. Both are on with GIZZI_ENABLE_COMPUTER_TOOL.
  • pane_browser is unchanged. It drives the page the person can see in the app’s browser pane. That pane is a different target from the headless gateway session, so the two stay separate.
  • allternit-api computer_screenshot / computer_mouse / computer_keyboard tools (/tools/execute). These stay as compatibility aliases on the shared control plane. New work uses the toolset.

​
Contract files and generated types

Outputs: sdk/computer-toolset (@allternit/computer-toolset), gizzi contract.gen.ts, and domains/computer-use/core/contracts/toolset_v1.py. allternit-api reads the JSON directly (computer_toolset.rs reads the v2 file; v1 stays as the pixel-only reference for the Claude-native configs).