Status: Implementation IMPLEMENTED × Exposure PUBLIC PREVIEW (driver phase D5, decision-runtime phase E3). Code: cmd/allternit-api/src/computer_safety.rs (wired into computer_toolset::execute and execute_step) and cmd/allternit-api/src/computer_ocr.rs (the server OCR engine), the driver’s context and ocr methods (domains/computer-use/driver/allternit_driver/safety.py). Settings tables: migrations V246__computer_safety_settings.sql, V250__computer_safety_project_scope.sql.

​
Overview

Every call to the computer toolset, and every step a run_subtask runs, goes through one safety assessment after the control lease and the declarative policy and before the approval gate. It ends in one of four verdicts: Denied and paused steps are written to the policy audit log as denied rows. Allowed and confirmed steps carry a safety note in their audit row (safety=… irreversible=… watch=… monitor=… host=…).

​
Untrusted observations (spotlighting)

Everything a model reads back from the screen is data, not instructions. Text results of read_ui, act, run_batch, verify, run_subtask and the browser members that return page text (read_page, find, get_page_text, read_console, read_network, javascript_exec, the tab members, navigate) arrive wrapped like this:
  • The nonce is fresh for every block, so a page can’t forge the end marker. Any marker-looking text inside the content is neutralized.
  • The person’s vault secrets are replaced with [secret redacted] if they show up in the text.
  • Every screenshot is followed by a short text note that says its text is data, and how many regions were redacted.
  • The decision contexts run_subtask and the monitor send to /v1/decisions wrap their screen text the same way.

​
Irreversible actions

One classifier decides what is irreversible, for every member and for run_subtask’s options: Labels match on whole words, so “Post” is publish and “Postal code” isn’t. Text, headings and fields are never classified, only controls. The executor reads what a step acts on from the Allternit Driver: the element id’s role and name for act and run_batch, an accessibility hit test under the point for pixel clicks, and the menu path for menu. In the gateway browser, element refs carry the role and name read_page and find returned. An irreversible step always needs the person’s confirmation, on every target, cloud computers included. Inside run_subtask, irreversible elements are not options unless constraints.allow_irreversible is set. When the subtask chooses one, it doesn’t run it. The subtask ends with status needs_confirmation and held_step, and the planner runs that step itself with act, which asks the person.

​
The per-step monitor

Before each mutating step on this device (contract risk risky or irreversible), the executor asks POST /v1/decisions one question of kind safety: allow, confirm or pause. The context is the step, its target element, the app and page, and a narrow read of the window. It runs on the fast tiers only (no oracle), with a 2 s budget (ALLTERNIT_SAFETY_MONITOR_BUDGET_MS). The safety kind’s confidence threshold is 0.9. Read-only steps (risk reversible: screenshots, read_ui, verify, scrolls, waits) are never sent to the monitor. They can’t change anything, so they fail open by design. They’re still audited, and their output is spotlighted and redacted. Cloud computers and gateway browsers are sandboxed, so the monitor doesn’t run there. Irreversible classes, watch mode and the lists still apply. Each monitored decision’s outcome is written back to the decision store for the flywheel. Turn it off for one computer with "monitor": false, or for the whole host with ALLTERNIT_SAFETY_MONITOR=off.

​
Watch mode

When a step lands in an email, banking or admin app or site, the computer is in watch mode:
  • Every step, reads included, is sent to the live view as a computer.safety event with phase: "watch".
  • Every mutating step needs the person’s confirmation first.
The built-in lists (returned by GET /computers/:id/safety):
  • Email: Mail, Outlook, Thunderbird, Spark, Superhuman, Mimestream, Airmail, Gmail, Outlook on the web, Yahoo, Proton, Fastmail, iCloud, HEY and others.
  • Banking: the major US and UK banks, brokers, PayPal, Venmo, Wise, Revolut, Mercury, Brex, Ramp, Coinbase, the Stripe dashboard, QuickBooks, Xero, and any host with “bank” in a label.
  • Admin: System Settings, Keychain Access, Passwords, Terminal, iTerm2, Disk Utility, and the AWS, Azure, Google Cloud, Google Admin, Microsoft 365, Cloudflare, Netlify, Vercel, Heroku, DigitalOcean, Hetzner, Okta, Auth0 and Shopify admin consoles, plus any admin. or console. host.
Add your own with watch_apps and watch_domains, or turn the built-ins off with "builtin_watch": false.

​
App and domain lists

  • scope is computer (this computer) or default (all of your computers). A computer’s own settings override the default field by field. GET returns computer, default, effective, the built-in watch lists and the monitor’s backends.
  • Settings can also be scoped to a project with PUT /api/v1/projects/:id/safety (the project must be yours). The chain for a toolset call is computer → project → owner default: the most specific scope that sets a field wins, exactly like the computer-over-default rule. GET /computers/:id/safety?project_id=… previews that chain for one computer.
  • Which project applies is decided by the run, not by the model: a call’s project is its run’s session project. A toolset request may also carry project_id, but it is verified — it must name one of your projects and match the run’s project, or the call is refused (403 safety_denied).
  • Apps match by name or bundle id; * globs. Domains: example.com covers its subdomains; *.example.com globs.
  • A deny entry always wins. When allow_apps is set, steps may only target those apps. When allow_domains is set, browser pages must be on those domains.
  • When a list is set and the executor can’t tell where a step lands, mutating steps are refused and reads are let through. The same goes for a browser page whose URL can’t be read.
  • The workspace host policy (ALLTERNIT_ACI_ALLOWED_HOSTS / ALLTERNIT_ACI_BLOCKED_HOSTS) and the declarative policy file apply on top.
Where a step lands comes from:
  • This device (macOS): the driver’s context method, which reports the app, bundle id, window title and, for browsers, the page URL from the window’s web area.
  • Linux cloud computers: the active X window’s class and title.
  • The gateway browser: the active tab’s URL.

​
Credentials stay in their domain

A credential with a bind types only into its app or domain:
  • use_credential checks the declared domain and, where the executor can see it, the app or page the agent is really typing into.
  • Every free-text path is screened for the person’s vault secrets: type, act / run_batch set_value and select, browser form_input, and run_subtask inputs.
    • A bound secret typed outside its binding is denied, and so is one typed where the executor can’t confirm the binding.
    • An unbound secret, or a bound one typed in the right place, is a credentials step that needs confirmation. Agents should use use_credential instead.

​
Screenshot redaction

Before a screenshot or zoom leaves the executor, personal data is blacked out on the full-size capture:
  • email addresses
  • card numbers (Luhn-checked)
  • phone numbers
  • US Social Security numbers
  • common API-key shapes
  • the person’s vault secrets
Text is located by OCR of the exact image. The preferred source is the target’s own driver — the driver’s ocr method (Apple Vision, fast level; set ALLTERNIT_REDACTION_OCR_LEVEL=accurate for the accurate level) answers on this Mac for every target, and when a cloud computer’s guest image carries the driver, its own OCR is preferred for its screenshots. When no driver OCR is available — a cloud-hosted allternit-api — the executor falls back to a built-in, pure-Rust OCR engine (the ocrs engine, MIT/Apache-2.0). Its models download once into the API’s data dir on first use, are pinned by URL and checked against a pinned SHA-256, and the engine runs off the async runtime with a size cap (40 MB PNG, 4096×4096 px) and a timeout. Its word boxes feed the same redaction code as the driver’s, so behaviour matches on every host. That fallback is why cloud computers and gateway browsers are fully redacted from a cloud-hosted API too, not best effort: as long as the models can be fetched, OCR runs. Set "redaction": "required" on a computer to refuse unredacted screenshots there — when OCR is unavailable for any reason (no driver, download blocked, timeout), no screenshot leaves.

​
Rollback snapshots

On cloud computers, the first risky step of a run (irreversible, watch mode, or a confirmation) takes a VM snapshot through the existing snapshot driver (snap-safety-…) before it runs:
  • The live view gets a computer.safety event with phase: "snapshot".
  • The result says which snapshot was saved.
  • Restore it with POST /api/v1/computers/:id/snapshots/:snapshot_id/restore.
There’s one snapshot per run (and at most one every 30 minutes for calls without a run id). A failed snapshot is logged and announced (snapshot_failed), and the confirmed step still runs. Turn snapshots off with "snapshot_before_risky": false.

​
Live-view events

computer.safety events go on the same stream as computer.action (the events websocket and GET /computers/:id/toolset/events):
phase is watch, confirm, paused, denied, snapshot or snapshot_failed. Steps refused inside a run_subtask carry within_subtask: true. watch is the watch category (email, banking, admin) or null.

​
In the app

Every computer live view (Allternit Desktop, the ai.allternit.com phone layout and the m.allternit.com app) shows these events over the screen:
  • confirm: a banner with the step, the app or site and the reason, and Approve / Deny buttons. They resolve the same approval (approval_id) through POST /api/aci/handoff/:id/approve|deny. The banner clears once the approval resolves or the next action arrives.
  • paused: “Paused by safety check” with the reason and Take over, which opens the person’s control of the computer.
  • denied: a dismissible notice with the reason, pointing to the computer’s safety settings.
  • watch: a “Watch mode” badge while consecutive steps are on a watch-listed app or site. It clears on the first ordinary step.
  • snapshot / snapshot_failed: a “Rollback point saved” or “Rollback point failed” toast.
Settings → Computers → Safety edits a computer’s lists (or the owner-wide default) through GET/PUT /computers/:id/safety and shows the effective result. The built-in watch lists and the monitor state are shown read-only.

​
Not measured yet

The prompt-injection benchmarks in the driver spec’s D5 row (an RTC-Bench subset and AgentDojo) haven’t been run. They’re metered model spend and wait for a cost estimate.