​
The rule

Voice is input/output only. The agent loop doesn’t move.
  • A finished spoken turn (turn.ended) is sent through the same function the composer’s Send button calls. Tools, approvals, steering and the thread behave as if it were typed.
  • The assistant reply the thread already renders is streamed to the engine as speak.delta / speak.done.
  • Barge-in (speak.interrupted) calls the composer’s Stop.
There is no separate voice path to the model.

​
Modules (src/lib/voice/)

UI lives in src/components/voice/ (VoiceSession hosts VoiceBar and VoiceFullScreen; VoiceOrb wraps the brand AllternitOrb; SpokenLabel) and src/components/settings/VoiceSettingsPanel.tsx.

​
Surfaces

All three surfaces build from the same components: Phone sizing comes from useIsPhoneViewport(): full-screen controls are 60 px round, padding respects env(safe-area-inset-*), and the PWA’s interactive-widget=resizes-content viewport keeps the bar above the keyboard. Under prefers-reduced-motion the orb is still (its own CSS) and the level ring is off; every state is also text (role="status").

​
Endpoint resolution

  • Device (Desktop only): window.allternit.voice.getSessionEndpoint() → { wsUrl?, httpUrl?, port?, token? } from the preload bridge. Desktop builds without that bridge fall back to http://127.0.0.1:8001 (platform PORTS.VOICE) and no token. isAvailable() false → engine-not-installed. Web and the PWA never connect to loopback.
  • Cloud: POST /api/v1/voice/tickets (allternit-api via runtimeApiUrl, bearer auth) → { ticket, url?, expiresAt? }. The session URL is url or VITE_VOICE_CLOUD_URL; it must be wss:// or the client refuses. Status mapping: no token or 401 → signed-out; 402 → no-cloud-minutes; other errors → cloud-unavailable (retried with backoff).
  • Automatic (chooseEngine): device when packs report ready and the measured RTF ≤ 0.9 (averaged over the first 3 utterances; unmeasured counts as fine); else cloud when signed in; else a message naming what’s missing.
RTF is measured only on audio received after the client sent speak.done, so waiting for model text doesn’t count against the device.

​
Engine HTTP contract assumed by the UI (Phase 1 engine)

Parsed defensively; confirm or adjust with the engine owner.
  • GET /v1/models → { ready?: boolean, packs: [{ id, name, sizeBytes, state: "installed"|"downloading"|"available", progress?: 0–1 or 0–100 }] }. A missing endpoint is shown as “Voice engine not installed”.
  • POST /v1/models/{id}/download starts a pack download. The “Get Accurate pack” button looks for a pack whose id is accurate.
  • GET /v1/voices → ["af_heart", …] or { voices: [{ id, name?, language?, sampleUrl? }] }. Samples default to GET /v1/voices/{id}/sample (any browser-playable audio).

​
Errors and states

​
Phone calls: joinCallAudio

joinCallAudio(callId, { mode: 'listen' | 'talk', token, url, room }) in src/lib/gateway/call-webrtc.ts. The caller (CallScreen) gets {token, url, room} from the listen or takeover call control and passes them in; the function doesn’t fetch tokens. It lazy-loads livekit-client (Apache-2.0), connects to url (production: wss://livekit.allternit.com), plays every remote audio track through hidden <audio data-call-audio> elements, unlocks playback with startAudio() (the call runs from a click), publishes the mic only in talk, and returns { callId, leave }. It resolves null (and disconnects) when credentials are missing, the connect fails, or the mic is refused in talk, so the UI never claims a person is on the call when they aren’t.

​
Tests

src/lib/voice/__tests__/: an in-memory mock Voice Session server (mock-voice-server.ts) drives session-client.test.ts; conversation.test.ts, voice-lib.test.ts (PCM, the shipped playback worklet, endpoints, Automatic mode, engine parsing) and call-audio.test.ts (fake LiveKit room). UI: src/components/voice/__tests__/.

​
What replaced what

The quick voice bar and full-screen voice replace components/chat/VoiceModeOverlay.tsx and components/ai-elements/voice-call-mode.tsx. The orphaned ai-elements/voice-{presence,overlay,session,toolbar,controls,selector,visualizer,provider}.tsx were removed with the ShellApp VoicePresence mount. Dictation (useComposerVoice) keeps Desktop’s IPC transcription and uses a Voice Session elsewhere instead of a loopback /v1/stt. The legacy providers/voice-provider + services/voice (bot voice previews/TTS on Desktop) remain for the bot voice settings panel.