Voice Service

Overview

The Allternit Voice Service (services/voice, crate voice-service) is a Rust HTTP sidecar used by Gizzi Code and Allternit Desktop. Both directions are real and run on sherpa-onnx inside the single binary — no cloud voice APIs, no Python, no whisper.cpp:

STT: Silero VAD finds speech segments; Moonshine tiny (English, default) or Parakeet TDT 0.6B v3 (accurate) transcribes them. Any sample rate is accepted (resampled to 16 kHz); 8 kHz phone audio including G.711 μ-law WAV works.
TTS: Kokoro-82M v1.0 (fp32, Apache-2.0), 28 English voices (default af_heart), 24 kHz, run by the separate program allternit-tts (GPL-3.0-or-later, services/voice-tts), which the service starts as a child process.

Voice cloning is not part of this service. The remaining product surfaces (Desktop speak button, voice picker UI, Gizzi /speak) are specified in docs/specs/tts-product.md. Status: Implementation IMPLEMENTED (STT + TTS) / Exposure INTERNAL.

Models

Model packs download on first use (sha256-verified, resumable) into ~/.allternit/models/voice/<pack>/; nothing model-related ships in the installer. Poll GET /v1/models for state (missing, downloading with pct, ready, error).

Pack	Contents	Size
`small` (default)	Silero VAD + Moonshine tiny EN + Smart Turn v3.2	~39 MB
`tts`	Kokoro-82M v1.0 fp32 (first text-to-speech use)	~350 MB
`accurate`	Parakeet TDT 0.6B v3 int8 (CC-BY-4.0)	~487 MB

Packs download from our own mirror, https://runtime.allternit.com/voice-packs/v1/<file>, and fall back to the upstream URL (GitHub or Hugging Face) if the mirror fails (logged once). The same pinned sha256 is checked either way. ALLTERNIT_VOICE_MODEL_BASE overrides the base and is then used alone, at <base>/<asset>. See Owned downloads.
ALLTERNIT_VOICE_THREADS sets inference threads (default 2), CPU provider — sized for a ~5-year-old 8 GB laptop.
ALLTERNIT_VOICE_MODEL_DIR moves the pack directory.
POST /v1/models/<pack> starts a download ahead of first use.

Licences

Component	Licence
sherpa-onnx (engine)	Apache-2.0
onnxruntime	MIT
Silero VAD	MIT
Moonshine tiny EN	MIT
Kokoro-82M	Apache-2.0
Smart Turn v3.2	BSD-2-Clause
Parakeet TDT 0.6B v3	CC-BY-4.0, attribution required
espeak-ng (Kokoro phonemiser, only in `allternit-tts`)	GPL-3.0-or-later
allternit-tts (the TTS program)	GPL-3.0-or-later

Parakeet attribution: Speech recognition by Parakeet TDT 0.6B v3 by NVIDIA, licensed under CC-BY-4.0. Show it wherever the accurate option is offered. Full list: services/voice/THIRD_PARTY_NOTICES.md.

Base URL

The service binds to 127.0.0.1 on PORT (default 8001).

http://127.0.0.1:8001

cargo run -p voice-service
# or
./services/voice/start.sh

The sherpa-onnx and onnxruntime native libraries link statically (crate sherpa-onnx default static feature) on macOS, Windows (static CRT) and Linux. Desktop ships two files in resources/bin: allternit-voice-service (STT and the API, no GPL code, checked by scripts/check-voice-no-gpl.sh) and allternit-tts (Kokoro TTS, GPL-3.0-or-later, started by the service as a child process that exits with it). No dylib/DLL accompanies them.

REST endpoints

Method	Path	Description
GET	`/health`, `/v1/health`	Health: engine, threads, readiness, pack states
GET	`/v1/models`	Pack download states
POST	`/v1/models/:pack`	Start a pack download in the background
GET	`/v1/voices`	List the 28 Kokoro English voices
GET	`/v1/voices/:id`	Get one voice (404 if unknown)
GET	`/v1/stt/models`	List STT models (moonshine, parakeet)
POST	`/v1/tts`	Synthesise → `audio/wav` bytes (or `format: "pcm16"`)
POST	`/v1/tts/stream`	Sentence-streaming TTS (NDJSON per-sentence pcm16)
POST	`/v1/stt`	Transcribe uploaded audio (multipart; WAV any rate, or raw pcm16 + `sample_rate`)
POST	`/v1/stt/stream`	Streaming STT: chunked pcm16 in (`?sample_rate=`), NDJSON partial/final out
GET, POST	`/v1/sessions`	List or create sessions
GET, DELETE	`/v1/sessions/:id`	Get or delete a session
GET	`/v1/stats`	Service metrics

Health check

curl http://127.0.0.1:8001/v1/health

{
  "service": "voice",
  "status": "healthy",
  "version": "0.1.0",
  "timestamp": 1712345678901,
  "features": ["tts", "stt", "streaming"],
  "engine": "sherpa-onnx",
  "stt_ready": true,
  "stt_accurate_ready": false,
  "tts_ready": true,
  "num_threads": 2,
  "packs": [
    { "name": "small", "state": "ready", "pct": null, "error": null, "size_bytes": 203032121 },
    { "name": "accurate", "state": "missing", "pct": null, "error": null, "size_bytes": null }
  ]
}

Transcribe audio

Send the audio as a multipart form upload (audio field; optional language, model=moonshine|parakeet). WAV containers of any sample rate are decoded and resampled to 16 kHz; raw s16le 16 kHz PCM is also accepted.

curl -X POST http://127.0.0.1:8001/v1/stt -F "[email protected]"

{
  "text": "hello world",
  "confidence": 0.9,
  "language": "en",
  "segments": [
    { "start_time": 0.0, "end_time": 1.4, "text": "hello world", "confidence": 0.9 }
  ]
}

Speak text

curl -X POST http://127.0.0.1:8001/v1/tts \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello from Allternit", "voice": "af_heart"}' \
  --output hello.wav

The first STT request downloads the small pack (~39 MB) and the first TTS request the tts pack (~350 MB, a few minutes on a typical connection, including unpacking); poll GET /v1/models for progress, or start them early with POST /v1/models/<pack>. Gizzi Code’s /voice and Desktop dictation wait up to 5 minutes for that first request.

Performance

Measured with examples/voice_bench.rs (CPU only, 2 threads, 40 LibriSpeech test-clean utterances; clean 16 kHz, phone line, and raw 8 kHz μ-law). Per-host results: services/voice/bench/results-<host>.jsonl.

Faster first speech

Kokoro takes a fraction of a second to a couple of seconds to render the first words on a weak CPU. The voice service shortens that wait four ways:

Short first chunk. The first sentence is cut after 2–4 words, so the first audio is a very small render.
Warm start. On a server that takes phone calls, set ALLTERNIT_VOICE_WARM=1 so the voice service loads its speech models when it starts, not on the first call; the first call after a restart is then as quick as any other. The desktop app leaves it off and loads the models the first time you use voice.
Phrase cache. Common acknowledgements (“Sure.”, “Okay.”, “Got it.”) and the lines a call opens with are rendered once per voice and kept in phrase-cache/ inside the model directory. A cached phrase plays in about a millisecond. The cache is capped by ALLTERNIT_VOICE_PHRASE_CACHE_MB (default 64, 0 turns it off), and only fixed phrases are stored; ordinary replies are never written to disk.
Warm start. The speech engine renders a short sentence at startup, so your first reply isn’t a cold start.
Fillers. A Voice Session started with fillers: true says a short phrase such as “One moment.” if the reply hasn’t started 1.2 seconds after you stop talking, and another every 6 seconds (up to 3) while a slow reply is still pending. Phone calls turn this on; the in-app voice session leaves it off. See services/voice/spec/VOICE_SESSION.md for the protocol.

Get Started

Allternit Factory

Workspace workflows

Architecture

Work Runtime

Cognitive Runtime

Context & Memory

Tools & Execution

Verification & Safety

Observability & Evaluation

Kernel Contracts

Build on Allternit

Operations

Reference

Core Concepts

Gizzi Code CLI

Tools & ACI

Security & Enterprise

Admin API

BYOC Runtime

Surfaces

CI CD

Guides

Cookbooks

Release Notes

Overview

Models

Licences

Base URL

REST endpoints

Health check

Transcribe audio

Speak text

Performance

Faster first speech

​Overview

​Models

​Licences

​Base URL

​REST endpoints

​Health check

​Transcribe audio

​Speak text

​Performance

​Faster first speech

​Related pages

Overview

Models

Licences

Base URL

REST endpoints

Health check

Transcribe audio

Speak text

Performance

Faster first speech

Related pages