Sipral ships an example voice agent that needs no key and no account: whisper.cpp listens, a small model in Ollama answers, and a local voice speaks, all on one machine. Until today that voice was macOS’s own say. It worked only on a Mac, sounded like one, and started again for every sentence.
What changed
The agent now speaks with Pocket TTS, Kyutai’s small text-to-speech model. Its code is MIT and its weights CC BY 4.0, and nothing in the chain is GPL, which is why it was chosen over voices that depend on espeak-ng. It is loaded once, when the agent starts, and each sentence goes into the call in 80 ms pieces as they are made, so the caller hears the first words while the rest is still being produced. When the caller speaks over the agent, the voice stops at once.
It runs on the processor, with no graphics card. On Linux it replaces nothing, because there was no voice there before; on a Mac without it, say is still the fallback.
Starting at the first pause
A turn still ends after 700 ms of quiet, so the agent does not talk over a caller who only paused for breath. But the work no longer waits for it: at the first 200 ms of quiet the agent already transcribes, asks the model and starts the voice, and holds all of it back. If the caller goes on, the prepared answer is thrown away unheard; if the quiet holds, it is ready when the turn ends. The first piece of the answer also ends at its first comma (“Of course,”) instead of its first full sentence. Pauses of 350 and 600 ms in the middle of a question, on real calls, still got one answer, to the whole question.
The numbers
Six real calls between two Sipral stacks on one Apple M2, timed from the end of the caller’s question to the first audio the caller receives:
- with macOS’s
say: 3.4 to 5.1 seconds; - with Pocket TTS, waiting for the end of the turn: 1.08 to 1.61 seconds;
- with Pocket TTS, starting at the first pause: 0.62 to 0.80 seconds, all six under a second.
A caller who talks over the agent now silences it in about 0.1 seconds. The machine was doing other work at the same time, so these are not best cases; under heavy load the answers slow down, as the README shows. How to repeat them is in the agents package README.
Trying it
From a checkout of the repository:
cargo build --release -p sipral-ffi
python3 -m pip install websockets cffi './integrations/agents[local]'
export SIPRAL_LIBRARY=target/release PYTHONPATH=integrations/agents:bindings/python
python3 integrations/agents/examples/local_agent.py --port 5070On Linux, add PyTorch’s CPU index to the pip install, as the README explains, so no graphics libraries come with it. The first run fetches the model and its voice, about 225 MB. The voice speaks English, French, German, Spanish, Portuguese, Italian and Dutch; there is no Romanian model yet. For other languages, the same agent can use any of the speech services Sipral already connects to.
Licences, in short
Nothing of Pocket TTS ships with Sipral: your machine installs it and fetches its model. The code is MIT. The model and the default voice are CC BY 4.0, so an agent that callers hear should credit them, for example “Voice: Kyutai Pocket TTS (CC BY 4.0)”. Some other voices in Kyutai’s catalogue are for non-commercial use only, so check a voice before you pick it. Two small helper packages pip brings along, certifi and tqdm, are under MPL-2.0, a file-level licence that puts no obligation on your own code. The full list is in THIRD-PARTY-NOTICES.md.