realtime-multivoice-tts-client (1.0.0rc4)
Installation
pip install --index-url About this package
Typed HTTP and protocol-v3 streaming client for Realtime Multivoice TTS
Realtime Multivoice TTS Client
Lightweight typed clients for the Realtime Multivoice TTS HTTP API and raw protocol-v3 stream. This distribution does not install the server, NumPy, VoXtream, or audio playback dependencies.
The canonical wire protocol and identifier/playback semantics are documented in
docs/protocol-v3.md.
gh release download v1.0.0rc4 --pattern 'realtime_multivoice_tts_client-*.whl'
python -m pip install realtime_multivoice_tts_client-1.0.0rc4-py3-none-any.whl
Synchronous Streaming
SyncStreamingClient accepts any Iterable[str], sends each chunk unchanged,
then sends exactly binary b"<EOT>". It receives JSON controls and binary PCM
concurrently and keeps the connection for sequential cycles completed by eos.
from realtime_multivoice_tts_client import PcmEvent, SyncStreamingClient
with SyncStreamingClient("http://127.0.0.1:7860", cycle_timeout_seconds=300) as client:
for event in client.stream(["<ANNOUNCER>Hello ", "world"]):
if isinstance(event, PcmEvent):
consume_pcm(event.data)
Closing an iterator before eos, malformed or mismatched controls, sender
errors, and disconnects close and invalidate the socket. Input is never replayed.
cycle_timeout_seconds is optional and covers the entire cycle through eos;
expiry raises the public StreamingTimeout error. Use separate client instances
when normal and control traffic need different budgets, such as 300 and 10
seconds.
Asynchronous Streaming
StreamingClient provides the same events and guarantees for an
AsyncIterable[str]:
async with StreamingClient("http://127.0.0.1:7860", cycle_timeout_seconds=300) as client:
async for event in client.stream(text_chunks):
consume(event)
Both clients return shared frozen dataclasses: ConfigEvent, PcmEvent,
SegmentEvent, IdentifierEvent, WarningEvent, InterruptEvent, ErrorEvent,
and EosEvent.
Both raise StreamingTimeout when their optional whole-cycle budget expires.
Pass on_identifier to either constructor to receive each server identifier
event immediately before it is yielded. The async client accepts a synchronous
function or an async function and awaits the latter within the cycle timeout.
Listener exceptions abort the cycle and invalidate its connection.
Generated sync and async HTTP clients are constructed with create_http_clients
and create_async_http_clients; public HTTP models live in
realtime_multivoice_tts_client.models.
Identifier Playback Tracking
Inline <IDENTIFIER=value> tags produce IdentifierEvent with
IdentifierStage.SYNTHESIZED. Feed every stream event into the dependency-free,
thread-safe PlaybackTracker, and advance it by the number of sample frames the
audio device actually consumes. It returns each identifier exactly once with
IdentifierStage.PLAYED:
from realtime_multivoice_tts_client import (
IdentifierEvent,
PcmEvent,
PlaybackTracker,
SyncStreamingClient,
)
tracker = PlaybackTracker()
with SyncStreamingClient(
"http://127.0.0.1:7860",
on_identifier=on_identifier,
) as client:
for event in client.stream(["<P1>Hello.<IDENTIFIER=greeting>Next sentence."]):
for played in tracker.observe(event):
on_identifier(played)
if isinstance(event, PcmEvent):
frames_played = play_pcm(event.data)
for played in tracker.advance(frames_played):
on_identifier(played)
Call advance when frames are consumed by the playback callback, not when they
are merely queued. Positions are measured in sample frames, not scalar channel
samples or bytes. On interrupt, observe returns IdentifierStage.SKIPPED for
every synthesized identifier playback had not reached, then resets the tracker
to the new epoch. Identifiers cancelled before synthesis arrive from the server
with the same stage and a null sample index.
on_identifier receives server-emitted SYNTHESIZED, PLAYED, and SKIPPED
events. In stream output mode, playback-derived PLAYED and SKIPPED events are
created locally by PlaybackTracker; dispatch those returned events to the same
listener as shown above.
Local streaming playback
SyncPlaybackClient wraps the protocol-v3 client and automatically plays PCM
when the server advertises output_mode=stream. In speakers mode it passes
through the server's events without opening local audio output.
from realtime_multivoice_tts_client import IdentifierStage, SyncPlaybackClient
def on_identifier(event):
if event.stage is IdentifierStage.PLAYED:
print("Played:", event.identifier)
client = SyncPlaybackClient("http://tts:7860", on_identifier=on_identifier)
try:
for event in client.stream(["<ANNOUNCER>Hello!<IDENTIFIER=done>"]):
pass
finally:
client.close()
The default Linux sink requires paplay (pulseaudio-utils on Debian/Ubuntu)
and a reachable PulseAudio or PipeWire-Pulse server. Use the user's normal audio
session, or set PULSE_SERVER=unix:/path/to/pulse/native and optionally
PULSE_SINK. Containers must run under the audio session's UID and mount that
socket with host-path creation disabled. No Python audio/model dependency is
added. Other platforms can supply sink_factory(config) -> AudioSink.
Callbacks run on the iterator's consuming thread. Synthesized markers are still
reported, but a local played event is emitted only after a successful sink
drain. The default sink drains/reopens at marker boundaries; very frequent
markers may introduce small pauses. It does not estimate progress from elapsed
time or count writing bytes into a pipe as playback. Interrupt controls abort
old output, skip unreached markers, and reset the epoch before replacement PCM.
Cancellation, callback/sink failure, timeout, or early iterator close aborts
local audio and invalidates the underlying transport, without replaying text.
Completed eos is withheld until local playback drains. The cycle timeout
includes this final drain. Audio reception runs independently of playback so
interrupts can be received during a blocked write/drain; queued PCM is bounded
by max_buffer_seconds (default 10) and overflow fails closed. Set the optional
positive integer max_buffer_bytes to replace that duration-derived cap with
an exact PCM-byte cap, independent of sample rate and channel count. For example,
SyncPlaybackClient(url, max_buffer_bytes=1024**3) permits up to 1 GiB of pending
PCM, including an in-flight sink write. This is not a total-process RAM limit:
object metadata, network buffers, and audio-device buffers are separate. The
cycle deadline and task-count safety limit remain independent.
PCM is consumed directly by the sink; the playback iterator yields control
events, not duplicate PCM buffers. Local terminal marker events are deduplicated
by cycle and occurrence, so a later remote skip cannot revoke a reached marker.
Custom sinks implement write(data, *, deadline), drain(*, deadline), and
abort(). Deadlines use time.monotonic(). abort() must promptly unblock
concurrent writes/drains, permanently reject further writes, and discard audio
instead of draining it. A fresh sink is created for each cycle/interrupt epoch.