Real-Time Analysis with Tyto

Tyto can score a live stream while a call is running, so your application can react to bad audio during the call instead of after it. This page explains the two objects involved, where each one belongs in your application, and how to read the results. For scoring complete recordings you already hold in memory, use the file analyzer instead. See Batch Call Analysis.

For LiveKit Agents, the ai-coustics-maintained plugins provide an Analyzer that schedules inference and exposes its collector as a frame processor. See Audio-quality analysis with LiveKit.

Two objects, two threads

Analysis models process multiple audio blocks at once. Running that work inside a real-time audio callback would create CPU spikes that could interrupt audio playback. To keep callback processing predictable, the SDK separates audio collection from analysis. In Python, Rust, C++ and C, these roles use two objects that are created together and share one analysis model. The examples below use Python. Node.js and WASM expose both roles through one Analyzer object. In Node.js, use analyzeAsync() to run analysis on a worker thread while continuing to collect audio (see Entry points per language).

Object	Where it belongs	What it does
Collector	Your audio thread or callback	Buffers each mono block. Real-time safe, and it does not modify your audio.
Analyzer	A thread of your own	Runs the analysis model over the audio the collector holds. Not real-time safe.

Nothing is analyzed on its own. The collector only keeps audio, and the model runs when you call the analyzer. The two are safe to use at the same time: the collector may keep buffering on the audio thread while an analysis is in progress.

Never call the analyzer from an audio callback. A forward pass of an analysis model takes far longer than a real-time audio block, so it will cause dropouts.

The collector holds a rolling window whose length is determined by the analysis model, which is five seconds for Tyto. As new samples arrive, the oldest audio is discarded, so each analysis scores the most recent five seconds of the stream.

Set up the pair

Create the pair from an analysis model, then initialize the collector the same way you would initialize a processor. Analysis models are the only model type accepted here, so passing an enhancement or VAD model raises ModelTypeUnsupportedError.

import aic_sdk as aic

model = aic.Model.from_file(aic.Model.download("tyto-1.1-l-16khz", "./models"))
collector, analyzer = aic.analyzer_pair(model, license_key)

# Configure the collector for the audio you will feed it.
config = aic.ProcessorConfig.optimal(model)
collector.initialize(config)

initialize allocates memory, so call it during setup rather than on an audio thread. The collector takes mono float32 blocks and accepts the same variable_block_size flag as the processor. See Audio Format.

Buffer and analyze

Feed the collector wherever you receive audio, and drive the analyzer from a separate thread on a timer:

import threading

def on_audio_block(block):
    # Mono float32, config.block_size samples. Read-only, the collector never modifies it.
    collector.buffer(block)

def analysis_loop(stop: threading.Event):
    smoothed = None

    while not stop.wait(1.0):  # analyze once per second
        result = analyzer.analyze_buffered()

        # Raw per-window scores are intentionally responsive. Smooth before acting on them.
        smoothed = (
            result.risk_score
            if smoothed is None
            else 0.3 * result.risk_score + 0.7 * smoothed
        )

        if smoothed > 0.5:
            print(f"Degraded audio, risk {smoothed:.2f}, noise {result.noise:.2f}")

A one-second interval gives a smooth score timeline over a five-second window, the same cadence the batch tutorial uses. Analyzing more often than that mostly re-scores audio you have already seen, at full model cost each time.

Every call returns a complete AnalysisResult: the Tyto Risk Score plus all six dimensions. Use the smoothing and aggregation guidance before triggering interventions, so one aberrant window cannot flip your application’s behavior.

The first few seconds

The analysis model always consumes a fixed length of audio. If you analyze before the collector has buffered that much, the tail of the input is analyzed as silence, which skews the scores toward whatever a partly silent window looks like. Wait until the stream has run for one full window before you act on a score. Counting the samples you have passed to the collector is enough:

buffered_samples = 0
window_samples = 5 * config.sample_rate  # Tyto's fixed 5 s window

def on_audio_block(block):
    global buffered_samples
    collector.buffer(block)
    buffered_samples += len(block)

def window_is_full() -> bool:
    return buffered_samples >= window_samples

Reset between streams

Reset when the stream is interrupted, when you seek, or when you reuse the pair for a different call. Calling analyzer.reset() clears the paired collector’s buffered audio, and the collector stays initialized to its configured audio settings:

analyzer.reset()

Resetting the Python pair is real-time safe, so it is fine to call from an audio callback. Remember to clear your own sample counter alongside it, since a reset empties the window.

Running alongside enhancement and VAD

The collector reads its input without modifying it, exactly like the VAD. All three objects can therefore share one input block, and you should give the collector the original audio so Tyto scores what actually arrived from the user:

collector.buffer(audio)             # reads the block
vad.process(audio)                  # reads the block
enhanced = processor.process(audio) # returns the enhanced block

Each object needs its own model, and each may report a different optimal configuration. Initializing all of them for the format your host delivers is fine. See Using VAD alongside enhancement for the same pattern applied to the VAD.

When analysis stops being allowed

analyze_buffered() raises ProcessingNotAllowedError when the SDK key was not authorized or usage reporting failed, most often because the machine lost its internet connection. This can happen mid-stream and not only at startup, so handle it in your analysis loop rather than treating a successful setup as proof that analysis will keep working:

while not stop.wait(1.0):
    try:
        result = analyzer.analyze_buffered()
    except aic.ProcessingNotAllowedError:
        # Keep the audio path running and retry on the next tick.
        continue

Enhancement and VAD report the same condition on their own process calls, so an application that runs all three should expect it in each of them.

Entry points per language

Language	Create	Buffer	Analyze
Python	`aic.analyzer_pair(model, key)`	`collector.buffer(audio)`	`analyzer.analyze_buffered()`
Rust	`aic_sdk::analyzer_pair(&model, key)`	`collector.buffer(&audio)`	`analyzer.analyze_buffered()`
Node.js	`new Analyzer(model, key)` (collector included)	`analyzer.buffer(samples)`	`analyzer.analyze()` / `await analyzer.analyzeAsync()`
C++	`aic::AnalyzerPair::create(model, key)`	`collector.buffer(audio.data(), audio.size());`	`analyzer.analyze_buffered()`
C	`aic_analyzer_pair_create(&collector, &analyzer, model, key)`	`aic_collector_buffer(...)`	`aic_analyzer_analyze_buffered(...)`
WASM	`new Analyzer(model, key)` (collector included)	`analyzer.buffer(audio)`	`analyzer.analyze()`

In Node.js, call analyzer.initialize(sampleRate, blockSize, false) before buffering mono Float32Array blocks. analyze() blocks the calling thread; analyzeAsync() runs inference on a libuv worker thread so the event loop remains available and buffer() can continue collecting audio during analysis.

Find out more

Tyto: Audio Insight

What Tyto measures, how to interpret each dimension, and how to pick thresholds.

Batch Call Analysis

Score a folder of recordings offline and explore them in the dashboard.

SDK Examples

Runnable analysis examples across the SDK bindings, including the combined Analyzer in Node.js and the analyzer/collector pair in C.

Audio Format

Mono requirements, block sizes, and what each audio entry point expects.

Get started

Voice Focus

Audio Insight

Voice Activity Detection

Speech Enhancement

Perceptual SE

Older Models

Two objects, two threads

Set up the pair

Buffer and analyze

The first few seconds

Reset between streams

Running alongside enhancement and VAD

When analysis stops being allowed

Entry points per language

Find out more

Tyto: Audio Insight

Batch Call Analysis

SDK Examples

Audio Format

​Two objects, two threads

​Set up the pair

​Buffer and analyze

​The first few seconds

​Reset between streams

​Running alongside enhancement and VAD

​When analysis stops being allowed

​Entry points per language

​Find out more

Tyto: Audio Insight

Batch Call Analysis

SDK Examples

Audio Format

Two objects, two threads

Set up the pair

Buffer and analyze

The first few seconds

Reset between streams

Running alongside enhancement and VAD

When analysis stops being allowed

Entry points per language

Find out more