ai-coustics VAD

The ai-coustics SDK includes dedicated Voice Activity Detection models for real-time applications. Use VAD for general speech detection, or VAD Voice Focus when you need to detect only the primary speaker.

These models were previously called Quail VAD and Quail VAD Voice Focus. The product names dropped the Quail prefix, and the model IDs now start with vad-. See Renamed Models.

What is VAD?

A Voice Activity Detector is a system that identifies the presence of human speech in an audio stream. Voice agents use this signal for turn-taking, endpointing, and deciding when to send audio to downstream Speech-to-Text (STT) systems. VAD is a standalone model trained for noisy, multi-speaker environments. It predicts speech activity directly from the input audio and does not require Quail Voice Focus, Quail, or Rook enhancement to run. This makes it the recommended choice when you need a modular VAD component in a Voice AI pipeline. VAD Voice Focus is the primary-speaker counterpart. It predicts speech activity for the targeted primary speaker only, ignoring interfering and background speech, mirroring how Quail Voice Focus isolates the primary speaker for enhancement. Use it when turn-taking should be driven by a single target speaker rather than any audible voice.

Use Cases

Integrating the VAD into your pipeline can significantly improve your application’s performance and user experience.

Improved Turn-Taking: In voice agent or conversational AI applications, the VAD provides a reliable signal for detecting the end of a user’s turn.
Cost Reduction: By processing audio only when speech is present, you can reduce computational load and downstream processing costs (e.g., for STT).
Enhanced STT Accuracy: Sending audio to your Speech-to-Text engine only when speech is present can reduce insertions and substitutions caused by background noise or interfering speech.

How it Works

The VAD is a standalone object with its own lifecycle, independent of enhancement:

Load a dedicated VAD model, vad-2.1-xxs-16khz or vad-vf-2.0-s-16khz
Create a VAD from that model and initialize it for your sample rate and block size
Feed each mono audio block to the VAD
Query its context to see whether speech was detected

The model produces a speech probability for each processed block, which the SDK compares against the configured sensitivity threshold. Processing does not modify your audio, so the VAD can read the same buffer you pass to an enhancement processor.

import aic_sdk as aic
import numpy as np

model = aic.Model.from_file(aic.Model.download("vad-2.1-xxs-16khz", "./models"))
config = aic.ProcessorConfig.optimal(model)

vad = aic.Vad(model, license_key, config)
context = vad.get_context()

audio = np.zeros(config.block_size, dtype=np.float32)
vad.process(audio)

print(context.is_speech_detected())
print(context.raw_vad_probability())

Before SDK 0.22.0, the VAD came from an enhancement processor and could be derived from the enhanced signal’s energy. That energy-based VAD has been removed. See Migrate to the dedicated VAD.

The VAD keeps a history of the audio it has seen, so reset it through its context when the stream is interrupted, when you seek, or when you reuse it for a new call:

context.reset()

This clears the VAD’s state and immediately drops the published decision and probability, so queries cannot return stale values from the previous stream. The VAD stays initialized. See Streams and State.

Model Differences

Model	How it works	Recommended use
VAD (`vad-2.1-xxs-16khz`)	Predicts speech probability directly from the input audio, for any audible speaker.	New Voice AI pipelines, modular LiveKit or Pipecat setups, and noisy environments where background speech, music, or impulsive noise can trigger false positives.
VAD Voice Focus (`vad-vf-2.0-s-16khz`)	Predicts speech probability for the primary speaker only.	Single-target-speaker turn-taking where interfering speakers should not trigger the VAD.

Both models run independently of enhancement. Neither requires Quail Voice Focus, Quail, or Rook to be present in the signal chain.

Using VAD alongside enhancement

Run the VAD and the enhancement processor side by side on the same original input block, rather than chaining them:

vad.process(audio)                  # reads the block, does not modify it
enhanced = processor.process(audio) # returns the enhanced block

Feeding the VAD the enhanced output means detecting speech in audio that no longer matches what the VAD model expects, and it stacks the processor’s delay on top of the VAD’s own prediction delay. The two delays are independent and are reported separately: the processor’s context reports how far the enhanced audio lags its input, and the VAD’s context reports how far the published prediction lags the same input.

VAD Parameters

You can fine-tune the VAD’s behavior using the following parameters. All three defaults are read from the model file, so they are model-specific rather than fixed SDK constants. Read a parameter from the VAD context to see the value in effect for the loaded model, and treat that value as the starting point for tuning instead of hardcoding one.

Sensitivity

The probability threshold used to decide whether speech is detected.The VAD model outputs a probability of speech presence for each processed audio block, where 1.0 means the model is certain speech is present and 0.0 means it is certain speech is not present. A probability above this threshold triggers a speech detected decision.Higher values require more confidence from the model, so they detect speech less aggressively.Read raw_vad_probability() on the VAD context to see the model’s probability before thresholding and post-processing. This is the most practical way to choose a threshold from recorded audio.

Range: 0.0 to 1.0
Default: model-specific

Before SDK 0.22.0, sensitivity on an energy-based VAD was an energy threshold in the range 1.0 to 15.0, and higher values detected speech more aggressively. Those values are not valid probabilities and the direction is inverted, so they cannot be carried across. See Retune sensitivity.

Speech Hold Duration

Controls for how long the VAD continues to detect speech after the audio signal no longer contains speech.This affects the stability of speech detected -> not detected transitions. The VAD reports speech detected if the audio signal contained speech in at least 50% of the frames processed in the last speech_hold_duration * 2 seconds. For example, if speech_hold_duration is set to 0.5 seconds and the VAD stops detecting speech in the audio signal, the VAD will continue to report speech for 0.5 seconds assuming the VAD does not detect speech again during that period. If a few frames of speech are detected during that period, those frames will be included in the 50% calculation, which will extend the speech detection period until the 50% threshold is no longer met.

The VAD returns a value per processed buffer, so this duration is rounded to the closest model window length. For example, on a model with a 15 ms window the VAD rounds up/down to the closest multiple of 15 ms. Because of this, this parameter may return a different value than the one it was last set to.

Unit: Seconds
Range: 0.0 to 300x the model window length. Read the upper bound from the VAD context rather than assuming it
Default: model-specific

Minimum Speech Duration

Controls for how long speech needs to be present in the audio signal before the VAD considers it speech.This affects the stability of speech not detected -> detected transitions.

Unit: Seconds
Range: 0.0 to 1.0
Default: model-specific

Best Practices

Use VAD by default: It is the general-purpose model and the recommended option for new Voice AI integrations.
Tune Sensitivity: The optimal sensitivity may vary depending on your audio source, environment, and turn-taking behavior. Start with the default and adjust against real production audio.
Primary-speaker VAD: Use VAD Voice Focus to trigger the VAD only when the primary speaker is talking, instead of chaining Quail Voice Focus enhancement with the general VAD.
Feed the VAD your original audio: Run it beside an enhancement processor on the same input block, not on the processor’s output.
Align decisions with the prediction delay: Read the VAD context’s own delay rather than the processor’s audio delay.

Third-Party VADs

The enhanced audio output from our models will have reduced noise and reverberation, which can improve the accuracy of downstream VADs that may struggle in noisy conditions. However, when using a speech enhancement model with an enhancement level lower than 1.0, the output will have a noise component which helps improve ASR accuracy, but it may harm downstream VAD models if they are not robust to noise. For new integrations, use the ai-coustics VAD instead of adding a third-party VAD when possible. It is noise-robust, runs in the ai-coustics SDK and does not require Torch or ONNX runtime dependencies. If ASR accuracy is not a concern, you can still use ai-coustics enhancement models as a pre-processing step for third-party VADs:

If you need foreground speaker isolation, use one of our Quail Voice Focus models before the VAD to suppress background noise and interfering speech, so a generic VAD only triggers on the primary speaker.
If you need general speech detection, use one of our Rook models, which will preserve both foreground and background speech while suppressing non-speech sounds. Use an enhancement level of 100% for best VAD performance.

Get started

Voice Focus

Audio Insight

Voice Activity Detection

Speech Enhancement

Perceptual SE

Older Models

What is VAD?

Use Cases

How it Works

Model Differences

Using VAD alongside enhancement

VAD Parameters

Best Practices

Third-Party VADs

​What is VAD?

​Use Cases

​How it Works

​Model Differences

​Using VAD alongside enhancement

​VAD Parameters

​Best Practices

​Third-Party VADs

What is VAD?

Use Cases

How it Works

Model Differences

Using VAD alongside enhancement

VAD Parameters

Best Practices

Third-Party VADs