Python voice activity detection

Package: aic-sdk==3.3.0. Core SDK: 0.25.0. Source: Python wrapper 3.3.0. The fragments below use these imports. Supply license_key from your approved secret source and use the loaded model and initialized objects described in each section. For a complete file-processing example and tuning guidance, follow the ai-coustics VAD guide.

import typing
import numpy as np
import numpy.typing as npt
from aic_sdk import EnergyVadContext, Model, Processor, ProcessorAsync, ProcessorConfig, Vad, VadAsync, VadContext, VadParameter, OtelConfig

audio denotes a one-dimensional NumPy float32 array. Async fragments run inside an async function.

Vad

class

Stateful voice activity detector (VAD) backed by a dedicated VAD model. Use one instance per independent stream and serialize initialization and processing on each instance. Native resources are released when Python releases the object. To detect speech with an enhancement model instead of a dedicated VAD model, use EnergyVadContext. Feed mono audio to process() and read predictions through get_context(). The audio is not modified; processing only updates the detector’s prediction. When enhancement and VAD run together, feed the VAD the original input audio, not the enhanced output of Processor.process(). Run both on the same block instead of chaining them:

vad.process(audio)                   # reads the block, does not modify it
enhanced = processor.process(audio)  # enhances the same original block

Example

model = Model.from_file("/path/to/vad_model.aicmodel")
config = ProcessorConfig.optimal(model)
vad = Vad(model, license_key, config)
vad_context = vad.get_context()
audio = np.zeros(config.block_size, dtype=np.float32)
vad.process(audio)
print(vad_context.is_speech_detected())

Vad() constructor

Vad(
    model: Model,
    license_key: str,
    config: ProcessorConfig | None = None,
    otel_config: OtelConfig | None = None,
) -> Vad

Creates a voice activity detector. The model must be a dedicated VAD model, such as vad-ms-2.1-xxs-16khz. Enhancement models raise ModelTypeUnsupportedError; for an enhancement model, use Processor.get_energy_vad_context(). If config is provided, the VAD is initialized immediately. Otherwise, call initialize() before processing audio. Parameters

Model

required

A loaded dedicated VAD model. See Model.

str

required

SDK key or JWT for the ai-coustics SDK.

ProcessorConfig | None

default:"None"

Optional audio configuration. See ProcessorConfig.

OtelConfig | None

default:"None"

Optional per-instance OpenTelemetry configuration. See OtelConfig.

Vad.initialize()

def initialize(self, config: ProcessorConfig) -> None

Configures the VAD for a sample rate and block size. Unsupported configurations raise AudioConfigUnsupportedError. For the most frequent prediction updates, use ProcessorConfig.optimal(model). Parameters

ProcessorConfig

required

Audio configuration. See ProcessorConfig.

This method allocates memory and is not real-time safe.

Vad.process()

def process(self, audio: npt.NDArray[np.float32]) -> None

Accepts a one-dimensional NumPy float32 array of normalized mono samples. Its length must equal config.block_size, or be 1 through config.block_size when variable blocks are enabled. Native work releases the GIL. A contiguous array is read directly; a strided view is copied. Do not mutate a shared input while the call runs. Wrong dtype or dimensionality raises TypeError. Returns None: VAD processing does not modify the audio. Read the updated prediction through get_context(). When enhancement and VAD run together, pass the original input audio here, not the enhanced output of Processor.process(). Raises

NotInitializedError: If the VAD has not been initialized.
AudioConfigMismatchError: If the block size does not match the configuration.
ProcessingNotAllowedError: If processing is not authorized.

Vad.get_context()

def get_context(self) -> VadContext

Returns a VadContext for reading predictions and controlling the VAD.

Vad.terminate_session()

def terminate_session(self) -> None

Terminates the VAD’s telemetry session. Stop submitting audio and treat this session as closed once you request termination. Processing becomes disallowed when the native lifecycle task handles the signal. The call can return before that handling completes when other sessions remain alive; it is not proof of remote usage acknowledgment. The session is also stopped when the object is destroyed.

This method may block and is not real-time safe.

VadAsync

class

Async voice activity detector backed by a dedicated VAD model. Awaitable processing runs on the shared SDK processing pool. The constructor and get_context() are synchronous; both can block. Omit config at construction and await initialize_async(config) to initialize asynchronously. Await blocks in stream order and bound pending work. See async enhancement for pool settings and concurrency. When enhancement and VAD run together, feed the VAD the original input audio, not the enhanced output of ProcessorAsync.process_async().

VadAsync() constructor

VadAsync(
    model: Model,
    license_key: str,
    config: ProcessorConfig | None = None,
    otel_config: OtelConfig | None = None,
) -> VadAsync

Creates an async voice activity detector. The model must be a dedicated VAD model, such as vad-ms-2.1-xxs-16khz. Enhancement models raise ModelTypeUnsupportedError; for an enhancement model, use ProcessorAsync.get_energy_vad_context(). Parameters

Model

required

A loaded dedicated VAD model. See Model.

str

required

SDK key or JWT for the ai-coustics SDK.

ProcessorConfig | None

default:"None"

Optional audio configuration. See ProcessorConfig.

OtelConfig | None

default:"None"

Optional per-instance OpenTelemetry configuration. See OtelConfig.

VadAsync.initialize_async()

async

def initialize_async(self, config: ProcessorConfig) -> typing.Awaitable[None]

Returns an awaitable resolving to None after initialization. Unsupported configurations raise AudioConfigUnsupportedError when awaited. The configuration is copied into the operation.

VadAsync.process_async()

async

def process_async(self, audio: npt.NDArray[np.float32]) -> typing.Awaitable[None]

Copies the one-dimensional NumPy float32 input before dispatch, then updates the VAD prediction in the background. Await it to obtain None. Input remains unchanged; the length, dtype and SDK error rules of Vad.process() apply. After awaiting, read the updated prediction through get_context(). When enhancement and VAD run together, pass the original input audio here, not the enhanced output of ProcessorAsync.process_async().

VadAsync.get_context()

def get_context(self) -> VadContext

Returns a VadContext for reading predictions and controlling the VAD.

VadAsync.terminate_session_async()

async

def terminate_session_async(self) -> typing.Awaitable[None]

Terminates the VAD’s telemetry session asynchronously. Await the returned awaitable and stop submitting audio. Treat the session as closed immediately; native processing becomes disallowed when the lifecycle task handles the signal. Other live sessions can allow termination handling to continue after the awaitable resolves. Await completion does not prove remote usage acknowledgment.

VadContext

class

Shared control handle for a Vad. There is no public VadContext() constructor. Contexts created by the same Vad reference the same detector. They can be used from any thread while audio is being processed elsewhere.

VadContext.is_speech_detected()

def is_speech_detected(self) -> bool

Returns the post-processed VAD prediction. The prediction lags its input by get_prediction_delay() samples. If the backing Vad stops being processed, the prediction does not update.

VadContext.raw_vad_probability()

def raw_vad_probability(self) -> float

Returns the VAD model’s raw speech probability without SDK post-processing. The prediction lags its input by get_prediction_delay() samples.

VadContext.set_parameter()

def set_parameter(self, parameter: VadParameter, value: float) -> None

Modifies a VAD parameter. Out-of-range values, including NaN, raise ParameterOutOfRangeError. Query get_parameter() for the value used by the current model. Parameters

VadParameter

required

Parameter to modify. See VadParameter.

float

required

New parameter value.

VadContext.get_parameter()

def get_parameter(self, parameter: VadParameter) -> float

Retrieves the current value of a VAD parameter.

VadContext.parameter()

deprecated

Deprecated. Use get_parameter() instead.

def parameter(self, parameter: VadParameter) -> float

VadContext.get_prediction_delay()

def get_prediction_delay(self) -> int

Returns the total VAD prediction delay in samples. This includes input reblocking, model processing and buffering overhead for the current configuration. Use it to align speech decisions with the input timeline. This delay is not applied to the audio: Vad.process() leaves its input untouched. The value only describes how far behind its input the published prediction is, and it is independent of ProcessorContext.get_audio_delay().

VadContext.reset()

def reset(self) -> None

Clears the VAD’s internal state and published predictions. The VAD remains initialized. Reset clears the published predictions immediately and requests internal processing state reset for the next block. Immediately after reset(), is_speech_detected() is False and raw_vad_probability() is 0.0.

VadContext.update_bearer_token()

def update_bearer_token(self, token: str) -> None

Replaces the bearer token on the running VAD. Both the original key and new token must be JWTs. Otherwise TokenUnsupportedError is raised. Embedded NUL characters raise LicenseFormatInvalidError. A successful update is not proof of backend acceptance; continue handling processing errors.

EnergyVadContext

class

Control handle for an enhancement processor’s energy-based voice activity detector. Detection uses the enhanced signal before output mixing and runs no separate VAD model, so an enhancement model provides speech decisions without loading a dedicated VAD model. There is no public EnergyVadContext() constructor. Create one with Processor.get_energy_vad_context() or ProcessorAsync.get_energy_vad_context(). Contexts from the same processor share one detector and can be used from any thread. Creating a context keeps enhancement inference active even when the processor is bypassed or its enhancement level is zero. Inference stays active for the processor’s lifetime, even after every context is released. A context remains usable after the processor is destroyed but receives no further audio, and releasing a context neither destroys the processor nor disables detection. Sensitivity uses the energy range, 1.0 to 15.0, rather than the 0.0 to 1.0 probability threshold of a dedicated VAD model. See VadParameter.Sensitivity. Example

model = Model.from_file("/path/to/enhancement_model.aicmodel")
config = ProcessorConfig.optimal(model)
processor = Processor(model, license_key, config)
vad_context = processor.get_energy_vad_context()
vad_context.set_parameter(VadParameter.Sensitivity, 6.0)
audio = np.zeros(config.block_size, dtype=np.float32)
enhanced = processor.process(audio)
print(vad_context.is_speech_detected())

EnergyVadContext.is_speech_detected()

def is_speech_detected(self) -> bool

Returns the current speech prediction. This is False before processing and after reset(). It updates as the processor processes audio and lags its input by get_prediction_delay() samples.

EnergyVadContext.set_parameter()

def set_parameter(self, parameter: VadParameter, value: float) -> None

Modifies an energy VAD parameter. Parameters can be changed from any thread during processing. Parameters

VadParameter

required

Parameter to modify. See VadParameter.

float

required

New parameter value. Sensitivity ranges from 1.0 to 15.0 on this context.

Raises

ParameterOutOfRangeError: If the parameter value is out of range.

Example

vad_context.set_parameter(VadParameter.Sensitivity, 6.0)
vad_context.set_parameter(VadParameter.SpeechHoldDuration, 0.08)

EnergyVadContext.get_parameter()

def get_parameter(self, parameter: VadParameter) -> float

Returns an energy VAD parameter’s current value. Parameters

VadParameter

required

Parameter to query. See VadParameter.

Returns

float: The current parameter value.

Example

sensitivity = vad_context.get_parameter(VadParameter.Sensitivity)

EnergyVadContext.get_prediction_delay()

def get_prediction_delay(self) -> int

Returns the prediction delay in samples for the current audio configuration. The delay covers input reblocking, STFT and model processing, and matches the processor’s ProcessorContext.get_audio_delay(). Energy detection adds no audio delay. SpeechHoldDuration and MinimumSpeechDuration also affect decision timing but are not included. Before initialization the value uses the model’s optimal block size and native sample rate. After initialization it includes input buffering for the configured block size and is expressed in samples at the configured sample rate. A nonoptimal or variable block size adds buffering delay, which is included. Returns

int: The delay in samples.

Example

print(f"Prediction delay: {vad_context.get_prediction_delay()} samples")

EnergyVadContext.reset()

def reset(self) -> None

Clears the energy VAD’s state and prediction. Use this after an interruption or seek to discard predictions from earlier audio. Parameters are retained and the processor is not reset. ProcessorContext.reset() also resets the energy VAD.

Concurrency. The underlying SDK reset is real-time safe. Python calls still involve the interpreter; this is not a hard real-time guarantee for a Python callback.

Example

vad_context.reset()

VadParameter

enum

Parameter constants for voice activity detection. Use named constants directly; the runtime exposes PyO3 enum-like objects rather than promising standard-library enum.Enum iteration or .value behavior.

VadParameter members

VadParameter.SpeechHoldDuration

Controls how long the VAD continues to detect speech after the audio signal no longer contains speech. This affects the stability of speech detected -> not detected transitions. When the current probability is at or below the threshold, speech remains detected while at least half of the retained history was above threshold. The history spans approximately speech_hold_duration * 2 seconds. When the current probability is above threshold, the consecutive-frame test uses MinimumSpeechDuration. For example, a speech_hold_duration of 0.5 s sustains detection for 0.5 s after speech stops. Additional speech blocks during that period can extend detection until the 50% history threshold is no longer met.

Timing is rounded to the nearest model window: for a 10 ms window, to the nearest multiple of 10 ms. get_parameter() returns the stored requested value, not the effective transition timing.

Range: 0.0 to 300x model window length (seconds) Default: model-specific

VadParameter.Sensitivity

Threshold used to decide whether speech is detected. Its meaning and range depend on the detector. Vad and VadAsync run a dedicated VAD model, which outputs a speech probability for each processed audio block. A value above this threshold triggers a speech-detected decision. EnergyVadContext compares the energy left in the enhanced signal against 10 ** -sensitivity, so higher values detect quieter speech. Range: 0.0–1.0 for dedicated VAD models; 1.0–15.0 for the energy VAD Default: model-specific

VadParameter.MinimumSpeechDuration

Controls how long speech needs to be present in the audio signal before the VAD considers it speech. This affects the stability of speech not detected -> detected transitions.

Timing is rounded to the nearest model window: for a 10 ms window, to the nearest multiple of 10 ms. get_parameter() returns the stored requested value, not the effective transition timing.

Range: 0.0–1.0 (seconds) Default: model-specific See the Python API index, Python guide and troubleshooting.

SDK

API reference

Upgrades

LiveKit

AI tools

Historical migrations

Python voice activity detection

Vad

Vad() constructor

Vad.initialize()

Vad.process()

Vad.get_context()

Vad.terminate_session()

VadAsync

VadAsync() constructor

VadAsync.initialize_async()

VadAsync.process_async()

VadAsync.get_context()

VadAsync.terminate_session_async()

VadContext

VadContext.is_speech_detected()

VadContext.raw_vad_probability()

VadContext.set_parameter()

VadContext.get_parameter()

VadContext.parameter()

VadContext.get_prediction_delay()

VadContext.reset()

VadContext.update_bearer_token()

EnergyVadContext

EnergyVadContext.is_speech_detected()

EnergyVadContext.set_parameter()

EnergyVadContext.get_parameter()

EnergyVadContext.get_prediction_delay()

EnergyVadContext.reset()

VadParameter

VadParameter members

VadParameter.SpeechHoldDuration

VadParameter.Sensitivity

VadParameter.MinimumSpeechDuration

​Vad

​Vad() constructor

​Vad.initialize()

​Vad.process()

​Vad.get_context()

​Vad.terminate_session()

​VadAsync

​VadAsync() constructor

​VadAsync.initialize_async()

​VadAsync.process_async()

​VadAsync.get_context()

​VadAsync.terminate_session_async()

​VadContext

​VadContext.is_speech_detected()

​VadContext.raw_vad_probability()

​VadContext.set_parameter()

​VadContext.get_parameter()

​VadContext.parameter()

​VadContext.get_prediction_delay()

​VadContext.reset()

​VadContext.update_bearer_token()

​EnergyVadContext

​EnergyVadContext.is_speech_detected()

​EnergyVadContext.set_parameter()

​EnergyVadContext.get_parameter()

​EnergyVadContext.get_prediction_delay()

​EnergyVadContext.reset()

​VadParameter

​VadParameter members

​VadParameter.SpeechHoldDuration

​VadParameter.Sensitivity

​VadParameter.MinimumSpeechDuration

Vad

Vad() constructor

Vad.initialize()

Vad.process()

Vad.get_context()

Vad.terminate_session()

VadAsync

VadAsync() constructor

VadAsync.initialize_async()

VadAsync.process_async()

VadAsync.get_context()

VadAsync.terminate_session_async()

VadContext

VadContext.is_speech_detected()

VadContext.raw_vad_probability()

VadContext.set_parameter()

VadContext.get_parameter()

VadContext.parameter()

VadContext.get_prediction_delay()

VadContext.reset()

VadContext.update_bearer_token()

EnergyVadContext

EnergyVadContext.is_speech_detected()

EnergyVadContext.set_parameter()

EnergyVadContext.get_parameter()

EnergyVadContext.get_prediction_delay()

EnergyVadContext.reset()

VadParameter

VadParameter members

VadParameter.SpeechHoldDuration

VadParameter.Sensitivity

VadParameter.MinimumSpeechDuration