> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ai-coustics.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Pipecat quickstart
> Run a local audio loopback with speech enhancement, verify activation and add standalone voice activity detection.
**ai-coustics** enhances incoming audio inside Pipecat's input transport. This example sends microphone audio through `AICFilter` and returns it to your headphones. It does not require a speech-to-text (STT), language model or text-to-speech provider.
## Prerequisites
* **Use Python 3.11.** The example targets `pipecat-ai==1.11.0` and `aic-sdk==3.1.0`. Pipecat's `aic` extra requires the SDK's 3.1 series; do not force the 3.2 SDK used by the LiveKit example into this environment.
* **Get an SDK key.** Create one on the [developer platform](https://developers.ai-coustics.com/) and keep it on the server.
* **Use headphones and a microphone.** Open the local client in a browser with microphone permission. Headphones prevent feedback from the loopback.
* **Allow model provisioning and SDK authentication.** The first connection downloads model weights. See [authentication](/models/get-started/authenticate-apps) and [SDK telemetry](/reference/concepts/sdk-telemetry) for the separate network requirements.
## Installation
Create a directory and choose one setup below. Run the remaining commands there. In bash and zsh, keep package extras quoted.
```sh theme={null}
mkdir aic-pipecat-demo
cd aic-pipecat-demo
```
```sh uv theme={null}
uv venv --python 3.11
source .venv/bin/activate
uv pip install "pipecat-ai[aic,webrtc,runner]==1.11.0" "aic-sdk==3.1.0"
```
```sh pip theme={null}
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install "pipecat-ai[aic,webrtc,runner]==1.11.0" "aic-sdk==3.1.0"
```
The `runner` extra installs the local web client, FastAPI and Uvicorn. The `webrtc` extra supplies the browser transport. This example does not need PyAudio or the `local` extra.
On Windows, activate the environment with `.venv\Scripts\Activate.ps1` in PowerShell instead of `source .venv/bin/activate`.
## Usage
Save this complete file as `bot.py`:
```python bot.py theme={null}
import os
from pathlib import Path
from loguru import logger
from pipecat.audio.filters.aic_filter import AICFilter
from pipecat.frames.frames import Frame, InputAudioRawFrame, OutputAudioRawFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.transports.base_transport import TransportParams
from pipecat.workers.runner import WorkerRunner
class AudioFrameConverter(FrameProcessor):
def __init__(self):
super().__init__()
self.audio_frames = 0
self.audio_samples = 0
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, InputAudioRawFrame):
if frame.sample_rate != 16000 or frame.num_channels != 1:
raise ValueError("Expected 16 kHz mono PCM from the input transport")
if len(frame.audio) % 2:
raise ValueError("Expected complete signed 16-bit PCM samples")
self.audio_frames += 1
self.audio_samples += len(frame.audio) // 2
if self.audio_frames == 1 or self.audio_frames % 100 == 0:
logger.info(
"Loopback: frames={} samples={} rate={} channels={}",
self.audio_frames, self.audio_samples,
frame.sample_rate, frame.num_channels,
)
await self.push_frame(
OutputAudioRawFrame(
audio=frame.audio,
sample_rate=frame.sample_rate,
num_channels=frame.num_channels,
),
direction,
)
else:
await self.push_frame(frame, direction)
async def bot(runner_args: RunnerArguments):
key = os.environ.get("AIC_SDK_LICENSE", "").strip()
if not key:
raise RuntimeError("Set AIC_SDK_LICENSE in the terminal that starts bot.py")
aic_filter = AICFilter(
license_key=key,
model_id="quail-vf-2.2-l-16khz",
model_download_dir=Path("models"),
)
transport = await create_transport(
runner_args,
{
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
audio_out_enabled=True,
audio_in_channels=1,
audio_out_channels=1,
audio_in_filter=aic_filter,
),
},
)
pipeline = Pipeline([
transport.input(),
AudioFrameConverter(),
transport.output(),
])
worker = PipelineWorker(
pipeline,
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
),
)
runner = WorkerRunner(handle_sigint=runner_args.handle_sigint)
await runner.add_workers(worker)
@transport.event_handler("on_client_connected")
async def on_connected(transport, client):
logger.info("Client connected; check the AIC initialization log")
@transport.event_handler("on_client_disconnected")
async def on_disconnected(transport, client):
await runner.cancel()
await runner.run()
if __name__ == "__main__":
from pipecat.runner.run import main
main()
```
## Running the example
Set the key in the same terminal and start the server:
```sh bash / zsh theme={null}
export AIC_SDK_LICENSE="YOUR_LICENSE_KEY"
python bot.py -t webrtc -v
```
```powershell PowerShell theme={null}
$env:AIC_SDK_LICENSE = "YOUR_LICENSE_KEY"
python bot.py -t webrtc -v
```
Open [http://localhost:7860](http://localhost:7860), connect the client and allow microphone access. Speak, pause and speak again. The server creates a filter for each connection. Disconnect before restarting the server with Ctrl+C.
### Check the result
Check all three signals:
1. **Initialization:** verbose logs include `ai-coustics filter started:`, the selected model ID, sample rate and frames per chunk. `AIC model initialization failed` or `ai-coustics filter is not ready.` means enhancement did not activate.
2. **Audio flow:** `Loopback: frames=... samples=... rate=16000 channels=1` appears and its counts increase. You hear your microphone through the headphones. These counts confirm frame flow.
3. **Continued processing:** keep the connection open and check for SDK authentication or processing errors. Disconnect, reconnect and confirm initialization again.
If processor construction fails, `AICFilter` logs the failure and passes incoming audio through unchanged. Audible loopback alone is not proof of enhancement. Model download or model loading failures can instead stop startup. Fix the error and reconnect before evaluating the result.
Quail is intended for machine listening. Compare STT results on representative recordings when evaluating it; naturalness to a human listener and transcription accuracy are different measures.
### Recover from a failure
| Problem | Cause and fix | Retry check |
| - | - | - |
| zsh reports `no matches found` | Quote the full package specifier, including extras. | Rerun the installation command above. |
| The key error appears when connecting | The server's terminal has no `AIC_SDK_LICENSE`. Export it there and restart. | Reconnect and look for the initialization log. |
| Initialization or later authentication fails | Check the key and entitlement on the developer platform and review the SDK error. Do not print the key. | Restart and verify continued audio processing without the error. |
| Model download or loading fails | Check network access, free disk space and the writable `models` directory. Use a model compatible with SDK 3.1.0. | Restart, reconnect and verify the logged model ID. |
| The local client does not load | The `runner` extra or its `pipecat-ai-prebuilt` dependency is missing. | Repeat the pinned install and reopen the local URL. |
| No frames arrive | Check browser microphone permission and input device selection. | Reconnect and confirm that frame and sample counts increase. |
| Audio format error | The transport settings or converter were changed. Restore mono, 16 kHz signed 16-bit PCM. | Restart and confirm `rate=16000 channels=1`. |
## Architecture overview
`AICFilter` runs inside the input transport, before downstream processors. It converts mono signed 16-bit PCM to normalized float32 SDK blocks and returns signed 16-bit PCM. It buffers partial blocks, so a call can return no audio until a complete model block is available. Stopping the filter clears the remaining partial block and terminates its processing session.
```mermaid theme={null}
flowchart LR
input["Microphone"] --> filter["Input transport: AICFilter"]
filter --> downstream["Enhanced audio"]
downstream --> vad["Optional standalone VAD"]
vad --> stt["STT and agent pipeline"]
```
### AICFilter integration
At transport startup, the filter downloads or loads the model, creates `ProcessorAsync` for the transport sample rate and obtains a `ProcessorContext` for enhancement and bypass controls. It does not expose a VAD context. A process-wide model manager shares read-only enhancement weights; each filter has its own processing state.
To use a provisioned model, replace `model_id` and `model_download_dir` in `bot.py` with `model_path=Path("path/to/model.aicmodel")`. Use the path returned by `Model.download`; filenames can include a content identifier. SDK authentication is still required.
## Monitor SDK processing
[Enable OpenTelemetry for Pipecat](/models/get-started/opentelemetry#use-opentelemetry-with-pipecat) to send SDK processing and usage metrics to your collector. The guide covers shell and `.env` configuration, collector endpoints, containers and receipt checks.
## Standalone ai-coustics VAD
`AICQuailVADAnalyzer` performs independent voice activity detection (VAD). In Pipecat 1.11.0 its default model ID is `vad-2.1-xxs-16khz`. Enhancement models cannot be used in its place.
To test VAD with the loopback above, add these imports to `bot.py`:
```python theme={null}
from pipecat.audio.vad.aic_quail_vad import AICQuailVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams
from pipecat.processors.audio.vad_processor import VADProcessor
```
Inside `bot`, replace the `pipeline = Pipeline(...)` block with:
```python theme={null}
vad_analyzer = AICQuailVADAnalyzer(
license_key=key,
model_id="vad-2.1-xxs-16khz",
model_download_dir=Path("models"),
params=VADParams(confidence=0.6, start_secs=0.2, stop_secs=0.8),
)
pipeline = Pipeline([
transport.input(),
VADProcessor(vad_analyzer=vad_analyzer),
AudioFrameConverter(),
transport.output(),
])
```
Restart with `-v` and reconnect. As you speak and pause, look for `AICQuailVADAnalyzer initialized`, then `User started speaking` and `User stopped speaking` from `VADProcessor`. This loopback checks speech boundaries; it runs no LLM and cannot measure agent response latency.
With `audio_in_filter=aic_filter`, the VAD receives enhanced audio. The SDK VAD is intended for the original signal, and this combination also adds enhancement delay before VAD prediction. Pipecat 1.11.0 does not provide a pre-enhancement audio tap to this downstream analyzer. Evaluate missed speech, false starts and end-of-turn latency on your own recordings. For VAD on original audio, remove `audio_in_filter=aic_filter` from this example.
For an existing conversational pipeline, replace the loopback analyzer's `params` argument with:
```python theme={null}
params=VADParams(confidence=0.6, start_secs=0.2, stop_secs=0.2),
```
Pass that analyzer to `LLMUserAggregatorParams(vad_analyzer=vad_analyzer)` when creating `LLMContextAggregatorPair`. Let that aggregator own VAD instead of also adding `VADProcessor`. `TransportParams.vad_analyzer` was removed in Pipecat 1.0.
The loopback uses `stop_secs=0.8`. Pipecat 1.11.0's conversational `TurnAnalyzerUserTurnStopStrategy` assumes `stop_secs=0.2` in its built-in STT latency estimates. A stop delay at or above that estimate leaves no additional transcript wait time and can delay turn detection. For another delay, measure the 99th-percentile time from speech end to final transcript with your VAD/STT settings. Pass it in seconds as `ttfs_p99_latency` to the STT service constructor, such as `DeepgramSTTService`. Restart and reconnect the conversational pipeline to check timing warnings before measuring response latency. A custom estimate still produces the nondefault VAD warning; the loopback does not run these checks.
Tune Pipecat's `VADParams`: `confidence` is the probability threshold, and `start_secs` and `stop_secs` govern speech timing. The SDK arguments `sensitivity`, `speech_hold_duration` and `minimum_speech_duration` have been accepted but ignored since Pipecat 1.5.0 and are scheduled for removal in 2.0.0.
## Adapt this example
Continue with the [Pipecat 1.11.0 conversational example](https://github.com/pipecat-ai/pipecat/blob/v1.11.0/examples/voice/voice-aicoustics.py), which connects Deepgram speech-to-text (STT), an OpenAI large language model (LLM) and Cartesia text-to-speech (TTS). Keep the pinned Pipecat version from this guide and add the `deepgram`, `openai`, `cartesia` and `daily` extras to the installation command above. This released example imports the Daily transport even when you select WebRTC, so it needs the `daily` extra; the WebRTC path does not need Daily credentials.
The example reads `AIC_SDK_LICENSE`, `DEEPGRAM_API_KEY`, `OPENAI_API_KEY` and `CARTESIA_API_KEY`; configure any additional credentials required by your chosen transport. These are server-side environment variables.
The example gives VAD ownership to `LLMContextAggregatorPair` through `LLMUserAggregatorParams`; do not also add the loopback's `VADProcessor`. It applies enhancement before VAD, so review the [original-audio and timing constraints](/models/get-started/pipecat-quickstart#standalone-ai-coustics-vad) above. It also writes conversation audio to local WAV files; review that recording behavior before using it with other people's audio.
* **Evaluate changes one at a time.** Keep the input recording, STT version and VAD settings fixed when comparing models or enhancement levels. Measure VAD timing separately from loopback delay.
* **Keep streams separate.** Create a filter and VAD analyzer per concurrent stream. Let pipeline cancellation finish cleanup before reusing or discarding that session.
* **Upgrade deliberately.** Preserve your environment lock and model IDs, then rerun initialization, speech/silence/noise, disconnect and reconnect checks after changing Pipecat or SDK versions. `PipelineWorker` and `WorkerRunner` replace the deprecated task/runner aliases used in older examples.
## Further reading
After checking your pipeline, [evaluate audio and agent behavior](/guides/evaluate-audio), then [prepare your deployment](/production/deployment). Use [troubleshooting](/production/troubleshooting) to diagnose setup and runtime failures.
Read Pipecat's filter documentation.
Understand VAD models and the distinction between SDK and Pipecat timing controls.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.