> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ai-coustics.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LiveKit quickstart

> Add dedicated voice activity detection and Tyto analysis to a LiveKit room with complete Python and Node.js examples.

**ai-coustics** provides dedicated voice activity detection (VAD) and Tyto audio-quality analysis for LiveKit Agents.

<Note>
  Use the official LiveKit plugin for speech enhancement. Start with LiveKit's [voice agent quickstart](https://docs.livekit.io/agents/start/voice-ai/) and [noise cancellation documentation](https://docs.livekit.io/transport/media/noise-cancellation/).

  The ai-coustics-maintained plugins provide standalone VAD and Tyto analysis, billed through ai-coustics. These capabilities are unavailable in the compared official-plugin release.
</Note>

Create this VAD agent in a separate project from any existing official-plugin enhancement agent. See [which LiveKit plugin to use](/reference/livekit/plugin-scope) for package scope and coexistence limits.

## Packages

| Runtime | Example versions | Import |
| - | - | - |
| Python 3.11 | `ai-coustics-livekit-plugin==0.2.0`, `aic-sdk==3.2.0`, `livekit-agents==1.8.2` | `from livekit.plugins import ai_coustics` |
| Node.js 24 | `@ai-coustics/livekit-plugin@0.2.0`, `@ai-coustics/aic-sdk@0.24.0`, `@livekit/agents@1.9.0` | `@ai-coustics/livekit-plugin` |

Both SDK packages wrap core SDK 0.24.0.

<Warning>
  Python plugin 0.2.0 and the official `livekit-plugins-ai-coustics` package write to the same `livekit.plugins.ai_coustics` namespace. Install them in separate environments. The Node packages have distinct import paths, but their VAD metadata and component APIs are not interchangeable. Do not mix implementations in the audio chain shown here.
</Warning>

## Prerequisites

* **Prepare a LiveKit Cloud project.** The complete examples use LiveKit Inference for speech-to-text (STT), a large language model (LLM) and text-to-speech (TTS). Obtain `LIVEKIT_URL`, `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET` from your project. The [LiveKit quickstart](https://docs.livekit.io/agents/start/voice-ai/) covers account setup and the agent playground.
* **Get an ai-coustics SDK key.** Create it on the [developer platform](https://developers.ai-coustics.com/). The ai-coustics-maintained plugin uses this key separately from LiveKit credentials.
* **Install your runtime.** Use Python 3.11 and `uv`, or Node.js 24 and npm. Use a microphone and headphones for the room test.
* **Allow model downloads and authentication.** The examples provision one dedicated VAD model before starting the worker. See [authentication](/models/get-started/authenticate-apps) and [SDK telemetry](/reference/concepts/sdk-telemetry) for network requirements.

Self-hosted LiveKit users can use these plugins with their own LiveKit connection and separately configured STT, LLM and TTS providers. The LiveKit Inference provider configuration below is the Cloud path.

## Setup guide

<Steps>
  <Step title="Create a separate project">
    Run one setup from an empty working directory:

    <CodeGroup>
      ```sh Python theme={null}
      mkdir aic-livekit-python
      cd aic-livekit-python
      uv venv --python 3.11
      source .venv/bin/activate
      uv pip install "ai-coustics-livekit-plugin==0.2.0" "aic-sdk==3.2.0" "livekit-agents==1.8.2" "python-dotenv==1.2.2"
      ```

      ```sh Node.js theme={null}
      mkdir aic-livekit-node
      cd aic-livekit-node
      npm init -y
      npm pkg set type=module
      npm install --save-exact @ai-coustics/livekit-plugin@0.2.0 @ai-coustics/aic-sdk@0.24.0 @livekit/agents@1.9.0 @livekit/rtc-node@0.13.35 dotenv@17.4.1
      npm install --save-dev --save-exact tsx@4.20.6 typescript@5.9.3 @types/node@24.10.1
      ```
    </CodeGroup>

    Run the remaining commands from this project directory. On Windows, activate Python with `.venv\Scripts\Activate.ps1` in PowerShell.
  </Step>

  <Step title="Configure credentials">
    Create `.env.local` in the project directory:

    ```dotenv .env.local theme={null}
    LIVEKIT_URL=wss://YOUR_PROJECT.livekit.cloud
    LIVEKIT_API_KEY=YOUR_LIVEKIT_API_KEY
    LIVEKIT_API_SECRET=YOUR_LIVEKIT_API_SECRET
    AIC_SDK_LICENSE=YOUR_LICENSE_KEY
    ```

    Replace each placeholder. Both agents explicitly load this file. Keep it out of source control:

    ```gitignore .gitignore theme={null}
    .env.local
    .venv/
    node_modules/
    models/
    ```
  </Step>

  <Step title="Provision the VAD model">
    Download the dedicated model to `models/`:

    <CodeGroup>
      ```sh Python theme={null}
      python -c 'from aic_sdk import Model; from shutil import copyfile; path = Model.download("vad-ms-2.1-xxs-16khz", "models"); print(copyfile(path, "models/vad.aicmodel"))'
      ```

      ```sh Node.js theme={null}
      node --input-type=module -e 'import { Model } from "@ai-coustics/livekit-plugin"; import { copyFileSync } from "node:fs"; const path = await Model.download("vad-ms-2.1-xxs-16khz", "models"); copyFileSync(path, "models/vad.aicmodel"); console.log("models/vad.aicmodel");'
      ```
    </CodeGroup>

    The command uses the returned download path and copies the file to `models/vad.aicmodel`, the stable path used below. Keep the project working directory unchanged so the agent can load it. These plugins do not register models with LiveKit's `download-files` command.
  </Step>

  <Step title="Save the complete agent">
    Save the file for your runtime. Models load once per worker process; each room creates its own stateful VAD.

    <CodeGroup>
      ```python agent.py theme={null}
      import logging
      import os

      from dotenv import load_dotenv
      from livekit.agents import (
          Agent, AgentServer, AgentSession, JobContext, JobProcess,
          cli, inference, room_io,
      )
      from livekit.plugins import ai_coustics

      load_dotenv(".env.local")
      logger = logging.getLogger("aic-example")


      def setup(proc: JobProcess):
          proc.userdata["vad_model"] = ai_coustics.Model.from_file(
              "models/vad.aicmodel"
          )


      server = AgentServer(setup_fnc=setup)


      @server.rtc_session(agent_name="aic-vad-demo")
      async def entrypoint(ctx: JobContext):
          key = os.environ.get("AIC_SDK_LICENSE", "").strip()
          if not key:
              raise RuntimeError("Set AIC_SDK_LICENSE in .env.local and restart the agent")

          vad = ai_coustics.VAD(model=ctx.proc.userdata["vad_model"], license_key=key)

          @vad.on("metrics_collected")
          def on_vad_metrics(metrics):
              logger.info("AIC VAD inference count: %s", metrics.inference_count)

          session = AgentSession(
              vad=vad,
              stt=inference.STT("deepgram/nova-3", language="multi"),
              llm=inference.LLM("openai/gpt-4.1-mini"),
              tts=inference.TTS("cartesia/sonic-3"),
          )
          await session.start(
              agent=Agent(instructions="You are a helpful voice assistant. Keep replies brief."),
              room=ctx.room,
              room_options=room_io.RoomOptions(
                  audio_input=room_io.AudioInputOptions(
                      sample_rate=16000,
                      num_channels=1,
                      noise_cancellation=vad.processor,
                  ),
              ),
          )
          await ctx.connect()


      if __name__ == "__main__":
          cli.run_app(server)
      ```

      ```ts agent.ts theme={null}
      import { cli, defineAgent, inference, ServerOptions, voice } from '@livekit/agents';
      import { Model, VAD } from '@ai-coustics/livekit-plugin';
      import dotenv from 'dotenv';
      import { fileURLToPath } from 'node:url';

      dotenv.config({ path: '.env.local' });

      export default defineAgent({
        prewarm: (proc) => {
          proc.userData.vadModel = Model.fromFile('models/vad.aicmodel');
        },
        entry: async (ctx) => {
          const key = process.env.AIC_SDK_LICENSE?.trim();
          if (!key) throw new Error('Set AIC_SDK_LICENSE in .env.local and restart the agent');

          const vad = new VAD({ model: ctx.proc.userData.vadModel as Model, licenseKey: key });
          const session = new voice.AgentSession({
            vad,
            stt: new inference.STT({ model: 'deepgram/nova-3', language: 'multi' }),
            llm: new inference.LLM({ model: 'openai/gpt-4.1-mini' }),
            tts: new inference.TTS({ model: 'cartesia/sonic-3' }),
          });
          await session.start({
            agent: new voice.Agent({ instructions: 'You are a helpful voice assistant. Keep replies brief.' }),
            room: ctx.room,
            inputOptions: {
              audioSampleRate: 16000,
              audioNumChannels: 1,
              noiseCancellation: vad.processor,
            },
          });
          await ctx.connect();
        },
      });

      cli.runApp(new ServerOptions({
        agent: fileURLToPath(import.meta.url),
        agentName: 'aic-vad-demo',
      }));
      ```
    </CodeGroup>

    Install `vad.processor` in RoomIO as shown. It runs inference and attaches results to the original audio frame for the session's VAD stream to read. Passing `vad` to the session alone does not connect it to microphone audio.
  </Step>

  <Step title="Run and join a room">
    <CodeGroup>
      ```sh Python theme={null}
      python agent.py dev
      ```

      ```sh Node.js theme={null}
      npx tsx agent.ts dev
      ```
    </CodeGroup>

    Open your LiveKit project's agent playground, select `aic-vad-demo` and connect with microphone access. Use a room connection for this check: Python's `console` mode does not exercise the RoomIO input path used here. Speak, pause and speak again.

    ### Check the result

    * **Worker connection:** the worker registers with your LiveKit project and the playground dispatches `aic-vad-demo` into a room.
    * **Python VAD activation:** confirm repeated positive `AIC VAD inference count:` values while microphone audio arrives, with no inference errors.
    * **Node.js VAD activation:** run the local diagnostic below to verify SDK inference through the plugin. The pinned `@livekit/agents` 1.9.0 counts speech-start events in its VAD metrics counter, so its `metrics_collected` event cannot reliably confirm inference during a short conversation. The local diagnostic does not verify microphone routing in your room.
    * **Conversation:** the selected STT/LLM/TTS pipeline responds. Inspect provider errors separately from VAD initialization and inference errors.
    * **Cleanup:** disconnect and start a new playground session. The new room must initialize again without VAD errors; Python must resume positive inference counts. Do not reuse the closed frame processor from the previous room.

    This example runs dedicated VAD on unchanged audio, without enhancement. Evaluate speech boundaries and response timing on speech, silence and noise before tuning VAD.
  </Step>
</Steps>

## Check Node.js VAD locally

After provisioning `models/vad.aicmodel`, run this Node.js diagnostic with your ai-coustics credential. It sends 1.5 s of silence through the frame processor and reads VAD events through a separate local stream. No LiveKit room is needed; the check verifies inference execution, not detection quality or room routing.

<Accordion title="Save and run check-vad.ts">
  ```ts check-vad.ts theme={null}
  import { initializeLogger, VADEventType } from '@livekit/agents';
  import { AudioFrame } from '@livekit/rtc-node';
  import { Model, VAD } from '@ai-coustics/livekit-plugin';
  import { ReadableStream } from 'node:stream/web';
  import dotenv from 'dotenv';

  dotenv.config({ path: '.env.local' });
  initializeLogger({ pretty: false, level: 'info' });
  const key = process.env.AIC_SDK_LICENSE?.trim();
  if (!key) throw new Error('Set AIC_SDK_LICENSE in .env.local and retry');

  const model = Model.fromFile('models/vad.aicmodel');
  const vad = new VAD({ model, licenseKey: key });
  const stream = vad.stream();
  const blockSize = model.getOptimalBlockSize(16000);
  let count = 0;
  const deadline = setTimeout(() => stream.close(), 10000);

  try {
    stream.updateInputStream(new ReadableStream<AudioFrame>({
      start(controller) {
        for (let i = 0; i < 100; i++) {
          const frame = new AudioFrame(new Int16Array(blockSize), 16000, 1, blockSize);
          controller.enqueue(vad.processor.process(frame));
        }
        controller.close();
      },
    }));
    for await (const event of stream) {
      if (event.type === VADEventType.INFERENCE_DONE) count++;
      if (count === 100) break;
    }
    if (count !== 100) throw new Error(`Expected 100 inference events; received ${count}`);
    console.log(`AIC VAD diagnostic passed: ${count} inference events`);
  } finally {
    clearTimeout(deadline);
    stream.close();
    await vad.close();
  }
  ```

  ```sh theme={null}
  npx tsx check-vad.ts
  ```

  Expected output: `AIC VAD diagnostic passed: 100 inference events`. If construction fails, check the credential and model first. If inference fails or the deadline expires, preserve the error and check the pinned package versions. Then retry this diagnostic before debugging room dispatch or microphone permissions.
</Accordion>

## Recover from a failure

| Problem | Cause and fix | Retry check |
| - | - | - |
| `ImportError` for `Model` or `Analyzer`, or `VAD(model=...)` fails | The Python environment contains the official plugin or a mixed namespace. Create a fresh environment with the pinned packages above. | Verify imports, then rerun `agent.py dev`. |
| Model file not found | Provisioning did not finish or the worker started from another directory. | Run the download command, then start from the project directory. |
| Missing or rejected key | `.env.local` is missing, the SDK key is absent or its entitlement is invalid. Check the file location and developer platform. | Restart, join a new room and check for continued successful inference. |
| No VAD metadata | `vad.processor` is missing from RoomIO, or audio reaches the VAD through another path. | Restore the shown RoomIO input options and test a room, rather than console mode. |
| No Node.js VAD metrics during a short conversation | The pinned agents version counts speech starts in its inference counter. Missing metrics alone do not establish an inactive VAD. | Run the local diagnostic above, then inspect room routing and inference errors separately. |
| Worker registers but no room starts | The playground dispatches a different agent name or project. | Select `aic-vad-demo` in the same project used by `.env.local`. |
| Inference falls behind real time | The worker lacks CPU capacity for its current load. | Reduce concurrent rooms, retest on target hardware and monitor the warning. |

## Model provisioning

`Model.download` returns a file path. `Model.from_file` / `Model.fromFile` loads that file into memory. In production, provision models during deployment and replace the relative paths with your deployed model paths. Reuse read-only `Model` objects within a worker; use separate VAD and Analyzer instances for every concurrent room.

## Choose models

| Purpose | Model ID for this example |
| - | - |
| Dedicated voice activity detection | `vad-ms-2.1-xxs-16khz` |
| Tyto audio-quality analysis | `tyto-1.1-l-16khz` |

These are different model types. Do not load an enhancement model as a VAD or Analyzer. See the [model reference](/reference/sdk/models) for alternatives and [ai-coustics VAD](/models/voice-activity-detection/vad) for parameters.

## Audio-quality analysis

To add Tyto to this agent, first provision its model:

<CodeGroup>
  ```sh Python theme={null}
  python -c 'from aic_sdk import Model; from shutil import copyfile; path = Model.download("tyto-1.1-l-16khz", "models"); print(copyfile(path, "models/tyto.aicmodel"))'
  ```

  ```sh Node.js theme={null}
  node --input-type=module -e 'import { Model } from "@ai-coustics/livekit-plugin"; import { copyFileSync } from "node:fs"; const path = await Model.download("tyto-1.1-l-16khz", "models"); copyFileSync(path, "models/tyto.aicmodel"); console.log("models/tyto.aicmodel");'
  ```
</CodeGroup>

Add the model to the existing Python `setup` function or Node `prewarm` callback:

<CodeGroup>
  ```python Python theme={null}
      proc.userdata["analysis_model"] = ai_coustics.Model.from_file(
          "models/tyto.aicmodel"
      )
  ```

  ```ts Node.js theme={null}
      proc.userData.analysisModel = Model.fromFile('models/tyto.aicmodel');
  ```
</CodeGroup>

In Node, also add `Analyzer` and `FrameProcessorChain` to the existing `@ai-coustics/livekit-plugin` import. Inside the room entrypoint, after constructing `vad` and before `session.start`, add:

<CodeGroup>
  ```python Python theme={null}
      analyzer = ai_coustics.Analyzer(
          model=ctx.proc.userdata["analysis_model"],
          analysis_interval=5.0,
      )

      @analyzer.on("analysis_result")
      def on_analysis(event):
          logger.info("Tyto risk score: %s", event.result.risk_score)

      frame_processor = ai_coustics.FrameProcessorChain(vad.processor, analyzer.collector)
  ```

  ```ts Node.js theme={null}
      const analyzer = new Analyzer({
        model: ctx.proc.userData.analysisModel as Model,
        analysisInterval: 5,
      });
      analyzer.on('analysisResult', (event) => {
        console.log('Tyto risk score:', event.result.riskScore);
      });
      const frameProcessor = new FrameProcessorChain(vad.processor, analyzer.collector);
  ```
</CodeGroup>

Replace `noise_cancellation=vad.processor` with `noise_cancellation=frame_processor` in Python, or `noiseCancellation: vad.processor` with `noiseCancellation: frameProcessor` in Node. Restart the worker and join a new room. After enough audio has accumulated, `Tyto risk score:` should appear. An Analyzer without its collector in RoomIO receives no audio.

Keep the default 5 s analysis interval. The API accepts positive finite intervals, but shorter intervals, such as 1 s, need CPU measurements at your expected room concurrency. If `analysis falling behind its interval` appears, increase the interval or reduce load. See [Tyto real-time analysis](/models/audio-insight/real-time-analysis) for buffering and result interpretation.

## Authentication

The ai-coustics-maintained components accept `AIC_SDK_LICENSE` by default, or `license_key` / `licenseKey` explicitly as shown for VAD. They do not use LiveKit Cloud authentication or metering for ai-coustics SDK usage. Keep the SDK key in the agent backend. Official-plugin authentication is described in [plugin scope](/reference/livekit/plugin-scope).

## Lifecycle

The examples ask RoomIO for mono audio at 16 kHz. The VAD processor converts incoming PCM to model blocks while preserving the original frame for downstream processing. Models are read-only weights; stream history belongs to each VAD or Analyzer instance. Reconnect with new stateful components.

RoomIO closes its directly configured frame processor or chain when the input stream closes. The chain closes its components. Do not manually terminate a component while RoomIO is still using it. Errors can leave original audio flowing, so monitor VAD inference and Analyzer result events separately from room connectivity.

<Note>
  With `livekit-agents==1.8.2` and `ai-coustics-livekit-plugin==0.2.0`, Python can log `VAD: no inference metadata found` after `input stream detached` during disconnect. The agents framework sends shutdown silence directly to VAD, bypassing the RoomIO frame processor. After a disconnect with `error: null`, verify that a new room resumes positive inference counts. If metadata is missing during live audio, follow the RoomIO recovery instructions above.
</Note>

## Adapt this example

* **Tune VAD after a baseline.** Pass `VADParameters` in Python or `vadParameters` in Node. SDK timing values use seconds; Node `prefixPaddingDuration` and `maxBufferedSpeech` use milliseconds.
* **Keep existing providers.** Copy the per-session VAD and RoomIO wiring into your own entrypoint after checking its framework version and package namespace. Your STT/LLM/TTS providers keep their own authentication requirements.
* **Add analysis without VAD.** Install `analyzer.collector` directly as the RoomIO frame processor and use your existing compatible VAD.
* **Evaluate and operate.** Measure VAD errors, end-of-turn latency and CPU at your expected room concurrency. Follow the [latency guide](/reference/concepts/latency) and [stream lifecycle guide](/reference/concepts/streams-and-state).

<Accordion title="Existing speech enhancement users">
  Plugin 0.2.0 also exports `Processor`. Existing applications can continue using the API in that release. New enhancement integrations should use the official LiveKit plugin. Keep any existing dedicated VAD processor before enhancement so it sees original audio. See [which LiveKit plugin to use](/reference/livekit/plugin-scope) before changing packages or authentication.
</Accordion>

## Next steps

After checking a conversation, [evaluate audio and agent behavior](/guides/evaluate-audio), then [prepare your deployment](/production/deployment). Use [troubleshooting](/production/troubleshooting) to diagnose setup and runtime failures.

<CardGroup cols={2}>
  <Card title="Plugin scope" href="/reference/livekit/plugin-scope">
    Choose the official or ai-coustics-maintained plugin for your task.
  </Card>

  <Card title="ai-coustics VAD" href="/models/voice-activity-detection/vad">
    Configure dedicated voice activity detection.
  </Card>

  <Card title="Tyto" href="/models/audio-insight/tyto">
    Interpret audio-quality results.
  </Card>

  <Card title="Plugin source" href="https://github.com/ai-coustics/livekit-plugins/tree/0.2.0">
    Inspect the released Python and Node.js implementations.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.