Skip to content

Agent Developer Guide

This guide walks you through building and deploying an AI agent that connects to Bayer's LiveKit infrastructure using the Bayer LiveKit SDK and the LiveKit Platform Service (LKPS).

Table of Contents

  1. Overview
  2. Prerequisites
  3. How Agent Registration Works
  4. Artifactory Access
  5. Step 1 — Install the Bayer LiveKit SDK
  6. Step 2 — Configure Environment Variables
  7. Step 3 — Build Your Agent
  8. Step 4 — Run Your Agent
  9. Step 5 — Verify Agent Registration
  10. Agent Registration API Reference
  11. Token Lifecycle
  12. Security Notes
  13. Troubleshooting
  14. Next Steps

Overview

As an agent developer, your workflow is:

  1. Develop your agent using the Bayer LiveKit SDK (Python).
  2. Authenticate with Microsoft Entra ID using your agent's app registration credentials.
  3. Register with LKPS — the SDK handles this automatically and receives a scoped, time-limited LiveKit token.
  4. Connect to the LiveKit server — the SDK connects using the token and waits for session dispatches.
  5. Serve sessions — when a client starts a session targeting your agent, LiveKit dispatches your agent into the room.

Important

You never handle LiveKit API keys or secrets. The SDK calls /api/agent/register; Ocelot verifies agent identity, LKPS issues the agent token, and the SDK registers the agent.

Tip

For detailed SDK documentation including authentication modes, plugin support, migration guides, and advanced configuration, see the Bayer LiveKit SDK Overview.

Prerequisites

Before building your agent, ensure you have completed:

How Agent Registration Works

Agent Registration Flow

The Bayer LiveKit SDK handles all of this automatically. You only write your agent logic — the SDK manages authentication, registration, and token renewal.

Artifactory Access

The Bayer LiveKit SDK is hosted on Bayer's internal JFrog Artifactory. You need read access to the repository, an Identity Token, and properly configured environment variables to install it.

1. Get Read Access to Artifactory

The SDK repository (agenticlabs-pypi-prod-bayerlivekitsdk) is a private repository. You need to complete both of the following:

a) Request general Artifactory read access via one of:

b) Request SDK repository access from the Bayer LiveKit Platform team

General Artifactory read access does not cover private repositories. Contact the Bayer LiveKit Platform team ⧉ to get your account added to the agenticlabs-pypi-prod-bayerlivekitsdk permission target.

Tip

If you can log in to Artifactory and generate a token but pip install returns "no matching distribution found", this means you are missing step (b).

2. Log in to Artifactory

Navigate to artifactory.bayer.com ⧉ and click SAML SSO to log in with your Bayer identity.

3. Generate an Identity Token

  1. Click your username in the top-right corner and select Edit Profile.
  2. Click Generate an Identity Token, give it a meaningful name (e.g., livekit-sdk), and click Next.
  3. Copy the token immediately — it is displayed only once. Store it in a password manager.

4. Set Environment Variables

Export the credentials so the pip install command in the next step can authenticate.

Important

The @ in your email must be URL-encoded as %40 in the username variable, since it is used inside a pip index URL.

export ARTIFACTORY_USER_NAME=jane.doe%40bayer.com      # Replace with your email, using %40 instead of @
export ARTIFACTORY_USER_TOKEN=<your-identity-token>

Tip

For CI/CD pipelines, use a Transient User (Bearer Token) instead of a personal token. See the Artifactory token generation guide ⧉ for details.

Info

For the full Artifactory documentation (JFrog CLI setup, .netrc configuration, repository requests, etc.), see Bayer Cloud Docs — Artifactory ⧉.

Step 1 — Install the Bayer LiveKit SDK

Install the SDK from Bayer's internal Artifactory. Make sure ARTIFACTORY_USER_NAME and ARTIFACTORY_USER_TOKEN are set (see Artifactory Access above).

Note

You must use --extra-index-url (not --index-url) so that pip can also resolve dependencies from PyPI.

pip install bayer-livekit-sdk \
  --extra-index-url "https://${ARTIFACTORY_USER_NAME}:${ARTIFACTORY_USER_TOKEN}@artifactory.bayer.com/artifactory/api/pypi/agenticlabs-pypi-prod-bayerlivekitsdk/simple"

With plugins (e.g., for OpenAI, AWS services):

pip install "bayer-livekit-sdk[openai,aws]" \
  --extra-index-url "https://${ARTIFACTORY_USER_NAME}:${ARTIFACTORY_USER_TOKEN}@artifactory.bayer.com/artifactory/api/pypi/agenticlabs-pypi-prod-bayerlivekitsdk/simple"

See the Bayer LiveKit SDK — Plugin Ecosystem for the full list of available plugin extras.

Step 2 — Configure Environment Variables

Create a .env file or set these environment variables:

# Entra ID credentials (your agent's app registration)
AZURE_CLIENT_ID=<your-agent-client-id>
AZURE_CLIENT_SECRET=<your-agent-client-secret>
AZURE_TENANT_ID=fcb2b37b-5da0-466b-9b83-0014b67a7c78

# LiveKit Platform Service
LKPS_URL=https://eu-livekit-platform-service-np.int.bayer.com

# API audience (used for token acquisition — must match environment)
AZURE_API_AUDIENCE=api://livekit-platform-service-np.int.bayer.com
Variable Required Description
AZURE_CLIENT_ID Yes ¹ Your agent's Entra app registration Client ID
AZURE_CLIENT_SECRET Yes ¹ Your agent's Entra app registration Client Secret
AZURE_TENANT_ID No Bayer Entra tenant ID (defaults to Bayer tenant)
LKPS_URL No LKPS base URL (defaults to EU production)
AZURE_API_AUDIENCE No Token audience — use api://livekit-platform-service-np.int.bayer.com for non-prod, api://livekit-platform-service.int.bayer.com for prod
ENFORCE_CLIENT_AUTHZ No When true (default), LKPS validates client identity before allowing sessions

Note

¹ Required when authenticating as a service principal (CI/CD, production). Omit when using az login (local dev) or Managed Identity (Azure-hosted workloads) — the SDK falls back to DefaultAzureCredential automatically. See Authentication Modes for details.

Tip

Start with the non-production environment for development and testing. See Environments & URLs for all available endpoints.

Step 3 — Build Your Agent

Here is a minimal agent that responds to voice input:

from bayer_livekit_sdk import Worker, CLI
from bayer_livekit_sdk.agents import AgentSession, Agent, function_tool, RunContext
from bayer_livekit_sdk.plugins import openai as openai_plugin

# Define your agent
class MyAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are a helpful assistant for Bayer employees."
        )

    @function_tool
    async def get_greeting(self, context: RunContext) -> str:
        """Return a friendly greeting."""
        return "Hello! I'm your LiveKit-powered assistant."

# Entrypoint called for each session
async def entrypoint(ctx):
    # Wait for a participant to join
    participant = await ctx.wait_for_participant(timeout=20)

    # Create and start a session
    session = AgentSession(
        llm=openai_plugin.LLM(model="gpt-4o"),
    )
    agent = MyAgent()
    await session.start(agent=agent, room=ctx.room)

# Bootstrap
if __name__ == "__main__":
    worker = Worker()
    worker.rtc_session(entrypoint)
    CLI().run_app(worker)

Note

The Worker class handles Entra ID authentication, agent registration, and token renewal internally. You only need to define your agent's behavior.

Tip

For more realistic examples including voice assistants and video agents, see the official LiveKit Python agent examples ⧉.

Step 4 — Run Your Agent

Development Mode

python agent.py dev

Development mode provides verbose logging, which is helpful for debugging authentication and registration issues.

Production Mode

python agent.py start

Console Mode (Local Testing)

python agent.py console

Console mode lets you test your agent locally using your microphone and speakers (or text mode) without needing a client application.

Step 5 — Verify Agent Registration

Once your agent is running, verify it registered successfully:

  1. Check the agent logs for a successful registration message: 

    INFO  Registration successful — livekit_token received, expires_in=3600

  2. Test with a session start request against your target LKPS environment: 

    curl -X POST <lkps-url>/api/session/start \
  -H "Authorization: Bearer <your-client-entra-token>" \
  -H "Content-Type: application/json" \
  -d '{"agent_entra_app_id": "<your-agent-client-id>"}'

    Note

    Replace <lkps-url> with the appropriate environment URL (see Environments & URLs). If client-side integration is complete, use the <your-client-entra-token> your client application already acquires at runtime.

  3. A 200 OK response with room_name, livekit_url, and participant_token confirms the agent is registered and reachable.

  4. A 404 Not Found indicates the agent has not registered or the registration token has expired.

Agent Registration API Reference

Note

The Bayer LiveKit SDK calls this endpoint automatically. This reference is provided for understanding and debugging.

POST /api/agent/register
Content-Type: application/json
Authorization: Bearer <entra_jwt>

Request body (optional):

{
  "service_config": {
    "enforce_client_authz": true
  }
}
Field Type Default Description
service_config.enforce_client_authz boolean true When true, LKPS validates that client applications are pre-authorized before allowing sessions.

Response 200 OK:

{
  "livekit_token": "eyJhbGciOiJIUzI1NiJ9...",
  "livekit_url": "wss://eu.livekit-np.int.bayer.com",
  "expires_in": 3600
}
Field Description
livekit_token Scoped LiveKit JWT with agent grants (agent, canPublish, canSubscribe, canPublishData)
livekit_url LiveKit server WebSocket URL
expires_in Token TTL in seconds (default: 3600)

Identity resolution:

The agent's identity is derived from Ocelot-injected headers (not from the request body):

Header Used as
client-id (from Entra appid claim) Agent app ID (app_id)

Token Lifecycle

Phase What happens
Initial registration SDK authenticates with Entra ID, calls /api/agent/register, receives a LiveKit token valid for 1 hour (default).
Active service Agent is connected to the LiveKit server and serving sessions.
Token renewal SDK automatically re-registers 5 minutes before token expiry. A new token is obtained and applied seamlessly — no downtime.
Failure handling If renewal fails, the SDK retries. If the Entra app is disabled or removed, the agent can no longer register.

Security Notes

  • No static LiveKit credentials — your agent never holds LIVEKIT_API_KEY or LIVEKIT_API_SECRET.
  • ctx.api is blocked — the SDK's JobContext intentionally prevents direct LiveKit REST API access to enforce security boundaries.
  • Scoped tokens — each registration token grants only agent-level permissions (join rooms, publish/subscribe). No room management or admin capabilities.
  • Audit trail — every registration is persisted in the LKPS database with your agent's app ID, email, timestamp, and worker identity.
  • Instant revocation — disabling your Entra app registration immediately prevents further registrations. No secret rotation required.

Troubleshooting

Authentication Errors

Error Possible Cause Resolution
AADSTS70011: Invalid scope Incorrect API audience Verify AZURE_API_AUDIENCE matches your environment — api://livekit-platform-service-np.int.bayer.com for non-prod, api://livekit-platform-service.int.bayer.com for prod
AADSTS700016: Application not found Wrong client ID or tenant Verify AZURE_CLIENT_ID and AZURE_TENANT_ID
AADSTS7000215: Invalid client secret Expired or incorrect secret Generate a new client secret in the Azure Portal

Registration Errors

Error Possible Cause Resolution
401 Unauthorized Entra token not accepted by Ocelot Verify your client ID is in the LKPS Ocelot route's Client Access List
403 Forbidden (from Ocelot) Client ID not in Ocelot access list Request access to the LKPS Ocelot route (self-service)

Connection Errors

Error Possible Cause Resolution
Agent registers but no sessions arrive Agent name mismatch Ensure clients use your agent's Entra App ID as agent_entra_app_id
Token expired / disconnected Renewal loop not running Check logs for renewal errors; verify Entra credentials are still valid

Tip

For additional troubleshooting scenarios, see the Bayer LiveKit SDK Overview.

Next Steps

Next Guide
Detailed SDK documentation Bayer LiveKit SDK Overview
SDK architecture and flows Bayer LiveKit SDK — System Architecture
Full LKPS API reference LiveKit Platform Service — API Reference
Configure client authorization Client-to-Agent Authorization
View all environment URLs Environments & URLs