ElevenLabs Agent

Bring your own ElevenLabs agent and give it a Beyond Presence avatar. ElevenLabs runs the conversation with the agent’s own prompt, voice, knowledge base, tools, and turn-taking. Beyond Presence renders the avatar and streams the video call.

For now, an ElevenLabs agent can only be set up through the API, not in the dashboard.

When to use this

Use an ElevenLabs agent when:

You already built and tuned an agent in ElevenLabs and want to add a video avatar to it
You want to keep managing the prompt, voice, knowledge base, and tools in ElevenLabs

To use your own Chat Completions endpoint instead, see External LLM.

Prerequisites

An ElevenLabs account API key
The ID of an agent in that ElevenLabs account
A Beyond Presence API key

Requirements for the ElevenLabs agent

Configure your agent in ElevenLabs as follows:

PCM audio: Set both the user input audio format and the agent output audio format to a PCM format, for example pcm_16000.
agent_response_complete client event: Enable this client event so Beyond Presence knows when the agent has finished a reply.
Server-side tools only: Client tools are not supported. When the ElevenLabs agent calls a client tool, Beyond Presence returns an error result saying the tool is not available. Run tools on the ElevenLabs side, for example as server tools or webhooks.

Configuration

Create an External API configuration of type elevenlabs with your ElevenLabs API key using the Create External API Configuration endpoint. The response contains the configuration id.

Create the agent

Create an agent with the Create Agent endpoint and set llm to type elevenlabs_agent:

api_id: ID of the elevenlabs External API configuration from the previous step
agent_id: ID of the agent in ElevenLabs

Python

import requests

API_URL = "https://api.bey.dev/v1"
HEADERS = {"x-api-key": "sk-your-api-key-here"}

# 1. Register the ElevenLabs account that owns the agent
response = requests.post(
    f"{API_URL}/external-apis",
    headers=HEADERS,
    json={
        "type": "elevenlabs",
        "name": "My ElevenLabs account",
        "api_key": "your-elevenlabs-api-key",
    },
)
response.raise_for_status()
elevenlabs_api_id = response.json()["id"]

# 2. Create a Beyond Presence agent that runs the ElevenLabs agent
response = requests.post(
    f"{API_URL}/agents",
    headers=HEADERS,
    json={
        "name": "My ElevenLabs avatar",
        "avatar_id": "your-avatar-id",
        "system_prompt": "Configured in ElevenLabs.",
        "llm": {
            "type": "elevenlabs_agent",
            "api_id": elevenlabs_api_id,
            "agent_id": "your-elevenlabs-agent-id",
        },
    },
)
response.raise_for_status()
agent = response.json()

system_prompt is required by the Create Agent endpoint, but the ElevenLabs agent uses its own prompt.

Unsupported settings

ElevenLabs owns speech recognition, the voice, the greeting, and every reply. The API rejects creating or updating an elevenlabs_agent agent that sets any of the following:

Speech-to-text (STT) configuration
tts
greeting
User-away ping
The webcam_vision capability
The wakeup_mode capability
Dynamic avatar emotions

A static avatar emotion still works. Configure the other settings in the ElevenLabs agent instead. You also can’t attach knowledge files with knowledge_file_ids. Add your content to the ElevenLabs agent’s knowledge base instead.

How the call ends

The call ends when the ElevenLabs conversation ends.

The agent hangs up: Beyond Presence lets the farewell finish playing, for up to 30 seconds, before ending the call. The call UI shows that the agent ended the call.
The conversation drops: Beyond Presence ends the call right away. The call UI shows it as an error.

Get Started

Core Concepts

Integrations

Production Guides

Learn More

When to use this

Prerequisites

Requirements for the ElevenLabs agent

Configuration

Unsupported settings

How the call ends

​When to use this

​Prerequisites

​Requirements for the ElevenLabs agent

​Configuration

​Unsupported settings

​How the call ends

When to use this

Prerequisites

Requirements for the ElevenLabs agent

Configuration

Unsupported settings

How the call ends