REST API
Everything the dashboard does, from your server. JSON over HTTPS. Base URL https://app.avaclone.ai/api/v1.
Authentication
Create a secret key under Settings. Send it as a bearer token. Keys are shown once, stored hashed, and can be revoked at any time. They belong on servers only; browsers use an embed’s publishable id instead.
curl https://app.avaclone.ai/api/v1/organization \
-H "Authorization: Bearer ak_live_…"Errors are JSON with a stable code and a human message:
{ "code": "tokens.exhausted", "message": "No tokens left this period." }Status codes: 400 invalid body, 401 no or bad key, 403 the key or role may not do this, 404 not yours or gone, 409 state (an unpublished agent, an avatar still building), 429 a plan cap, 402 a plan limit such as seats.
Organization
GET /organization
Plan, limits and the live token balance (1 token = $0.01). tokens_included is the plan’s number each period; tokens_granted is what this period actually holds, the plan plus any top-ups and grants, so tokens_left = tokens_granted − tokens_used. tokens_available adds the extra tokens still allowed under the spending cap and is the figure every gate checks.
{ "id": "org_…", "name": "Harbor Dental", "plan": "pro", "plan_label": "Pro",
"token_usd": 0.01, "tokens_included": 22500, "tokens_granted": 23500, "tokens_left": 16369, "tokens_used": 7131, "tokens_available": 56169, "minutes_left": 545.6,
"live_at_once": 3, "avatars_max": 3, "embed_domains_max": 5, "premium_multiplier": 2,
"live_tokens_per_minute": { "lite": 30, "premium": 60 },
"studio_tokens": { "openai": { "standard": 4, "high": 13, "max": 50 }, "google": { "standard": 40, "high": 40, "max": 72 }, "google-nb21": { "standard": 10, "high": 15, "max": 34 } },
"extra_token_usd": 0.01, "spending_cap_usd": 398, "extra_tokens_allowed": 39800, "extra_tokens_used": 0, "extra_cost_usd": 0,
"period_start": "2026-10-01T00:00:00.000Z", "billing_interval": "month", "paid_until": null, "transcript_days": 30 }PATCH /organization
Owners and admins set the spending cap for extra tokens, { "spending_cap_usd": 200 } (0 stops at the allowance), and how long conversations are kept, { "transcript_days": 90 } (1 to 365, default 30). Either field alone.
Avatars
POST /avatars
Build an avatar from a photo. multipart/form-data:
| Field | Type | Meaning |
|---|---|---|
| photo | file | A front-facing image (JPEG, PNG or WebP, up to 20 MB). Or pass image_id for a Studio image instead. |
| image_id | string | A Studio image id to build from. |
| name | string | Display name, up to 80 characters. |
| real_person | yes | no | Whether the photo shows a real person (default yes). It sets the badge: a generated face is labeled "AI avatar, not a real person". |
| voice_id | string | The avatar's own voice, from GET /voices. Default: a stock voice. |
| framing | head_and_shoulders | portrait | head_and_shoulders (default): the whole person, as you framed the photo. portrait: cropped to the face, centred and sized for the model. |
By building an avatar you confirm you have the right to use the photo and the person’s likeness, as the terms set out; nothing is signed per avatar. If you keep your own releases you may send person_name, person_email, is_adult=yes, uses and signed_name and the release is kept on file.
Every photo is checked first (a few seconds, free). A photo with no face or more than one person is refused with 422 photo.no_face or photo.faces, and nothing is stored. The photo is built as you framed it and never cropped by itself, so a full figure or a wide composition stays as it is. When the face would come out small that way, photo_check.crop_suggested is true (the dashboard shows “Your avatar might look better with a crop”): send framing=portrait, or rebuild with it later, to crop to the face, free.
Returns 202 with the avatar in status building. Poll GET /avatars/:id until ready (about a minute on a warm engine) or failed with an error.
When the expression would animate badly (teeth showing, a big smile, an open mouth, eyes looking away or shut) the status is needs_approval instead, photo_check.findings says why, and approval carries a link and a price: { "url": "https://app.avaclone.ai/avatars/ava_…?approve=1", "tokens": 40 }. Send a person to that link: they sign in to AvaClone and approve the tokens, the photo is regenerated with a calm, closed-mouth smile and nothing else changed, checked again, and the build starts (status fixing, then building). The fix is charged only when it passes; if it does not, the avatar goes back to needs_approval with photo_check.fix_error and nothing is charged. An API key cannot approve.
{ "id": "ava_…", "status": "needs_approval",
"photo_check": { "status": "fix", "findings": ["Teeth are showing. The avatar draws teeth poorly; they can blur into the gums."],
"expression": "big_smile", "teeth_visible": true, "mouth_open": true, "gaze": "camera", "eyes": "open",
"crop_suggested": false, "fixed": false, "fix_error": null },
"approval": { "url": "https://app.avaclone.ai/avatars/ava_…?approve=1", "tokens": 40 }, … }GET /avatars
Your avatars with status, engine (lite or premium) and the idle loop URL once ready.
GET /avatars/:id
One avatar. framing is head_and_shoulders (the whole person, as framed) or portrait (cropped to the face, centred and sized for the model); photo_check.crop_suggested says when the crop would likely look better.
PATCH /avatars/:id
Any of name, voice_id and face_restore. { "face_restore": true } turns on Face Restore (beta): sharper eyes, lips and teeth on every frame of this avatar’s live sessions and videos, for restore_tokens_per_minute more tokens a minute (5 today, on either tier; the number is on GET /organization). It applies to sessions and videos made after the change, and is charged only when it ran: when the GPU is busy a session can run without it. The idle loop (idle_loop_url, what a page shows before the stream is live) is always made with Face Restore, free, whatever this says; idle_restored tells you it was.
POST /avatars/:id/rebuild
Rebuild from the stored photo, free, with everything the engine now does. The avatar keeps its framing unless you send { "framing": "portrait" } (crop to the face) or { "framing": "head_and_shoulders" } (back to the whole person). 409 while a build runs or the avatar waits for its expression fix.
DELETE /avatars/:id
Remove it. Agents using it stop starting sessions until they are given another.
Agents
POST /agents
{ "name": "Front desk", "avatar_id": "ava_…" }. Creates a draft with sensible defaults (a stock voice, mixed grounding, vision on, 40 turns an hour, 400 a day, a two-minute quiet cut-off).
GET /agents
All agents with their draft and publish state.
GET /agents/:id
One agent: the draft configuration flattened, status (draft or published), published_version, draft_changed.
PATCH /agents/:id
Change the draft. Any subset:
{ "name": "Front desk", "display_name": "Lindsey",
"avatar_id": "ava_…", "voice_id": "stock_warm_f",
"charter": {
"purpose": "Answer questions about the practice and book appointments by phone.",
"audience": "Patients and people choosing a dentist.",
"tone": "Warm, brief, plain.",
"greeting": "Hi, I'm the Harbor Dental assistant. What can I help with?",
"return_greeting": "Welcome back! Where were we?",
"topics": ["hours", "insurance", "services"],
"boundaries": ["No medical advice; suggest a visit."],
"unknown_handling": "Say you are not sure and offer the phone number.",
"facts": ["Open Monday to Friday 8 to 5."],
"sample_qa": [{ "q": "Do you take Delta Dental?", "a": "Yes, we are in network." }]
},
"grounding": "knowledge_only",
"vision": true, "expressive_voice": true, "voice_cues": ["sympathetic", "thoughtful", "short pause"], "language": "en",
"limits": { "turns_per_hour": 40, "turns_per_day": 400, "quiet_cutoff_s": 120 } }grounding: knowledge_only answers only from your files and facts; mix adds general knowledge; model_only ignores files.
expressive_voice (Pro plan and above): the voice performs a cue such as a chuckle or a sympathetic tone when the moment in the conversation calls for one, always in keeping with charter.tone; a serious or gruff tone keeps cues out of nearly every answer and never laughs. It is on for every agent on Pro and above (and switches on for all of an organization’s agents when it moves up to Pro); turning it on below Pro returns 403 plan.required. voice_cues lists the cues the agent may use (null, the default, allows any that suits its tone); the voice drops any other, even if the model writes it. The cues are listed under Your own model, our face and voice. Captions and transcripts never show them.
language: the language visitors usually speak, as an ISO 639-1 code (en by default; also es, fr, pt, de, it, zh, vi, tl, ko, ja, hi, ar, ru, uk, pl, nl, tr, he). It guides rather than limits: speech recognition expects it and the agent answers in it, but a visitor who plainly speaks another language is understood and answered in theirs. One short line heard in another alphabet, such as a quick “yeah” or a TV mistaken for Korean, is ignored; a second line in a row in that alphabet is taken as the visitor’s language.
charter.return_greeting: what the agent says instead of the greeting when a visitor picks an earlier conversation back up, after the quiet cut-off ended it, after they closed the panel, or after a reload in the same tab. The agent remembers the earlier conversation (its last eight exchanges) either way. Blank: “Welcome back! What else can I help you with?” for an English agent, no scripted line for another language.
display_name: the name visitors see, above the face in the widget and as the hosted page’s title, and the name the agent gives when it introduces itself or is asked. name stays your own label for the agent (“Front desk”). null or blank, the default, uses name. Like everything else, visitors get it with the next publish. Use the same name in charter.greeting and the persona.
DELETE /agents/:id
Remove the agent and its embed.
POST /agents/:id/publish
Freeze the draft as the next version. Visitors get it at once; live sessions finish on the version they started with.
GET /agents/:id/versions
Every published version with its configuration.
POST /agents/:id/versions
{ "version": 3 } rolls back: that version becomes the published one and the draft.
Knowledge
POST /agents/:id/knowledge
multipart/form-data with file (PDF, text or Markdown, up to 10 MB). The file is chunked and embedded in the background; GET shows its status.
GET /agents/:id/knowledge
Files with status (indexing, ready, failed) and chunk counts.
DELETE /agents/:id/knowledge?file=kf_…
Remove a file and its chunks.
Embeds
GET /agents/:id/embed
The publishable id, allowed domains, appearance and ready-to-paste snippets:
{ "agent_id": "agt_…", "publishable_id": "pub_…",
"allowed_domains": ["harbordental.com", "www.harbordental.com"],
"appearance": { "position": "bottom_right", "launcher_size": "medium", "launcher_label": "Talk to us", "theme": "auto", "accent": "#0052FF", "captions": true, "background": "avatar" },
"snippets": { "script": "<script src=…>", "inline": "…", "iframe": "<iframe …>", "hosted_url": "https://app.avaclone.ai/a/pub_…" } }PATCH /agents/:id/embed
{ "allowed_domains": ["…"], "appearance": { … } }. Domain count is per plan.
POST /agents/:id/embed/rotate
A new publishable id; the old one stops working at once.
Sessions
POST /sessions
Start a conversation for your own UI (the SDK). Checks tokens and the live cap first.
| Field | Type | Meaning |
|---|---|---|
| kind | agent | stream | preview | agent: the published agent answers. preview: the draft (for your own testing). stream: no agent; your own model drives the conversation and you send either text (we voice it) or speech audio (the avatar lip-syncs). |
| agent_id | string | For agent and preview. |
| avatar_id | string | For stream. |
| voice_id | string | For stream: the voice that speaks the text you send, from GET /voices (a stock voice or one you trained). Default: the avatar's own voice. |
| resume_session_id | string | For agent and preview: an earlier session with the same agent, from the last 24 hours. The new session picks its conversation up: the agent remembers it and says its return greeting. Any other id is ignored and the conversation starts fresh. |
| timezone | string | The visitor's IANA time zone, e.g. America/New_York (in the browser: Intl.DateTimeFormat().resolvedOptions().timeZone). The agent can then tell them the time, day and date. Without it the agent knows the time in UTC and says so. |
{ "id": "ses_…", "ws_url": "wss://rt.avaclone.ai/v1/sessions/ses_…", "token": "…",
"expires_at": "2026-10-07T21:10:00.000Z", "idle_loop_url": "https://app.avaclone.ai/m/ses_…/idle", "packed": true,
"engine": "lite", "resumed_from": null }Give ws_url, token and idle_loop_url to the browser. The token is single-use and expires in ten minutes.
Your own model, our face and voice
To keep your own LLM, knowledge and conversation logic and use AvaClone only for the avatar and the voice, start a stream session with an avatar_id and, if you like, a voice_id. Then, each time your model has something to say, send the text and the avatar says it: the words are read aloud in that voice and the face is lip-synced live. Nothing else is needed; there is no agent, no listening and no knowledge on our side. Send a line per sentence or paragraph as your model produces it; lines are spoken in order, and interrupt() cuts the current one and drops the rest. If you already generate speech yourself, send 24 kHz PCM instead and the face follows it.
curl https://app.avaclone.ai/api/v1/sessions \
-H "Authorization: Bearer ak_live_…" -H "Content-Type: application/json" \
-d '{ "kind": "stream", "avatar_id": "ava_…", "voice_id": "vce_…" }'// in the browser, with the ws_url and token from your server
import { AvaClone } from "https://cdn.avaclone.ai/v1/sdk.js";
const avatar = AvaClone.mount(document.querySelector("#face"), { wsUrl, token, idleLoopUrl });
avatar.sendText("Thanks for calling. We're open until 6 today, and parking is free after 5."); // from your own modelBefore anything is spoken, symbols are read the way a person would say them: $20.10 is “twenty dollars and ten cents”, 72°F is “seventy-two degrees Fahrenheit”, 5–10 is “five to ten”, markdown and emoji are dropped. The transcript event carries your text as you sent it, without audio tags.
Audio tags. A cue in square brackets is performed by the voice rather than read: avatar.sendText("[chuckles] Fair enough. We open at nine."). The cues are [laughs] [chuckles] [sighs] [exhales] [clears throat] [short pause] [long pause] [warmly] [softly] [happy] [excited] [curious] [thoughtful] [surprised] [sympathetic] [reassuring]; any other bracketed text is dropped, never read aloud. Put the cue just before the words it colours. A cue counts toward the line’s length but never shows in captions or transcripts. Cues are a Pro feature: below the Pro plan they are dropped, never read aloud. Hosted agents use the same cues on their own when expressive voice is on.
GET /sessions/:id
Status (created, live, ended), seconds live, tokens charged, turns, how it ended.
POST /sessions/from-embed
What the widget calls; no key, checked against the embed’s domains by the request’s Origin. { "publishable_id": "pub_…", "visitor_ref": "optional id" }. The widget also sends "resume": { "session_id", "token" }, the tab’s previous session, so the agent picks the conversation up, and timezone, the browser’s, so the agent knows the visitor’s date and time. Same response as above plus the greeting, the appearance and whether the branding line shows.
Studio images
POST /images/generate
{ "prompt": "…", "quality": "standard" | "high" | "max", "count": 1, "model": "google" | "google-nb21" | "openai" }. Draws 4 to 72 tokens per image by model and quality (see studio_tokens on the organization). Returns a job; poll GET /images/jobs/:id for image_ids.
POST /images/:id/edit
{ "prompt": "make the background a bright office", "quality": "high" }. Edit with words; lineage is kept.
POST /images/:id/style
{ "style": "cartoon", "quality": "high" }. Style ids from GET /styles.
POST /images/:id/background
{ "background": "remove" } for a transparent PNG, or { "background": "replace", "background_value": "a bright office" }. Priced like an edit, by model and quality.
GET /images?library=1
Your images; library=1 only the ones saved to the library. Each image carries prompt, the words that made it: the generate prompt, or the instruction for an edit (null for uploads, styles and backgrounds). Follow parent_id to the generated image to find the original prompt of an edited one.
PATCH /images/:id
{ "saved": true, "name": "…" }.
GET /images/:id/download
The file as a download.
DELETE /images/:id
Remove it.
GET /styles
The style catalog (id, name, description, version), the tokens per model and quality, and the models available.
Conversations
What visitors asked and what your agents answered, one line per turn, for every agent and preview session. Lines are kept for the organization’s transcript_days (default 30, set on PATCH /organization or in Settings) and then deleted. Visitors are never identified; visitor_ref is only what you passed yourself. Stream sessions are not recorded: that text is yours.
GET /conversations
Newest first. ?agent_id= narrows to one agent; ?limit= up to 100; pass the response’s next as ?before= for the next page.
curl "https://app.avaclone.ai/api/v1/conversations?agent_id=agt_…&limit=20" \
-H "Authorization: Bearer ak_live_…"{ "items": [ { "id": "ses_…", "agent_id": "agt_…", "agent_name": "Front desk", "kind": "agent",
"started_at": "2026-10-12T18:04:11.000Z", "ended_at": "2026-10-12T18:06:40.000Z", "live_seconds": 149, "turns": 4, "lines": 9,
"first_question": "What are your opening hours?", "last_line": "See you Saturday, then.", "end_reason": "quiet", "origin": "harbordental.com", "visitor_ref": null } ],
"next": "2026-10-12T18:04:11.000Z", "transcript_days": 30 }GET /conversations/:id
The conversation with messages: role (user or assistant), text, via (how the visitor said it: voice or text), at.
import os, requests
r = requests.get("https://app.avaclone.ai/api/v1/conversations/ses_…",
headers={"Authorization": f"Bearer {os.environ['AVACLONE_API_KEY']}"})
for line in r.json()["messages"]:
print(line["role"], ":", line["text"])Voices
Every avatar and agent speaks with a voice from your library: the stock voices every account shares, and the voices you train from audio of one speaker. Training takes under a minute and costs no tokens; Pro keeps one trained voice, Business three, Enterprise as agreed. Use a voice’s id as voice_id on POST /avatars, PATCH /avatars/:id and PATCH /agents/:id. The person whose voice you train must have agreed to it.
GET /voices
Your library. Each voice carries kind (stock or cloned), sample_url (the voice saying one line, MP3; fetch it with the same key) and used_by (the avatars and agents that speak with it). ?kind=cloned returns only the voices you trained. voices_max and voices_used are your plan’s allowance.
curl "https://app.avaclone.ai/api/v1/voices?kind=cloned" \
-H "Authorization: Bearer ak_live_…"{ "items": [ { "id": "vce_7f3a…", "name": "Maya", "description": "Your upload", "gender": null, "kind": "cloned",
"sample_url": "https://app.avaclone.ai/api/v1/voices/vce_7f3a…/sample", "created_at": "2026-10-12T18:04:11.000Z",
"used_by": { "avatars": [ { "id": "ava_…", "name": "Maya" } ], "agents": [] } } ],
"voices_max": 3, "voices_used": 1 }POST /voices
Train a voice from audio. multipart/form-data:
| Field | Type | Meaning |
|---|---|---|
| name | string | What to call the voice, up to 80 characters. |
| files | file (one or more) | mp3, wav, m4a, webm, ogg or flac; 25 MB each, 50 MB in all. Thirty seconds to three minutes of one speaker, little background noise. A single `file` field works too. |
| recorded | yes | no | Whether the audio was recorded for this purpose (default no). Shown as the voice's description. |
Returns 201 with the voice, ready to use as voice_id at once. Errors: 400 audio.short under thirty seconds, 413 audio.size, 415 audio.type, 402 plan.voices on a plan without training, 409 voices.limit when the plan’s trained voices are all used (delete one first), 502 voice.provider if the provider refuses.
curl https://app.avaclone.ai/api/v1/voices \
-H "Authorization: Bearer ak_live_…" \
-F "name=Maya" \
-F "files=@maya-sample.mp3"// Node 20+ (fetch, FormData and Blob are built in)
import { readFile } from "node:fs/promises";
const form = new FormData();
form.append("name", "Maya");
form.append("files", new Blob([await readFile("maya-sample.mp3")], { type: "audio/mpeg" }), "maya-sample.mp3");
const res = await fetch("https://app.avaclone.ai/api/v1/voices", {
method: "POST",
headers: { Authorization: `Bearer ${process.env.AVACLONE_API_KEY}` },
body: form,
});
const voice = await res.json(); // { id: "vce_…", name: "Maya", kind: "cloned", sample_url: "…" }import os, requests
with open("maya-sample.mp3", "rb") as f:
res = requests.post(
"https://app.avaclone.ai/api/v1/voices",
headers={"Authorization": f"Bearer {os.environ['AVACLONE_API_KEY']}"},
data={"name": "Maya"},
files=[("files", ("maya-sample.mp3", f, "audio/mpeg"))],
)
voice = res.json() # voice["id"] is the voice_id to use on an avatar or agentGET /voices/:id
One voice, with used_by.
PATCH /voices/:id
{ "name": "…" } on a voice you trained; stock voices keep their names.
DELETE /voices/:id
Remove a trained voice. It is deleted at the provider; avatars and agent drafts that used it fall back to Harper.
GET /voices/:id/sample
The voice saying one line, as MP3.
curl https://app.avaclone.ai/api/v1/voices/vce_7f3a…/sample \
-H "Authorization: Bearer ak_live_…" -o maya.mp3Videos
An avatar says a script in its voice, lip-synced, as an MP4 you keep in your video library. Frames: 1:1 (1080×1080), 16:9 (1920×1080), 9:16 (1080×1920), 3:4 (1080×1440). Every video renders on our sharpest model (Pro). Price: 60 tokens a minute of finished video (video_tokens_per_minute on GET /organization), plus Face Restore’s 5 when the avatar has it on, by the second, charged when the video is ready; nothing on failure. Renders run one at a time, so a video can wait in queued while another renders.
POST /videos
| Field | Type | Meaning |
|---|---|---|
| avatar_id | string | A ready avatar. |
| script | string | What the avatar says, word for word, up to 3,000 characters (about 3.5 minutes). Money, units and symbols are read naturally; audio tags such as [chuckles] or [warmly] are performed on the Pro plan and above, dropped below it, never read aloud (the list is under Your own model, our face and voice). |
| ratio | 1:1 | 16:9 | 9:16 | 3:4 | The frame. Default 16:9. |
| name | string | Optional; the first words of the script otherwise. |
| voice_id | string | Only read when the avatar has no voice yet; it becomes the avatar's voice. |
| background | object | { mode, value?, scene?, prompt?, blur? }. auto (default): the photo when it covers the frame, else blurred sides. blur is free. extend has the image model continue the photo's scene (a Studio edit, paid once per avatar and frame, reused after). colour takes value, a hex colour. scene takes scene: office, studio-gray, studio-gradient, bookshelf, reception or outdoors, or custom with prompt, your own description of the setting (3 to 300 characters; people, animals, text and logos are left out). A scene is paid once per scene and frame; the same words reuse it. blur sets how soft the scene sits behind the avatar: blurred (default), light or none. The scene is painted sharp and blurred per video, so changing blur later is free. Avatars without a photo background take colour or scene. |
Returns 202 with the video in status queued. Poll GET /videos/:id until ready (about as long as the video, plus a minute if the engine was cold) or failed with an error. GET /avatars/:id/framing says beforehand which frames the photo covers, what a painted background costs and which are already paid for.
curl https://app.avaclone.ai/api/v1/videos \
-H "Authorization: Bearer ak_live_…" -H "Content-Type: application/json" \
-d '{ "avatar_id": "ava_…", "ratio": "16:9", "script": "Hi, I'"'"'m Harper from Harbor Dental. Here is what to expect at your first visit." }'const res = await fetch("https://app.avaclone.ai/api/v1/videos", {
method: "POST",
headers: { Authorization: `Bearer ${process.env.AVACLONE_API_KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({ avatar_id: "ava_…", ratio: "9:16", script: "Hi, I'm Harper…", background: { mode: "extend" } }),
});
let video = await res.json(); // { id: "vid_…", status: "queued", tokens_estimated: 24, … }
while (video.status !== "ready" && video.status !== "failed") {
await new Promise((r) => setTimeout(r, 5000));
video = await (await fetch(`https://app.avaclone.ai/api/v1/videos/${video.id}`, { headers: { Authorization: `Bearer ${process.env.AVACLONE_API_KEY}` } })).json();
}
console.log(video.url, video.download_url); // valid for an hourimport os, time, requests
h = {"Authorization": f"Bearer {os.environ['AVACLONE_API_KEY']}"}
video = requests.post("https://app.avaclone.ai/api/v1/videos", headers=h,
json={"avatar_id": "ava_…", "ratio": "1:1", "script": "Hi, I'm Harper…"}).json()
while video["status"] not in ("ready", "failed"):
time.sleep(5)
video = requests.get(f"https://app.avaclone.ai/api/v1/videos/{video['id']}", headers=h).json()
open("harper.mp4", "wb").write(requests.get(video["download_url"]).content)GET /videos
The library, newest first, without playback URLs. ?avatar_id= narrows it to one avatar.
GET /videos/:id
One video. Once ready, url plays and seeks straight from storage and download_url saves it as a file; both are valid for an hour, so fetch the video again for fresh ones.
PATCH /videos/:id
{ "name": "…" }.
DELETE /videos/:id
Remove it from the library; the file is deleted at once. A video still rendering waits until it finishes.
GET /videos/:id/download
A 302 to a short-lived storage URL that saves under the video’s name.
Keys and team
POST /api-keys
{ "name": "production server" }; the secret is in the response once.
GET /api-keys
Names, prefixes, last use.
DELETE /api-keys/:id
Revoke.
GET /team
Members, open invites and seats (signed-in people only).
POST /team
{ "email": "…", "role": "member" | "admin" }; returns the one-time invite link.
Vision: showing the agent things
An agent with vision on can see what the visitor shows it, over the API as well as in the widget. Two ways, both on an agent session:
- Send a snapshot. Your client sends an image whenever it likes: a webcam frame, a photo the visitor picked, a canvas. Without text it is what the visitor sees now, and the next thing they say or type is answered while seeing it (a newer snapshot replaces it; it is forgotten after a minute). With text it is a turn of its own. Send one when the visitor presses a button, when they pick a photo, or on a timer of your choosing.
- Let the agent ask. Hand the SDK a
CameraCapture. The browser keeps the last three seconds of frames in memory and sends one still only when the visitor refers to something in view; the relay asks, the SDK answers. Nothing goes up on a timer.
// in the browser, on an agent session mounted with the SDK
const cam = new CameraCapture();
await cam.start("environment"); // the back camera on a phone
await avatar.sendImage(cam, "What is this part?"); // a turn with the sharpest recent frame
await avatar.sendImage(fileInput.files[0]); // a photo: the visitor's next question is answered seeing it
avatar.on("image", ({ ok, reason }) => { if (!ok) console.warn("not seen:", reason); });Rules that hold for every image: at most ten a minute per session, 1,536 px on the long side (the SDK resizes), JPEG, PNG or WebP under 3 MB on the wire, nothing stored anywhere (the image goes to the model for that one answer and is dropped), and the agent never describes or identifies people. On an agent session looking costs nothing beyond the session’s minutes.
Looking for your own model (stream sessions)
On a stream session your model is the brain, so there is no agent to answer about an image. Instead the same image message is a look: the relay describes what is in view (what it is, likely make and model, every piece of text read exactly, how sure it is; never people) and hands the description back as text for your model to read, in { type: "image", ok: true, id, description, tokens }. Send text with it (what the person just said) so the description dwells on what matters, and an id of your own to match the answer. A look draws look_tokens (on GET /organization; one token today) from your balance and is refused with tokens.exhausted when none are left, or look.failed when the vision model did not answer (nothing is charged then). The same ten-a-minute budget and size rules apply.
// in the browser, on a stream session mounted with the SDK
const seen = await avatar.look(cam, "What is this part?"); // or a photo, a canvas, a data URL
if (seen.ok) myModel.ask(`[The person is showing you: ${seen.description}] What is this part?`);
else console.warn("not seen:", seen.reason); // image.rate, image.large, image.invalid, tokens.exhausted, look.failedRealtime protocol
Over the session WebSocket, text frames are JSON messages and binary frames carry audio and video. Up, from your client:
| Field | Type | Meaning |
|---|---|---|
| text | { text } | An agent session: the visitor's typed turn. A stream session: a line for the avatar to say. |
| image | { data, text?, id? } | A snapshot as a data URL (data:image/jpeg;base64,…, also png or webp, under 3 MB). An agent session with vision on: without text it rides on the next turn; with text it is a turn. A stream session: a look, answered with a description for your own model, for look_tokens. At most ten a minute. Answered with image; id is echoed. |
| camera | { on } | The visitor's camera is on (or off): the relay may send look when the words refer to something in view. |
| still | { id, jpeg | null } | The answer to look: the sharpest recent frame as a data URL, or null when there is none. |
| interrupt | Barge-in: stop the answer within a frame and drop what was queued. | |
| end | A stream session: the utterance is complete, flush the last partial chunk. | |
| rest | End the session on purpose. | |
| binary | 24 kHz s16le mono PCM | An agent session: the visitor's microphone. A stream session: your own speech audio for the face to follow. |
Down, from the relay:
| Field | Type | Meaning |
|---|---|---|
| session | { id, kind, agent } | The session was accepted; agent is the name to show for it (its display_name, else its name). |
| status | { stage, waited_s } | Waiting for a GPU: starting (a cold pool) or busy (every engine taken), and for how long. |
| ready | { avatar, packed, fps, chunk_s, in_sr, model } | The stream is up. |
| state | { speaking } | The avatar started or stopped speaking. |
| transcript | { role, text, final, via } | What the visitor said (user; via voice or text) and what the agent is saying (assistant; final false while it streams). |
| image | { ok, id?, reason?, description?, tokens? } | Your image was taken, or refused: vision.off, image.rate, image.large, image.invalid, tokens.exhausted, look.failed. A look on a stream session carries the description and the tokens it drew. |
| look | { id } | The relay wants one still of what the visitor is showing; answer with still. |
| flush | Drop queued frames and audio (after an interrupt). | |
| binary | chunk | u32 little-endian header length, a JSON header, then JPEG frames and PCM for about a second of the face. |
Close codes: 4001 quiet cut-off, 4002 the avatar is loading, 4003 a plan cap, 4009 every engine busy, 4401 bad token, 4404 gone. A session ends whenever its socket closes, and its token works once, so a client never redials a closed session: it starts a new one (the SDK's ended event). The SDK implements all of it; anyone building their own client has the format above and the SDK source.