REST API: Conversation
Every operation of this group with its parameters, body, answer and errors, as the OpenAPI document states them.
Live AI: a question or an AI Line answered as an event stream, its cancel, the shared transcript, and the session-bound media links the events hand out.
Run a turn (event stream)
POST/v1/sessions/{id}/turnsViewer token
A fetch POST answered with Server-Sent-Events framing: each event is an event: <type> line and a data: <json> line, and a comment line (: hb) comes every 15 s. Errors before the stream opens are /v1 errors; a refusal is the turn.refused event, not an error. Closing the request cancels the turn.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
idRequired | path | string | The viewer session's id ( |
Request body (application/json)
| Field | Type | Description |
|---|---|---|
| When kind is “viewer” A typed (or locally transcribed) question to ONE addressee at an open Beat. | ||
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
epochRequired | integer0–9007199254740991 | |
positionRequired | object | WHERE in the story something happened: |
position.visitRequired | integer1–9007199254740991 | |
position.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
position.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
view | object | What the viewer sees, by alias only — the focused object, the ones in view — validated by the server against what is perceivable at the turn's position. Never free text. |
view.focus | string | A speaker's or an object's alias, as the documents spell it ( |
view.lookingAt | string[]≤ 16 items | |
locale | string≤ 35 characters | The reply language; absent: the session's. |
output | object | Absent: |
output.audioRequired | boolean | |
addresseeRequired | string | The speaker asked: the narrator's or a speaking character's or prop's alias. |
textRequired | string1–500 characters | |
source | "viewer_text" | "viewer_voice" | Absent: |
beatIdRequired | string1–160 characters | The Beat it is asked at (an interactive scene's synthetic Beat is |
| When kind is “viewer_audio” A recorded push-to-talk clip for networks that block WebRTC: uploaded first (≤ 8 MB), named here by its upload reference, transcribed by the server, then answered like a typed question. | ||
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
epochRequired | integer0–9007199254740991 | |
positionRequired | object | WHERE in the story something happened: |
position.visitRequired | integer1–9007199254740991 | |
position.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
position.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
view | object | What the viewer sees, by alias only — the focused object, the ones in view — validated by the server against what is perceivable at the turn's position. Never free text. |
view.focus | string | A speaker's or an object's alias, as the documents spell it ( |
view.lookingAt | string[]≤ 16 items | |
locale | string≤ 35 characters | The reply language; absent: the session's. |
output | object | Absent: |
output.audioRequired | boolean | |
addresseeRequired | string | A speaker's or an object's alias, as the documents spell it ( |
beatIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
uploadRefRequired | string1–200 characters | |
| When kind is “ai_line” An AI Line the story reached (a clip's start, or a rule's | ||
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
epochRequired | integer0–9007199254740991 | |
positionRequired | object | WHERE in the story something happened: |
position.visitRequired | integer1–9007199254740991 | |
position.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
position.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
view | object | What the viewer sees, by alias only — the focused object, the ones in view — validated by the server against what is perceivable at the turn's position. Never free text. |
view.focus | string | A speaker's or an object's alias, as the documents spell it ( |
view.lookingAt | string[]≤ 16 items | |
locale | string≤ 35 characters | The reply language; absent: the session's. |
output | object | Absent: |
output.audioRequired | boolean | |
lineIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
| When kind is “arrival” The narrator's answer on arrival: once the client's | ||
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
epochRequired | integer0–9007199254740991 | |
positionRequired | object | WHERE in the story something happened: |
position.visitRequired | integer1–9007199254740991 | |
position.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
position.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
view | object | What the viewer sees, by alias only — the focused object, the ones in view — validated by the server against what is perceivable at the turn's position. Never free text. |
view.focus | string | A speaker's or an object's alias, as the documents spell it ( |
view.lookingAt | string[]≤ 16 items | |
locale | string≤ 35 characters | The reply language; absent: the session's. |
output | object | Absent: |
output.audioRequired | boolean | |
Response 200
A stream of events: an event: <type> line and a data: <json> line per event, a blank line after each, and a comment line every 15 s.
text/event-stream: each event's data
| Field | Type | Description |
|---|---|---|
| When type is “turn.accepted” The turn is the server's now: its id, and who speaks. | ||
turnIdRequired | string (uuid) | |
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
kindRequired | "viewer" | "viewer_audio" | "ai_line" | "arrival" | |
speakerRequired | string1–200 characters | The speaker key of who answers. |
lineId | string1–160 characters |
|
| When type is “speaker.state” | ||
turnIdRequired | string (uuid) | |
speakerRequired | string1–200 characters | |
stateRequired | "thinking" | "speaking" | "idle" | |
| When type is “effect” | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
effectRequired | one of | |
| When kind is “gesture” | ||
effect.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.motionIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.intensityRequired | number0–1 | |
effect.pinRootRequired | boolean | Live gestures never move a character: root motion discarded. |
effect.expression | object | |
effect.expression.expressionRequired | string1–64 characters | |
effect.expression.intensityRequired | number0–1 | |
| When kind is “expression” | ||
effect.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.expressionRequired | string1–64 characters | |
effect.intensityRequired | number0–1 | |
| When kind is “look_at” | ||
effect.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.targetRequired | one of | |
| When kind is “viewer” | ||
| When kind is “entity” | ||
effect.target.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
| When kind is “navigate” | ||
effect.toBeatIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.landingRequired | "chapterStart" | "hold" | |
| When kind is “show_place” | ||
effect.entityIdRequired | string1–160 characters | The AI-showable viewpoint of the current scene. |
| When kind is “object” | ||
effect.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.behaviorId | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
effect.actionRequired | string1–64 characters | |
effect.changeRequired | object | What changed about an object: a toggle's state, a book's spread, a visibility override. |
effect.change.state | string1–64 characters | |
effect.change.spread | integer0–9007199254740991 | |
effect.change.visible | boolean | |
| When type is “text.delta” Streamed reply text (the transcript's interim line). | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
textRequired | string≤ 4000 characters | |
| When type is “audio.format” How the | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
encodingRequired | "pcm_s16le" | |
sampleRateRequired | integer0–9007199254740991 | |
channelsRequired | integer1–2 | |
| When type is “audio.chunk” About 100 ms of streamed speech, base64, numbered from 0 per speech. | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
seqRequired | integer0–9007199254740991 | |
dataRequired | string≤ 1048576 characters | |
| When type is “audio.url” A whole prepared speech (an AI Line made ahead) instead of streamed chunks. | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
audioRequired | object | A whole speech file: its renditions and how long it plays. |
audio.renditionsRequired | object[]1–4 items | |
audio.renditions[].codecRequired | "opus" | "aac" | |
audio.renditions[].urlRequired | string1–2048 characters | |
audio.renditions[].offsetMsRequired | number≥ 0 | |
audio.renditions[].primingMs | number≥ 0 | |
audio.durationMsRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
alignment | object | Where a speech file's alignment sidecar is: the same kind of link as its audio. |
alignment.urlRequired | string1–2048 characters | |
| When type is “alignment” | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
alignmentRequired | object | Word and viseme timings of streamed speech, in ms from the start of its audio (the alignment sidecar's shape). A streamed speech may send several |
alignment.wordsRequired | object[] | |
alignment.words[].textRequired | string | |
alignment.words[].startMsRequired | number≥ 0 | |
alignment.words[].endMsRequired | number≥ 0 | |
alignment.visemesRequired | object[] | |
alignment.visemes[].visemeRequired | string1–8 characters | |
alignment.visemes[].startMsRequired | number≥ 0 | |
alignment.visemes[].endMsRequired | number≥ 0 | |
alignment.visemes[].weight | number | |
alignment.visemeSourceRequired | string1–32 characters | |
alignment.captions | object[] | |
alignment.captions[].textRequired | string | |
alignment.captions[].startMsRequired | number≥ 0 | |
alignment.captions[].endMsRequired | number≥ 0 | |
alignment.generator | object | The machine-readable mark of synthetic speech: the speech is AI-generated ( |
alignment.generator.syntheticRequired | true | |
alignment.generator.byRequired | string1–64 characters | |
alignment.generator.providerRequired | string1–64 characters | |
alignment.generator.modelRequired | string1–128 characters | |
| When type is “speech.end” | ||
turnIdRequired | string (uuid) | |
speechIdRequired | string1–160 characters | |
| When type is “line.fallback” An AI Line answered with its pre-generated fallback: the variant the server selected for the session (one per combination of its tokens' values — "Cat, you chose B; the right answer was C"), spoken and captioned like a scripted line. Nothing about it is billed. | ||
turnIdRequired | string (uuid) | |
lineIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
speechIdRequired | string1–160 characters | Groups the effects, audio and alignment of one utterance. |
speakerRequired | string1–200 characters | The speaker key: |
textRequired | string≤ 4000 characters | |
audio | object | A whole speech file: its renditions and how long it plays. |
audio.renditionsRequired | object[]1–4 items | |
audio.renditions[].codecRequired | "opus" | "aac" | |
audio.renditions[].urlRequired | string1–2048 characters | |
audio.renditions[].offsetMsRequired | number≥ 0 | |
audio.renditions[].primingMs | number≥ 0 | |
audio.durationMsRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
alignment | object | Where a speech file's alignment sidecar is: the same kind of link as its audio. |
alignment.urlRequired | string1–2048 characters | |
| When type is “turn.refused” The turn was refused — not an error: the stream ends here, playback goes on. | ||
turnIdRequired | string (uuid) | |
clientTurnIdRequired | string (uuid) | A server-minted id (sessions, projects, releases, users): a UUID. |
reasonRequired | string1–64 characters | |
canned | object | |
canned.speechIdRequired | string1–160 characters | Groups the effects, audio and alignment of one utterance. |
canned.speakerRequired | string1–200 characters | The speaker key: |
canned.textRequired | string≤ 4000 characters | |
canned.audio | object | A whole speech file: its renditions and how long it plays. |
canned.audio.renditionsRequired | object[]1–4 items | |
canned.audio.renditions[].codecRequired | "opus" | "aac" | |
canned.audio.renditions[].urlRequired | string1–2048 characters | |
canned.audio.renditions[].offsetMsRequired | number≥ 0 | |
canned.audio.renditions[].primingMs | number≥ 0 | |
canned.audio.durationMsRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
canned.alignment | object | Where a speech file's alignment sidecar is: the same kind of link as its audio. |
canned.alignment.urlRequired | string1–2048 characters | |
ai | object | The session's live-AI status: the level and every reason that holds at it, and the scopes the token carries. The UI hides voice at |
ai.levelRequired | "off" | "text" | "voice" | |
ai.reasonsRequired | string[] | |
ai.scopesRequired | string[] | |
ai.voiceReplies | string1–32 characters | Added within v1: whether live replies are spoken here — the project's setting, one of |
| When type is “turn.final” The turn is complete: everything it said has been sent, and what it cost is recorded. | ||
turnIdRequired | string (uuid) | |
usage | object | What a turn's provider calls came to — on Studio draft sessions only, for the AI panel's Test chat: the tokens of every round (cached ones included in |
usage.promptTokensRequired | integer0–9007199254740991 | |
usage.cachedTokensRequired | integer0–9007199254740991 | |
usage.completionTokensRequired | integer0–9007199254740991 | |
usage.roundsRequired | integer0–9007199254740991 | |
usage.firstTextMs | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
usage.tools | object[]≤ 16 items | |
usage.tools[].nameRequired | string1–64 characters | |
usage.tools[].outcomeRequired | string1–32 characters | |
usage.creditsMilli | integer0–9007199254740991 | |
| When type is “error” The turn failed after the stream began; | ||
turnId | string (uuid) | The server's id of the turn: on every event ( |
codeRequired | string1–64 characters | |
messageRequired | string≤ 2000 characters | |
| When type is “facts” The server facts the turn journaled — | ||
turnIdRequired | string (uuid) | |
factsRequired | any[] | A journal as a reader takes it: every fact it knows, in order; a fact of a kind it does not know (a newer server's) is skipped rather than failing the whole answer. |
Errors
400invalid_input401invalid_token,session_expired403scope_required404not_found409stale_epoch,rewind_required429rate_limited
Open a session's private media
GET/v1/media/{token}/{path}No bearer token
The token is part of the path: the link itself is the credential.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
tokenRequired | path | string | The link's signature, as a turn event handed it out. |
pathRequired | path | string | The file, after the token (it may contain slashes). |
Response 302
A redirect to a short-lived download of the file. An unknown, tampered, expired or foreign link and an ended session are all the same 404.
Errors
404not_found
Cancel a turn
DELETE/v1/sessions/{id}/turns/{turnId}Viewer token
Parameters
| Name | In | Type | Description |
|---|---|---|---|
idRequired | path | string | The viewer session's id ( |
turnIdRequired | path | string | The turn's id ( |
Response 204
No content: whether the turn still ran, had ended or never existed.
Errors
401invalid_token,session_expired
Read the shared transcript
GET/v1/sessions/{id}/conversationViewer token
Parameters
| Name | In | Type | Description |
|---|---|---|---|
idRequired | path | string | The viewer session's id ( |
afterSeq | query | integer | The entries after this one (exclusive): the last |
limit | query | integer |
Response 200
The transcript since the conversation's window start, a page at a time.
| Field | Type | Description |
|---|---|---|
epochRequired | integer0–9007199254740991 | The session's epoch: a rewind since erased what came after where it went back to. |
entriesRequired | object[] | |
entries[].seqRequired | integer1–9007199254740991 | The log's order; the next page starts after the last one. |
entries[].kindRequired | string1–32 characters |
|
entries[].sourceRequired | string1–32 characters | Where it came from: |
entries[].anchorRequired | object | WHERE in the story something happened: |
entries[].anchor.visitRequired | integer1–9007199254740991 | |
entries[].anchor.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].anchor.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
entries[].beatId | string1–160 characters | The Beat it happened at (an interactive scene's synthetic Beat is |
entries[].speaker | string1–200 characters | Who said it: |
entries[].addressee | string1–200 characters | Who it was said to: a speaker key, or |
entries[].text | string≤ 4000 characters | |
entries[].lang | string≤ 35 characters | A UI or reply language: a BCP 47 tag ( |
entries[].turnId | string (uuid) | The server's turn that wrote it. |
entries[].clientTurnId | string (uuid) | The client's id of the turn: on the viewer's question, and on an AI Line's words. |
entries[].event | string1–32 characters | An event's kind: |
entries[].quiz | object | A |
entries[].quiz.quizIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].quiz.questionIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].quiz.respondentRequired | one of | Who answers a quiz: the viewer, or the viewer playing a cast member ("as Cat"). |
| When kind is “viewer” | ||
| When kind is “character” | ||
entries[].quiz.respondent.castIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].quiz.answerTextRequired | string≤ 4000 characters | The answer in words. |
entries[].quiz.correctRequired | boolean | |
entries[].marker | string1–32 characters | A |
entries[].navigation | object | A |
entries[].navigation.eventRequired | string1–32 characters | |
entries[].navigation.toBeatId | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].object | object | An |
entries[].object.entityIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].object.changeRequired | object | What changed about an object: a toggle's state, a book's spread, a visibility override. |
entries[].object.change.state | string1–64 characters | |
entries[].object.change.spread | integer0–9007199254740991 | |
entries[].object.change.visible | boolean | |
entries[].object.byRequired | string1–16 characters | |
entries[].line | object | A |
entries[].line.lineIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
entries[].line.fallbackRequired | boolean | |
moreRequired | boolean | More entries follow: ask again with |
Errors
400invalid_input401invalid_token,session_expired404not_found
Prepare an AI Line ahead
POST/v1/sessions/{id}/ai-lines/{lineId}/prepareViewer token
Fire and forget, about 12 s before the line: at most twice per line per epoch, never while live AI is off.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
idRequired | path | string | The viewer session's id ( |
lineIdRequired | path | string | The AI Line's id. |
Request body (application/json)
| Field | Type | Description |
|---|---|---|
epochRequired | integer0–9007199254740991 | |
positionRequired | object | Where the story is now. |
position.visitRequired | integer1–9007199254740991 | |
position.sceneIdRequired | string1–160 characters | An id from the project's documents: a scene, a Beat (an interactive scene's synthetic Beat is |
position.msRequired | integer0–9007199254740991 | A non-negative integer count or time in milliseconds. |
locale | string≤ 35 characters | The reply language the line will be taken in; absent: the session's. |
output | object | Absent: |
output.audioRequired | boolean |
Response 202
Prepare an AI Line ahead
| Field | Type | Description |
|---|---|---|
stateRequired | string1–32 characters |
Errors
400invalid_input401invalid_token,session_expired404not_found409stale_epoch