Chat

Use /v1/chat for anything that is not a live UI — a background job, a Slack bot, an evaluation harness. One request, one JSON answer, no plumbing.

Use /v1/chat/stream when a person is watching a cursor blink. Tokens arrive as they are produced, which makes a four-second answer feel immediate.

Worth knowing about each field:

• session_id — send it back to continue this conversation. Omit it and the next message starts a new one.
• conversation_id — the conversation's id, for GET /v1/conversations/{id} and its messages.
• sources — what the answer was grounded in: each entry names the knowledge source (source), its collection, the passage (content) and how well it matched (relevance_score, 0–100). Empty means nothing relevant was retrieved, which is your signal that the answer is not from your content.
• confidence_score — confidence in the answer, 0–100: how strong the evidence was and how closely the reply sticks to it. How well the knowledge base matched the question alone is retrieval_confidence.
• actions — anything the bot triggered this turn, such as a handoff request or a CTA. See Actions.
• usage — tokens for this turn, so you can attribute spend per customer.

Pass the session_id you got back. History is kept server-side; you do not need to resend previous messages. The request still accepts a conversation_history array, but the server ignores it and rebuilds the history from the conversation's stored turns.

"How much is it?" only makes sense with the previous turn in view — that is what the session buys you.

POST /v1/chat/stream answers with text/event-stream. The request body, scope and billing are the same as POST /v1/chat; only the response differs. Frames arrive in this order:

A content frame is OpenAI-shaped, so an existing chunk parser reads it as is:

finish_reason is always null: the stream ends with [DONE], never with a finishing chunk. Lines starting with : (for example : ping) are keep-alive comments sent while the model is silent; every SSE parser skips them.

Every frame

Unknown event names can appear as the API grows; ignore them rather than failing.

When the loop ends, answer is the reply exactly as stored and metadata carries the same query_id, session_id, sources and usage the blocking endpoint returns in its body (query_id is the message id).

The stream carries no conversation_id. Continue the conversation with the session_id from metadata; when you need the conversation itself, look it up with GET /v1/conversations/by_session/{sessionId}. The SDKs parse frames the same way and hand you each one as { event, data, json }.

Retrying a stream

An SSE body cannot be replayed frame by frame, so a retry with the same Idempotency-Key answers with a JSON summary of the original turn and Idempotency-Replayed: true instead of a second stream. You still recover query_id and session_id, and you are not charged twice.

When something goes wrong mid-stream

Problems before the first byte (a bad key, a missing scope, no credits) arrive as an ordinary JSON error response with a status code, never as a stream. Once the stream has started, the status is already 200, so the trailing metadata frame carries the outcome instead:

• error: "turn_persistence_failed" — the answer was generated but could not be recorded. The text you received is valid; there is no query_id to attach feedback to. Treat it as a warning, not a failed request. The Idempotency-Key is released, so a retry runs a new turn.
• error: "chat_generation_failed" — generation failed after the stream opened. The content frames carried the chatbot's fallback message instead of an answer, then [DONE], then this frame. Nothing was stored, and the Idempotency-Key is released, so you may retry.

Everything a chat turn produces is readable afterwards:

• GET /v1/conversations/{id}/messages — the whole thread.
• GET /v1/messages/{id} — one message, including its sources. A stored message keeps its sources as compact references (dataset_id, scores on a 0–1 scale, and a title / source_url when known), not the passages POST /v1/chat returned: keep the turn's response if you need those.

See Conversations.

• No Idempotency-Key. The single most expensive omission on the API.
• Resending history yourself. The session already has it; extra context costs tokens and can confuse the answer.
• Ignoring empty sources. That is the difference between "answered from your content" and "answered anyway".
• Buffering the stream. A proxy that buffers turns streaming back into a slow blocking call. Disable buffering for this path.
• Stopping at [DONE]. The metadata frame comes after it.
• Ignoring final_text. The streamed text can differ from the stored answer; the frame is how you learn the stored one.

• Conversations — read, review and export threads.
• Actions — let a turn call your systems.
• Rate limits — chat has its own budget.