Chat completions

POST /v1/chat/completions accepts the OpenAI chat format. Use an OmniRelay API key and SDK base URL https://api.omnirelay.ai/v1. First query List models: public catalog entries do not grant access.

Non-streaming call

This example uses gpt-6-luna; it must appear in your own list with mode: live. See First API call for complete SDK initialization.

curl -i https://api.omnirelay.ai/v1/chat/completions \
  -H "Authorization: Bearer $OMNIRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Hello OmniRelay"}],"max_completion_tokens":64}'

response = client.chat.completions.create(
    model="gpt-6-luna",
    messages=[{"role": "user", "content": "Hello OmniRelay"}],
    max_completion_tokens=64,
)
print(response.choices[0].message.content)
print(response.usage)

const response = await client.chat.completions.create({
  model: "gpt-6-luna",
  messages: [{ role: "user", content: "Hello OmniRelay" }],
  max_completion_tokens: 64,
});
console.log(response.choices[0].message.content);
console.log(response.usage);

Parameters and capabilities

Field	Live text contract
`model`	Exact callable ID from `GET /v1/models`; display names are not IDs.
`messages`	1–2,000 messages. Use text; `system` and `developer` require the corresponding capability. Each text value holds at most 1,048,576 characters, and the request body at most 4 MiB; the model context window still applies.
`max_tokens` / `max_completion_tokens`	Positive integer; send one field, never both. The gateway normalizes it for the model. Read `limits.output_tokens` and `limits.default_output_tokens`. GPT examples use `max_completion_tokens`; other text examples use `max_tokens`.
`stream`	Omitted means `false` (one JSON response). Recommended `true` when `capabilities.stream` is true, especially for agents and long generations.
`stream_options.include_usage`	With `stream: true`, request a final empty-choices chunk containing `usage` and settled `cost_microusd`.
`reasoning_effort`	Consult documented `reasoning_efforts`; ordinary values are forwarded and the model provider decides. Omitted effort uses `default_reasoning_effort` when configured.
`tags`	Optional request labels; see Tags.

Most integrated text models allow up to 128,000 output tokens and default to 32,000 (Gemini 3.1 Pro Preview and Gemini 3.6 Flash: at most 65,536); DeepSeek V4 Pro, GLM 5.3 and Kimi K3 default to and allow at most 4,096; GPT 5.4 Mini defaults to 64 and allows at most 256. Your model-specific values take precedence. GPT routes expose documented function tools (strict/named/parallel), JSON object/schema and Responses; Astra tools require Responses. Sampling, reasoning and tool combinations are model-specific. Vision is not integrated into the billing contract. tool_choice accepts auto, none and required; selecting a specific function additionally requires capabilities.named_tool_choice. A normal JSON HTTP response is not structured JSON generation. Unsupported fields return 400 unsupported_parameter; invalid live values return 400 invalid_parameter.

GPT 5.6 Sol

gpt-5.6-sol accepts text, system/developer messages, function tools (including strict definitions, named selection and parallel calls), JSON object and strict JSON Schema output. Chat accepts reasoning_effort: none | low | medium | high | xhigh; omitted effort defaults to none. With tools or tool-result history, Chat requires none. Use Responses for reasoning with tools or max effort. temperature (0–2) and top_p (0–1) are recommended with none effort. The model provider decides incompatible combinations: it may reject them (identifiable rejections include error.param) or ignore the field and answer normally. These newly enabled combinations follow the model’s published documentation; live validation of each combination is pending. Read sampling_parameters, sampling_reasoning_efforts, chat_tools_reasoning_efforts and responses_reasoning_efforts from your own model list. null means no additional effort restriction; an empty parameter/effort list means unsupported. A minimal request needs no optional parameters:

{"model":"gpt-5.6-sol","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Reply with exactly: OK."}]}

Omitted output limits default to 32,000 tokens (DeepSeek V4 Pro, GLM 5.3 and Kimi K3: 4,096; GPT 5.4 Mini: 64); the maximum is each model’s limits.output_tokens, including reasoning tokens. Set the limit explicitly for longer reasoning tasks. JSON Schema keywords are left to the model provider.

Response and reasoning content

A successful JSON response includes choices[0].message.content, usage and cost_microusd. The charge is in micro-USD: divide by 1,000,000 for USD. If capabilities.reasoning_content is true, optional reasoning text may appear in choices[0].message.reasoning_content. This is distinct from accepting reasoning_effort: displaying reasoning does not imply it can be switched off or adjusted. Currently DeepSeek V4 Pro and GLM 5.3 can return this field. Omitted reasoning detail counters do not remove completion tokens from billing. Save the X-Request-ID response header; support uses it to look up a call. Optional Request metadata provides additional routing information.

Streaming

Check capabilities.stream before enabling SSE. Successful streams end with data: [DONE]; the final choice chunk carries a finish reason such as stop or length. A usage chunk has no choices, so handle it separately.

Python

# Only use this ID if your own list includes it with stream support.
stream = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Hello OmniRelay"}],
    max_tokens=128,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta
        reasoning = getattr(delta, "reasoning_content", None)
        if reasoning:
            print(reasoning, end="", flush=True)
        if delta.content:
            print(delta.content, end="", flush=True)
    if chunk.usage:
        print("Usage:", chunk.usage)
        print("Cost (micro-USD):", getattr(chunk, "cost_microusd", None))

For supported models, streaming reasoning text is choices[0].delta.reasoning_content. The gateway always collects final usage, even when the client does not request the usage chunk.

Disable automatic SDK retries. A timeout, error event or missing [DONE] can leave a live charge pending reconciliation. Use the error code to decide whether the call was charged before sending another paid call. An HTTP 200 at stream opening alone does not establish success. See Usage and billing and Errors.

See the Create chat completion for the full schema.

Capability matching across callable models

Every callable route, including sale aliases, has an explicit contract. Choose parameters from your own /v1/models response. Documented features are forwarded; budget and protocol checks remain necessary. New parameter combinations are documentation-based and await individual live validation.

Routes	Integrated features and combination constraints
GPT 5.4 Mini	Function tools, strict/named/parallel, JSON object/schema, Responses, none/low/medium/high/xhigh; sampling requires none.
GPT 5.6 Sol/Terra/Luna; GPT 6 Sol/Luna	Same; Chat tools require none; Responses supports reasoning with tools and adds max.
GPT 6 Astra	Chat text/JSON; tools require Responses; low/medium/high/xhigh, plus max in Responses; no none or sampling.
All Claude and sale/Claude routes	Compatible function tools, named selection, parallel calls, developer, temperature/top_p and stop. The compatibility layer ignores strict and response_format, so no schema guarantee is advertised.
DeepSeek V4 Pro	Function tools, JSON object, temperature/top_p, stop and assistant reasoning history, plus enabled/disabled thinking and documented effort aliases; required/named tool choice requires disabled thinking. The model may ignore sampling in thinking mode.
GLM 5.3	Function tools, tool_choice:auto, JSON object, tool_stream, reasoning history and sampling; thinking only enabled; low/high/max effort.
Gemini 3.1 Pro Preview, Gemini 3.6 Flash, Kimi K3	Text with system/user messages and SSE are verified. Tools, JSON modes and sampling are not yet reviewed and are not advertised; other fields are still forwarded and the model provider decides.

Preserve assistant reasoning_content when replaying DeepSeek tool conversations. Read tool_choice_values, tool_choice_reasoning_efforts, thinking_modes and parameter_ranges; Claude/GLM temperature is at most 1 and GLM top_p at least 0.01. Ordinary optional null values are forwarded and the model provider decides; empty tool lists are accepted. Multiple candidates, extra paid services, hosted tools and image/audio inputs require their own settlement contracts.

Parameter passthrough

All callable models use parameter_policy: upstream-first. Capability flags are documented guidance. Ordinary fields are not rejected because a local flag is false, an effort is absent from a list or a schema keyword is new. The model provider decides on fields and combinations and may ignore unsupported ones. Function definitions, legacy functions, reasoning history, ordinary extension fields and schemas are preserved. Sampling is not silently removed; the gateway does not change requested effort or protocol. Some models ignore fields they do not support instead of rejecting them, so a successful response does not prove that an optional field took effect; rely on the capability list for what is supported. Model access, authentication, body size, output budgets and fee reservation still apply. Multiple candidates, paid service tiers, pro reasoning, hosted tools, media, stored state and background requests need separate settlement contracts. Output-limit aliases are normalized, with a default when omitted; default effort is supplied only without explicit effort/reasoning/thinking. JSON/stream extension fields are retained; OmniRelay supplies settled cost. Model provider 400/422 rejections release holds and are not charged; provider failures (5xx), timeouts and interrupted streams remain pending reconciliation. New combinations await individual paid validation.

V1 Documentation

Quick Start

User manual

Overview

LLMs

Images

Videos

General

Non-streaming call

Parameters and capabilities

GPT 5.6 Sol

Response and reasoning content

Streaming

Capability matching across callable models

Parameter passthrough

​Non-streaming call

​Parameters and capabilities

​GPT 5.6 Sol

​Response and reasoning content

​Streaming

​Capability matching across callable models

​Parameter passthrough

Non-streaming call

Parameters and capabilities

GPT 5.6 Sol

Response and reasoning content

Streaming

Capability matching across callable models

Parameter passthrough