实时会话

Choose a Talk voice from chat¶

During an active browser, iOS, or Android realtime Talk call, ask the assistant to list the available voices or switch to one. The browser composer keeps only call controls; choose the default voice in Settings → Talk. The talk_voice tool lists the current provider, model, voice, and supported voice IDs for the call in the current conversation. Setting a voice reconnects that call while preserving its chat and captions. The replacement waits for finalized speech to be saved and receives bounded conversation history, including speech finalized while the old connection closes. Accepted agent work keeps running during this handoff. The assistant reports success only after the replacement call is ready; unsupported changes or connection failures return an error. Settings → Talk sets the voice default for future calls. In a Discord realtime voice channel, ask the agent to list or change voices using the same tool. The change applies to that connected room, including subsequent speakers, and leaves its saved Discord voice configuration unchanged. The Discord connection and agent conversation remain active while the provider connection is replaced.

iOS supports switching in WebRTC and Gateway-relay calls. Android supports it in Gateway-relay calls using its supported realtime models. Native speech and TTS fallback calls retain their existing voice settings. Switching uses the current Gateway connection and conversation; ending the call or changing Gateways cancels a pending switch.

Mobile apps enable switching when the connected Gateway advertises voice selection support. Calls still start on older Gateways, but switching remains unavailable until the Gateway is updated.

For Talk TTS playback, after setting talk.provider and the matching talk.providers.<provider> configuration, use /voice status to inspect the active provider and voice, /voice list [limit] to list its available voices, and /voice set <voiceId|name> to save a provider-scoped selection. Discord exposes the same command natively as /talkvoice.

Status and list are read-only. Setting a voice requires the message-channel owner or a Gateway client with operator.admin. Configuration, provider lookup, unknown-voice, and permission failures are returned visibly in chat. A masked API-key value in /voice status describes config only; it does not verify credential availability.

Client-owned realtime Talk normally forwards provider tool calls through talk.client.toolCall instead of calling chat.send directly. GPT-Live WebRTC sessions delegate on a Gateway-owned sideband, and the Gateway binds each delegation to the browser or Gateway-relay Talk session that owns it. Backend WebSocket bridges use the normal relay consult path. While a realtime consult is active, clients can call talk.client.steer or talk.session.steer to classify spoken input as status, steer, cancel, or followup; this includes GPT-Live delegations. Accepted steering queues into the active embedded run; rejected steering returns a reason such as no_active_run, not_streaming, or compacting. A newer GPT-Live spoken task also supersedes the running delegation.

Thin audio clients can request gateway-control-v1 in talk.client.create.capabilities. OpenAI GA Realtime requires a Platform API key for this mode. The released GPT-Live route keeps its existing ChatGPT OAuth or Platform authentication; unlisted routes require Platform authentication. Requesting Gateway control does not switch the selected model.

Success returns clientControl: { owner: "gateway" }, a 60-second single-use clientSecret, and the relative offer URL /plugins/openai/realtime/calls. The client posts an audio-only SDP offer and opens no provider data channel. The Gateway attaches the provider's server sideband and owns tools or native agent delegation, transcripts, steering, cancellation, and call cleanup while media continues directly between the client and OpenAI. Negotiated sessions share a two-session limit per client connection, including pending offers. Unsupported combinations, including GA with OAuth only, fail visibly instead of falling back to client-owned control. Existing browser clients omit this capability and keep their data channel and client transcript reporting.

In Gateway-controlled native calls and native Gateway relays, the provider's delegation starts each host action. Final speech transcripts are saved to history; they neither trigger actions nor repeat a delegation's action. Status keeps the current task running, cancellation stops it, and redirects or follow-ups target that call's active work. When the call has no active task, status and cancellation return a spoken no-active-run response, even if another call on the same connection and agent session has work in progress. Ordinary requests such as “Check the weather” still start tasks while idle. Genuine new tasks retain the native delegation replacement behavior.

These calls disable provider-generated delegation acknowledgments at creation. OpenClaw sends one neutral receipt when it launches a real task; status and cancellation requests wait for the host result instead, without waiting for final speech transcription. A full control queue produces a spoken refusal; retry after the pending controls finish. A task receipt is not confirmation that a model or tool has started, and submitting a spoken result is not proof of audible delivery.

Closing a native transport fences new delegations and late provider delivery; already accepted agent work retains its own cancellation lifetime. Spoken run cancellation is separate from ending the audio connection. Gateway-controlled native sessions acknowledge cancellation without speaking the canceled task's partial answer, empty-result fallback, or failed-task retry prompt. Timeouts remain failures rather than being silently treated as cancellations.

Finalized realtime user and assistant utterances are always appended live to the active agent session, so later chat and voice turns share one history. Client-owned transports report their finalized transcripts with stable entry ids; Gateway relay and Gateway-controlled WebRTC sessions append the same events server-side. Provider sessions also receive the bounded realtime profile context used by Discord voice.

Speech finalized while an accepted consult is starting remains visible in history and does not prevent the agent from adopting that consult's recorded input. Completed agent answers and newly admitted tasks still close the older input; speech records do not reopen it.

Gateway-controlled native WebRTC calls receive shared-session history as quoted historical background in their instructions, not as the new call's own user or assistant messages. This background can include prior calls and backing-agent answers; it does not establish the current call's live task state. It retains the newest history within 16 entries, 800 characters per entry, and 8,000 UTF-8 bytes including labels and quoting. This changes neither saved transcripts nor chat display. Native calls without negotiated host input control and direct WebSocket conversation seeds keep their existing representation.

Generated agent-consult prompts are internal input, not spoken user turns. New consult records are hidden from chat and excluded from later model context, while the active consult still receives the full question, context, and response style. Raw archives and session exports remain lossless. Existing consult records without the exclusion flag are not rewritten and remain eligible for model context.

Chat-backed Talk stores the spoken answer without a second copy of the successful consult answer in visible history; the internal answer remains in the raw transcript and model context. Tool activity, progress, errors, and interrupted replies retain their existing visibility.

Direct provider-owned consultations keep their own final answer visible in Chat. Accepted work can outlive a closed or replaced audio connection, so a spoken replacement is not guaranteed. If speech also arrives, both records may be visible; OpenClaw preserves the answer rather than guessing that the spoken text replaces it.

OpenAI GA browser Talk keeps provider conversation order even when an assistant reply finishes before the user's transcription or item announcements arrive out of order. Text streams immediately in the call view; late predecessor metadata places it beside the correct reply. Stopping a call drains finalized speech, skips unfinished transcriptions, and records a browser console warning for missing transcriptions or unresolved conversation links.

Google Live saves complete utterances during the call, including Gemini 3.1 transcriptions that omit an explicit transcription-finished flag. Partial text stays provisional until the provider's completion boundary.

Voice-originated consult runs require a new, exact spoken confirmation before high-impact actions such as sending messages, controlling nodes, browser/computer actions, service changes, destructive shell commands, or publication. The gate applies to runs started through talk.client.toolCall, the Gateway relay, and GPT-Live sideband delegations. The confirmation applies only to the canonical final execution arguments and is consumed once; if a policy or hook rewrites the approved action, OpenClaw blocks it until the rewritten action is confirmed. Unrelated concurrent runs remain unaffected. When a call closes, OpenClaw can send a compact Voice call changes digest for mutating tools to the session's last non-WebChat delivery target.

After a confirmation prompt, say yes to confirm the pending action or no to cancel it. Each confirmation permits one matching action; another action may need another confirmation. Native GPT-Live calls use the finalized user speech recorded for that call, so generated delegation text cannot supply confirmation. When a native consult is blocked by this gate, Talk returns a specific retry prompt. If the pending confirmation expires or is cleared before the result, Talk reports that the action did not run and asks for a fresh request.

Transcription-only Talk emits the same Talk event envelope as realtime and STT/TTS sessions, but uses mode: "transcription" and brain: "none". All Talk sessions broadcast events on the talk.event channel; clients subscribe to it for partial/final transcript updates (transcript.delta/transcript.done) and other session telemetry.

Transcription providers can advertise their model choices in talk.catalog.transcription.providers[].models. Pass model to talk.session.create to override the configured transcription model for that session. Omitting it keeps the provider configuration, then the matching agents.defaults.voiceModel, then the provider's own default.

Browser Video Talk is available for OpenAI Realtime WebRTC and Google Live provider-WebSocket sessions. OpenAI gets a single bounded JPEG when describe_view asks for visual context; it does not receive a continuous camera track. Google Live receives bounded JPEG frames directly from the browser at up to one frame per second, while describe_view reports the camera-stream state. In both cases, camera frames bypass the Gateway, and stopping Talk releases the camera and microphone tracks.

Browser Talk shows startup progress while preparing the session, waiting for microphone access, and connecting. Talk and dictation show microphone guidance while the browser's capture request is pending: bring the tab to the foreground and allow access if prompted. The browser can keep an unanswered permission request pending. In Talk, Stop voice input cancels startup and releases any microphone stream granted after cancellation.

Browser Talk acquires the microphone before creating the provider session, so time spent granting permission does not consume a short-lived connection token. If session creation fails, Talk releases the microphone before reporting the error.

If OpenAI cannot transcribe an utterance, browser Talk shows the provider's error without ending the call or inventing a transcript. You can speak again; audio responses continue independently of input transcription.

If the microphone disconnects or its permission is revoked, browser Talk ends the call and shows an error. Choose an available Microphone input, restore permission if needed, and start Talk again. An unexpected GPT-Live connection loss also ends the call with an error; automatic reconnection is not supported.