@elevenlabs/client

Minor Changes

• bac5c96: Add the experimental persistentSession session option for resumable text conversations over WebSocket, and the onConversationHistory callback for the transcript rows the server replays on resume. WebSocket sessions now also emit onConversationMetadata.

Minor Changes

• 33a5042: Add the transcriptEdit option to Scribe.connect() and dispatch the new edited_transcript server message:

  • transcriptEdit: a natural-language instruction applied to each committed transcript (max 2000 characters), sent as the transcript_edit query param. Cannot be combined with entityDetection.

  • edited_transcript (RealtimeEvents.EDITED_TRANSCRIPT), carrying the committed text and its edited_text (EditedTranscriptMessage). edited_text equals text when the instruction changed nothing.

  • useScribe() accepts the same transcriptEdit option, exposes an onEditedTranscript callback and attaches the edited text to the matching entry in committedTranscripts as editedText.

Patch Changes

• 32358cc: Export the MessagePayload type from the package root so consumers can type onMessage handlers.

• e2d1ac7: Expose stable response IDs for agent messages and use them to prevent a reply
  from rendering twice when it is resent during a turn, while retaining
  compatibility with older orchestrators.

Minor Changes

• afd2170: Forward the optional attachments field (url, name, mime_type) from the agent_response client event into the onMessage payload, so consumers can render files attached to agent messages, e.g. relayed from a human agent reply.

• aad288c: Add an optional webRtc.singlePeerConnection session option for WebRTC connections. Set it to false to force LiveKit's dual peer connection path, which restores the microphone request timing used before livekit-client began defaulting to the v1 join protocol. Leaving it unset keeps livekit-client's own default.

Minor Changes

• f94ef6d: Add fileIds to sendMultimodalMessage and dual-send file + files on the wire.

Minor Changes

• b60d460: Draw rich content alongside the first message, and echo button taps back to the
  server.

  A first_message_rich_content entry on the widget config is painted with the
  first message before any connection exists, so a text-only widget shows its
  buttons on a cold start. Tapping one starts the conversation with that button's
  message, the same way typing into the input does — previously a button was
  disabled until connected, which left a first-message button waiting for the
  session it would itself begin.

  A row the customer has already answered is retired rather than left sitting in
  the transcript. Whether a component does this is per-component, so display
  components can later stay and merely de-emphasise instead of disappearing.

  sendUserMessage accepts an optional attribution (richContentId) that rides
  the user_message event as rich_content_id; the widget sends it automatically
  when a button is tapped, on both the mid-conversation and conversation-starting
  paths. It names the row the button belonged to rather than the button itself
  given the row, the message text says which button was pressed. Purely additive:
  servers that predate the field ignore it, and the message text alone remains
  what the agent acts on.

• be9cb0c: Add support for the external_agent_disconnected client event. The client SDK exposes a new onExternalAgentDisconnected callback, and the widget leaves external-agent mode (and clears the typing indicator) when the external human agent disconnects and the AI agent resumes control.

• dce9815: Add an onMCPToolApprovalRequest handler option, which decides MCP tool calls that arrive in the awaiting_approval state and sends the approval result for you. Each tool_call_id is answered at most once; a handler that rejects or resolves to a non-boolean is reported through onError and denied, and a decision that arrives after the call left awaiting_approval or the session ended is dropped rather than sent. The handler receives an AbortSignal so approval UI can dismiss itself when that happens.

Patch Changes

• ce34dbc: Publish the configured inputDeviceId when a WebRTC session starts, instead of the browser's default microphone. The mic was enabled with LiveKit's setMicrophoneEnabled(true), which always captures the default device, so the selected device only drove the local level meter while the agent received a wrong or silent track. changeInputDevice now also acquires the new track before stopping the old one, so a failed re-acquire leaves the current microphone live instead of tearing it down.

• ce34dbc: Keep a muted WebRTC session muted across an input device change. changeInputDevice published the newly captured track without reapplying the session's mute state, so a user who switched microphones while muted kept sending audio to the agent while isMuted() and the volume meter still reported muted. The new track is now muted before it is published.

• 79a9d4f: Fail WebRTC session setup when the conversation initiation payload cannot be
  published, instead of resolving onto a session the server never initialized.

  WebRTCConnection.create() sent conversation_initiation_client_data through
  the same best-effort path as ordinary messages, which returns early when the
  room is no longer connected and swallows a publishData() rejection. Setup
  could therefore succeed, and onConnect fire, while the room and microphone
  stayed live for a conversation that was never initialized. The mandatory
  initiation send now surfaces those failures so the existing create() error
  path disconnects the room and reports the failure to the caller. Mid-session
  sends remain best effort.

Minor Changes

• eea19d2: Add support for the context_usage server event via a new onContextUsage callback. The event is emitted after each completed agent turn and reports { event_id, model, context_tokens, context_limit_tokens }, where context_tokens is the prompt size of the turn's last LLM generation and context_limit_tokens is that model's maximum context window — useful for surfacing how close a long conversation is to the context limit.

Patch Changes

• e3d1c40: Resolve React Native via the react-native export condition instead of sniffing globals, and let platform entry points inject the setup guidance shown when nothing is registered.

Minor Changes

• 880edd2: Expose missing realtime speech-to-text options on Scribe.connect(), matching elevenlabs-js#436:

  • secondaryLanguages: additional language codes that may be present in the audio, sent as repeated secondary_languages query params.

  • entityDetection: detect entities (PII, PHI, PCI, offensive language, or specific types) on committed transcripts. Accepts a single value or a list.

  • filterBackgroundAudio: enable background speech filtering to reduce false activations from nearby conversations and ambient noise.

  New server messages are now dispatched instead of being dropped:

  • final_transcript and final_transcript_with_timestamps (RealtimeEvents.FINAL_TRANSCRIPT / FINAL_TRANSCRIPT_WITH_TIMESTAMPS)

  • committed_transcript_entities, carrying DetectedEntity (RealtimeEvents.COMMITTED_TRANSCRIPT_ENTITIES)

  • invalid_request, sent when the server rejects the connection parameters (RealtimeEvents.INVALID_REQUEST), emitted alongside the generic ERROR event

  Also fixes vadSilenceThresholdSecs, minSpeechDurationMs, and minSilenceDurationMs lower-bound validation to treat the bound as inclusive, matching the API (e.g. vadSilenceThresholdSecs: 0.3 is now accepted instead of being rejected client-side).

  The scribe.asyncapi.yaml schema (and the generated Word/CommittedTranscriptWithTimestampsMessage/FinalTranscriptWithTimestampsMessage types) was cross-checked against the live contract at https://api.elevenlabs.io/speech-to-text-asyncapi.yml. The words shape is identical between committed_transcript_with_timestamps and final_transcript_with_timestamps on that contract (both support "audio_event", per-character timings, and channel_index), so both messages now share a single Word type instead of the narrower shape committed_transcript_with_timestamps previously used. Config also gained timestamps_granularity and max_tokens_to_recompute to match the session config the server echoes back.

  @elevenlabs/react's useScribe().committedTranscripts.words (WordTimestamp) is updated to match: type now includes "audio_event", characters is now WordTimestampCharacter (each with its own start/end) instead of string, and a channel_index field was added.

Patch Changes

• a068144: Fix a microphone-setup failure wedging useScribe, so a denied or dismissed
  permission prompt can be retried without remounting.

  A microphone-mode session whose getUserMedia call rejects can never send
  audio, but the socket was left open holding that session. useScribe kept its
  connection ref, and every later connect() short-circuited on "Already connected". The failed setup now closes the connection, which releases the
  stranded socket and lets the hook's existing close handling clear the ref.
  onError still fires first; the session then ends as disconnected with the
  error preserved.

  The hook's close handler also nulled its ref for whichever connection reported
  a close, so a late close from a replaced socket tore down the session that had
  replaced it — the race a consumer hit when working around the above with
  disconnect-then-reconnect. A close is now ignored when a newer connection owns
  the ref, while an explicit disconnect() still reports normally.

Minor Changes

• 93cf08c: Add the experimental rich_content event types and an onRichContent callback,
  called when the agent sends a component for the client to display, such as an
  item card. The callback receives { rich_content_id, component, props, event_id }
  and nothing is sent back, so the agent's turn does not wait on the client.

  This backs the embedded widget, which is the only client the server offers
  components to today, so the callback does not fire for other consumers. Treat
  props as agent-authored and untrusted when rendering it into a document.

• 1a1f440: Add workletPaths.scribeAudioProcessor to Scribe.connect({ microphone }) so the Scribe audio worklet can be self-hosted, matching the existing Conversation workletPaths option. This lets strict CSPs (script-src/script-src-elem without blob:/data:) load the worklet from a same-origin URL instead of falling back to a blob or data URL.

  The generated worklet processors (including scribeAudioProcessor.js) are now also published as static assets under @elevenlabs/client/worklets/*, so bundlers can copy them into your own static assets and serve them under script-src 'self'.

• fb06e12: Add self-hosted orchestrator session support. Passing orchestratorConfig to Conversation.startSession routes the conversation WebSocket to a self-hosted (private deployment) orchestrator and sends the agent configuration at connection time, mirroring the Python SDK's OnPremInitiationData.

Patch Changes

• 92e70c6: Fix conversation state consistency through disconnect:

  • BaseConversation now always reaches the disconnected status and fires onDisconnect, even when session teardown throws.

  • ConversationProvider no longer lets a late onDisconnect from a previous session clear a newer session's state, and no longer leaks unhandled rejections from endSession().

  • useConversationStatus now reports disconnected as soon as teardown starts, instead of holding connected while the conversation was already released — consumers guarding conversation access with status === "connected" could previously crash during disconnect.

Minor Changes

• a96f220: Add enableLogging option (boolean) to the Scribe realtime API, available on Scribe.connect and the useScribe hook. Setting it to false sends enable_logging=false on the WebSocket URL, which runs the session in zero retention mode so history features are unavailable for it. Zero retention mode may only be used by enterprise customers.

Minor Changes

• 139ed79: Add onAgentReasoningResponsePart callback to receive streaming reasoning response
  events from the agent. The callback receives { text, type, event_id } where type
  is one of "start", "delta", or "stop".