←back to manuals

Dispatching Stateful Conversations

Overview

The stateful dispatch endpoint allows you to run conversation completions in the background for an existing conversation. This is particularly useful for:

  • Long-running completions that may take several minutes
  • Scenarios where the client may disconnect (page refresh, mobile apps)
  • Batch processing where you want to fire-and-forget

How It Works

  1. Dispatch: Call the dispatch endpoint with the conversation ID and the same parameters as the complete endpoint. You'll receive a channelId in the response.

  2. Subscribe: Use the channel subscribe endpoint to receive real-time events as the completion progresses.

  3. Process: The completion runs in the background, publishing events to the channel including tokens, operations, and the final result.

Example Usage

Channel Events

The channel will receive the same events as the streaming complete endpoint:

  • send_result: The user message was stored
  • receive_result: The assistant response was generated
  • error: Any errors that occurred

Important Considerations

  • Channel Lifetime: Channels remain active during the completion. Once the completion finishes, the channel will no longer receive new events.

  • Missed Events: If you subscribe after the completion has started, you may miss some events. Subscribe as soon as you receive the channel ID.

  • Session Scope: The channel ID is scoped to your session for security.

  • Conversation State: The completion will use and update the conversation's existing message history.