Dispatching Conversations

Overview

The dispatch endpoint allows you to run conversation completions in the background without keeping a connection open. This is particularly useful for:

• Long-running completions that may take several minutes
• Scenarios where the client may disconnect (page refresh, mobile apps)
• Batch processing where you want to fire-and-forget

How It Works

1. Dispatch: Call the dispatch endpoint with the same parameters as the complete endpoint. You'll receive a channelId in the response.

2. Subscribe: Use the channel subscribe endpoint to receive real-time events as the completion progresses.

3. Process: The completion runs in the background, publishing events to the channel including tokens, operations, and the final result.

Example Usage

Channel Events

The channel will receive the same events as the streaming complete endpoint:

• token: Individual tokens as they are generated
• message: Activity messages, function calls, etc.
• result: The final result
• error: Any errors that occurred

Important Considerations

• Channel Lifetime: Channels remain active during the completion. Once the completion finishes, the channel will no longer receive new events.

• Missed Events: If you subscribe after the completion has started, you may miss some events. Subscribe as soon as you receive the channel ID.

• Session Scope: The channel ID is scoped to your session for security.