GPT-4o Realtime

Technical Specifications of gpt-4o-realtime

What is gpt-4o-realtime?

gpt-4o-realtime is a realtime multimodal model available through CometAPI for developers building highly responsive AI applications. It is designed for scenarios where low latency matters, such as live voice assistants, interactive speech-to-speech systems, and applications that need to process text and audio in the same workflow.

This model supports multimodal communication, allowing applications to send text or audio inputs and receive text or audio outputs. With a maximum context length of 128,000 tokens, gpt-4o-realtime can also support longer interactions and more context-aware conversations than smaller-session realtime systems.

Main features of gpt-4o-realtime

• Low-latency interaction: Built for realtime use cases where fast response times are essential for smooth user experiences.
• Multimodal input and output: Supports both text and audio workflows, enabling flexible application design.
• Speech-to-speech support: Well suited for conversational voice interfaces that take spoken input and return spoken output.
• Large context window: Supports up to 128,000 tokens of context for more coherent extended sessions.
• Flexible realtime application support: Useful for live assistants, interactive tools, customer support agents, and other responsive multimodal products.
• Separate text and audio pricing: Developers should account for text and audio usage independently when estimating costs.

How to access and integrate gpt-4o-realtime

Step 1: Sign Up for API Key

To get started, sign up on CometAPI and generate your API key from the dashboard. After that, store the key securely and use it to authenticate every request to the API.

Step 2: Connect to gpt-4o-realtime API

The Realtime API uses WebSocket connections. Connect to CometAPI's WebSocket endpoint:

const ws = new WebSocket(
  "wss://api.cometapi.com/v1/realtime?model=gpt-4o-realtime",
  {
    headers: {
      "Authorization": "Bearer " + process.env.COMETAPI_API_KEY,
      "OpenAI-Beta": "realtime=v1"
    }
  }
);

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      modalities: ["text", "audio"],
      instructions: "You are a helpful assistant."
    }
  }));
});

ws.on("message", (data) => {
  console.log(JSON.parse(data));
});

Step 3: Retrieve and Verify Results

The Realtime API streams responses through the WebSocket connection as server-sent events. Listen for response.audio.delta events for audio output and response.text.delta for text. Verify the session is established and responses are streaming correctly.