Accelerating Agentic Workflows with WebSockets in OpenAI's Responses API

OpenAI's Responses API has long been the backbone of agentic workflows, enabling models like Codex to perform iterative tasks such as debugging code, running tests, and applying fixes. However, as model inference speeds increased dramatically with the launch of GPT-5.3-Codex-Spark, the API itself became a bottleneck. Engineers at OpenAI tackled this challenge by introducing WebSockets, a persistent connection protocol that reduced API overhead and improved latency by 40%. This article explores the technical innovations behind this shift and provides practical guidance for builders integrating WebSocket-based workflows.

Understanding the Bottleneck

Historically, API service overhead was overshadowed by the slower inference speeds of earlier models like GPT-5 and GPT-5.2, which operated at 65 tokens per second (TPS). With GPT-5.3-Codex-Spark achieving speeds of over 1,000 TPS on specialized Cerebras hardware, the inefficiencies in the API became glaring. Each agentic workflow involved multiple back-and-forth API calls, processing redundant context and conversation history, which compounded latency as tasks grew more complex.

Optimizing for Speed: Initial Improvements

OpenAI began by optimizing critical-path latency for single requests. Key improvements included caching rendered tokens and model configurations in memory, reducing network hops by bypassing intermediate services, and streamlining safety classifiers to flag issues faster. These changes improved time-to-first-token (TTFT) by 45%, enhancing responsiveness. However, these optimizations were insufficient to fully leverage the speed of GPT-5.3-Codex-Spark, as the structural inefficiencies of treating each request independently persisted.

Introducing Persistent Connections with WebSockets

To address the structural inefficiencies, OpenAI explored persistent connection protocols such as WebSockets and gRPC bidirectional streaming. WebSockets emerged as the preferred choice due to its simplicity and compatibility with existing API input and output shapes. By maintaining a persistent connection, reusable context and state could be cached in memory, eliminating redundant processing for follow-up requests.

The WebSocket prototype redefined latency expectations. Using `asyncio`, the Responses API asynchronously blocked in the sampling loop after a tool call, sending a `response.done` event to the client. The client executed the tool call and returned a `response.append` event with the result, unblocking the sampling loop and allowing the model to continue. This approach minimized repeated API work, enabling preinference and postinference tasks to be performed only once per agent rollout.

The WebSocket prototype changed what we thought was possible for Responses API latency, enabling agentic workflows to scale with the speed of GPT-5.3-Codex-Spark.

Balancing Familiarity and Performance

To ensure developers could adopt WebSocket mode without significant rewrites, OpenAI retained the familiar API shape. Developers continued using `response.create` with the same body, adding `previous_response_id` to reference cached state. The server maintained an in-memory cache scoped to the connection, storing prior responses, tool definitions, and sampling artifacts. This design struck a balance between minimal latency and ease of integration.

1. Use `response.create` with `previous_response_id` to leverage cached state.
2. Ensure your client supports persistent WebSocket connections.
3. Test for edge cases where connection state might be lost or corrupted.

Adoption Results and Lessons Learned

During the alpha phase, key coding agent startups integrated WebSocket mode into their infrastructure, reporting up to 40% improvements in agentic workflows. The launch saw widespread adoption, with Codex ramping up the majority of Responses API traffic onto WebSocket mode. This transition unlocked the full speed of GPT-5.3-Codex-Spark, achieving bursts of up to 4,000 TPS.

• WebSocket mode enabled faster and more efficient agentic workflows.
• Caching reusable state reduced redundant processing and improved latency.
• Developers appreciated the familiar API shape, easing adoption.

• https://openai.com/index/speeding-up-agentic-workflows-with-websockets

Builder implications

For teams evaluating Accelerating Agentic Workflows with WebSockets in OpenAI's Responses API, the useful question is not whether the announcement sounds important. The useful question is whether it changes how an agent system is built, tested, operated, or bought. The source from openai.com gives builders a concrete signal to inspect: Speeding up agentic workflows with WebSockets in the Responses API - OpenAI. That signal should be mapped against the parts of an agent stack that usually become fragile first, including tool contracts, long-running state, evaluation coverage, cost visibility, failure recovery, and the handoff between prototype code and production operations.

Adoption checklist

1. Identify the workflow where WebSockets, Responses API, Codex, GPT-5.3 already creates measurable pain, such as slow triage, brittle handoffs, unclear ownership, or poor observability.
2. Write down the current baseline before changing the stack: latency, cost per run, recovery rate, review time, and the percentage of tasks that need human correction.
3. Prototype against a real internal workflow instead of a demo task. The workflow should include imperfect inputs, missing context, tool failures, and at least one approval step.
4. Add traces, event logs, and evaluation checkpoints before expanding usage. A new framework or model is hard to judge when the team cannot see where the agent made its decision.
5. Keep rollback boring. The first version should let an operator pause automation, inspect the last decision, and return control to a human without losing state.
6. Review the source again after testing. The source-backed claim should line up with observed behavior in your own environment, not just with launch copy or release notes.

What to watch next

The next signal to watch is whether builders start publishing implementation notes, migration stories, benchmarks, or reliability reports around this source. That secondary evidence matters because agent infrastructure often looks clean at release time and only shows its real shape once teams connect it to messy business workflows. Strong follow-on evidence would include reproducible examples, clear limits, documented failure recovery, and customer stories that describe what changed in the operating model.