How OpenAI Scaled ChatGPT Storage to Billion-User Scale: Engineering Lessons from Habitat

On September 11, 2026, OpenAI published the first engineering retrospective on Habitat, its online storage platform. The main lesson is that scaling is not only about adding machines. It also requires an evolvable platform for caching, access control, data residency, encryption, tenant isolation, rate limiting, routing, and database operations.

What OpenAI disclosed

• Habitat provides online storage for ChatGPT, the API, Codex, and internal services. OpenAI says the platform serves more than 500 petabytes of data and handles more than 70 million requests per second.
• Habitat brings together caching, ACL authorization, data residency, encryption, multi-tenancy isolation, rate limiting, and routing. The described stack includes Azure Cosmos DB, Nanobase, Valkey, blob storage, CDC services, Databricks, Rockset, and Kafka.
• OpenAI says the Python service once handled more than 20 million requests per second at peak. In Q2 2026, two engineers, Codex, and GPT-5.5 rewrote the entire service in Rust.
• According to OpenAI, the Rust service now handles 95% of production requests. The company reports 6x CPU efficiency and 15x memory efficiency compared with Python, with improvements in average and tail latency; Python is expected to be deprecated in the following weeks.
• OpenAI says a second retrospective will cover the storage layer and how Habitat serves more than 500 petabytes and more than 70 million requests per second.

This does not mean Rust automatically makes systems six times faster

These metrics are OpenAI’s disclosures about its own production system, not WayToClawEarn measurements. The post does not provide the complete hardware, compiler, traffic distribution, workload model, or baseline configuration. The CPU and memory figures therefore should not be generalized to an ordinary SaaS application.

The reusable lesson is the order of operations: define platform boundaries, data governance, and resource protection first, then decide whether a language migration is justified. If a team lacks clear service boundaries, instrumentation, and rollback conditions, rewriting in Rust may only move the complexity into another language.

A four-step checklist for smaller teams

1. Measure peak traffic, tail latency, memory pressure, connection pools, cache hit rate, and downstream errors instead of looking only at average response time.
2. Treat authorization, data residency, encryption, tenant isolation, and rate limiting as platform capabilities rather than scattered business logic.
3. During a migration, keep the old and new implementations under comparison traffic, recording inputs, versions, hardware, request types, failures, and rollback conditions.
4. Use a fixed load to compare CPU time, memory, average latency, P95/P99, downstream pressure, and operating cost. Expand the migration only when both the metrics and failure boundaries improve.

Why AI application builders should care

When agents, voice, and multimodal features accelerate request volume, the first bottleneck may be storage, permissions, queues, or data isolation rather than the model. Habitat is a reminder that a monetizable AI feature needs observable, recoverable, and governable infrastructure. Otherwise a growth spike can damage unit economics and reliability at the same time.

Evidence boundary

The 500 PB, 70-million-requests-per-second, 95% production share, 6x CPU, and 15x memory figures come from OpenAI’s engineering post. Toolify only aggregates the article title and release and does not independently validate the metrics. This article labels the numbers as vendor disclosures and does not turn them into universal performance guarantees.

Sources

• OpenAI engineering retrospective: https://openai.com/index/scaling-storage-one-billion-users-part-one/
• Toolify secondary aggregation: https://www.toolify.ai/daily-ai-news