XNews

OpenAI Habitat: From Python lib to Rust service handling 7000M QPS

OpenAI Habitat: From Python lib to Rust service handling 7000M QPS

OpenAI's internal storage platform Habitat evolved from a Python client library (2024) connecting to a single DB, to a Python service (2025) for deployment control, and finally to a Rust service (2026 Q2) rewritten by 2 engineers + Codex + GPT-5.5. It now spans ~40 regions, serves 10B+ weekly users, 500+ PB data, 70M+ QPS, with the Rust version handling 95% of traffic. Key lessons from Python scaling to 20M QPS: asyncio scheduling delays dominated tail latency (mitigated by limiting concurrent requests per process and adding jitter to periodic tasks), LIFO connection pools caused metastable failures (fixed by FIFO), and massive process counts led to downstream connection storms (solved via Envoy HTTP/2 multiplexing).

Source posts