OpenAI published an engineering piece that opens up its own storage platform. It is called Habitat, and the numbers are respectable: over 70 million requests per second, more than a billion people a week, close to 40 regions, over 500 petabytes.
The thing started in 2023 at DevDay as a small Python library for GPTs, with a single database behind it, Azure Cosmos DB.
The actual point
Authors Jon Lee, Chaomin Yu and Ben Ries drop one line worth keeping: building infrastructure at this scale is a lot of work, but not particularly hard. The pace is what is hard. Normally you build for 10x and hope it holds a few years. At OpenAI it was 10x three years running - per year.
What follows from that is not one grand architectural idea but a chain of tactical calls: turn the library into a service, hunt down delays in the asyncio loop, kill the tail latency in feature flag lookups, tame connection pools, stop downstream systems from being flooded. And move from Python to Rust in pieces.
A second part is promised, on multi-tenancy, read performance and the work with Cosmos DB.
Why this one stuck with me
Writeups like this are rare because they are uncomfortable. They show where things creaked, not where everything gleams. Python as a serving language for a storage platform is a decision you do not normally explain in public.
For those of us running agents on somebody else’s infrastructure, there is a plain reminder in here too. When Claude Code or Cowork hangs, it is rarely the model. It is a queue, a connection pool or a cache that someone else is trying to scale - three layers below anything we can see.
Sources: