Perplexity's CobbleDB: 5x Latency Cut, Costs Slashed

Beyond DynamoDB: AI's New Data Frontier

Perplexity's move from DynamoDB to CobbleDB is a compelling case study in how specialized, in-house solutions can outperform general-purpose managed services for specific, high-volume AI workloads. The core insight lies in the decoupling of durable storage (Pillar/S3) from hot-tier serving (CobbleDB), a pattern that addresses the unique read patterns of AI models requiring large document batches and embeddings. The achievement of a 5x latency reduction and over 20% cost savings at extreme scale (200k+ RPS) is particularly noteworthy. The technical details of CobbleDB's architecture – leveraging RocksDB, memory-mapped caching, NVMe, and speculative read hedging – highlight a deep understanding of performance bottlenecks in distributed key-value stores for this specific use case. The use of AI coding agents in development is also a fascinating glimpse into future software engineering practices.

However, this migration is not without its trade-offs, as the article rightly points out. The shift from a fully managed service like DynamoDB to an in-house solution places a significant burden on Perplexity's SRE team for operational management, including lifecycle, backups, and rebalancing. The reliance on eventual consistency, while acceptable for search serving, might be a disqualifier for other applications. Furthermore, the significant upfront investment in developing CobbleDB, even with AI assistance, represents a substantial undertaking. While Perplexity plans to open-source CobbleDB, the maturity and community support of such a new, specialized system will be crucial for wider adoption beyond its initial use case. Organizations considering a similar path must carefully weigh the operational overhead and development costs against the potential performance and cost benefits, especially if their workloads don't exhibit the same extreme read patterns as Perplexity's AI serving tier.

Key Points

• Perplexity replaced Amazon DynamoDB with an internally developed distributed key-value store called CobbleDB.
• The migration was driven by severe latency and cost issues when serving multi-kilobyte document batches (average 50KB) to large language models under heavy query volumes (200k+ RPS).
• CobbleDB achieved a 5x reduction in batch-read latency (median 31.4ms to 5.60ms) and over 20% reduction in storage costs.
• The new architecture decouples durable storage (Pillar/S3) from hot-tier retrieval (CobbleDB), using Lorry for batch aggregation.
• CobbleDB leverages RocksDB with memory-mapped caching and local NVMe, optimized for batched lookups and tolerating eventual consistency.
• Development of CobbleDB involved AI coding agents, contributing to its rapid creation.

---

📖 Source: Home Made CobbleDB Replaces DynamoDB at Perplexity to Cut Query Latency 5x and Reduce Cloud Storage