Contenu source (brut)
<p>When we tell people that we’ve moved over 300 terabytes of Solana archival data off of ClickHouse and onto RocksDB, the reaction is almost always the same: <em>Why would you ever do that?</em></p><p>ClickHouse is the obvious choice for petabyte-scale analytical workloads, whereas RocksDB is not. </p><p>ClickHouse is trusted by some of the world's largest data consumers. </p><p>For example, <a href="https://blog.cloudflare.com/clickhouse-query-plan-contention/" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">Cloudflare</span></a> runs it across more than a thousand replicas to handle hundreds of millions of inserts per second. </p><p><a href="https://clickhouse.com/blog/how-anthropic-is-using-clickhouse-to-scale-observability-for-ai-era" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">Anthropic</span></a> runs a custom, air-gapped ClickHouse deployment to power Claude's observability. <a href="https://www.uber.com/us/en/blog/logging/" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">Uber</span></a>, <a href="https://innovation.ebayinc.com/stories/ou-online-analytical-processing/" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">eBay</span></a>, and <a href="https://clickhouse.com/blog/nyc-meetup-report-large-scale-financial-market-analytics-with-clickhouse" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">Bloomberg</span></a> have also been using it in production for years.</p><p><a href="http://rocksdb.org/" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">RocksDB</span></a>, on the other hand, is an embedded key-value store with sparse documentation, primarily used as the engine within other databases (e.g., CockroachDB, TiKV, MyRocks), rather than as the direct foundation for a customer-facing historical data service.</p><p>However, the move cut our compressed storage footprint from ~330TB to ~190TB, fixed the long tail of our slowest queries (e.g., <a href="https://x.com/nick_pennie/status/2045213000596852861?s=20" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">p95 latency for </span><code><span style="text-decoration: underline">getTransactionsForAddress</span></code><span style="text-decoration: underline"> calls dropped from 350ms to 30ms</span></a>), and put us in a position to innovate on Solana’s read layer—a position that wouldn’t have been possible with ClickHouse at the performance and scale demanded at Helius.</p><p>This article explains why we migrated from ClickHouse, what we learned about the workload along the way, and how we’re using RocksDB in production at scale. </p><h2>What is Solana Archival and Why It Matters</h2><p>At its core, Solana is a network of nodes that communicate to agree on new information without having to trust one another. </p><p>A <em>node </em>is a computer on Solana that runs a client (e.g., Agave, <a href="https://www.helius.dev/blog/what-is-firedancer" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">Firedancer</span></a>) that abides by a specific set of rules to help facilitate the agreement of new information. </p><p>A <a href="https://www.helius.dev/validator" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">validator</span></a> is a Solana node that secures the network by producing blocks (i.e., appending groups of transactions to Solana’s ledger) and voting on the validity of other blocks. </p><p><a href="https://www.helius.dev/solana-rpc-nodes" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">RPC nodes</span></a> are validators that do not participate in block production or voting. Instead, they observe the network and track all the new information it produces. RPCs allow users to query the network for specific data using the <a href="https://www.helius.dev/docs/api-reference/rpc/http-methods" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">JSON-RPC specification</span></a>. </p><p>These nodes, however, do not keep this data forever. </p><p>To stay within Solana’s hardware requirements, older blocks, transactions, and account states are pruned so that nodes retain only the most recent view of Solana’s history. </p><p>This is problematic when trying to look up a transaction signature from a year ago, pull every signature that ever touched a given wallet across its lifetime, or scan a program’s full execution history.</p><p>Archival broadly refers to the entire layer that stores Solana’s data since genesis. It logs and indexes everything the chain produces—every block, transaction, account interaction, Cross Program Invocation (CPI), and transaction log—and keeps it queryable beyond the standard ~2-day (i.e., 1 <a href="https://www.helius.dev/blog/solana-slots-blocks-and-epochs#defining-epochs-in-solana" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">epoch</span></a>) short-term storage window. </p><p>Archival is what makes historical queries possible.</p><p>The default approach on Solana to storing archival data has been <strong>Google BigTable</strong>. Anza maintains a BigTable instance that RPC providers can reach on demand to serve historical queries. </p><p>It works, but BigTable is expensive, egress costs make running your own copy painful, and there’s almost no engineering work that can be done on your end to make it faster—you’re at the mercy of Google’s storage, with all of its cost structure and none of the control. </p><p><strong>Old Faithful </strong>is a collection of tooling maintained by Triton One that can produce Content Addressable Archives (CARs) from ledger RocksDB archives and serve them via Solana’s standard RPC and gRPC interfaces. It is a meaningful step toward decentralizing Solana’s archive layer, providing a valuable source of redundant, verifiable history. </p><p>However, its trade-offs with respect to developer experience and performance (e.g., getting started requires custom tooling rather than interfaces teams can build against, it’s optimized for durable archival rather than performance and scale) do not make it a meaningful alternative for latency-sensitive workloads. </p><p>The core problem with archival is that you have <em>N </em>petabytes of raw data. With over 500 billion transactions, ~1.3 trillion row account-to-transaction indexes, and random access patterns, how are sub-10ms lookups achieved? What do sub-50ms end-to-end queries look like? Both Google BigTable and Old Faithful fail to offer a solution to this.</p><p>Moreover, if you wanted to build custom filtering and sorting options, or improve the developer experience by offering anything richer than the standard JSON-RPC methods, you’d need your own archival index. </p><p>So, <a href="https://www.helius.dev/blog/introducing-gettransactionsforaddress" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">we built one</span></a>.</p><h2>ClickHouse: The Pragmatic First Bet</h2><p><a href="https://clickhouse.com/" rel="noopener noreferrer" target="_blank"><span style="text-decoration: underline">ClickHouse</span></a> was the most pragmatic place to start building out our new archival system. It was the fastest path to shipping a product that was already better than one built on BigTable.</p><p>ClickHouse is a mature, columnar database with great tooling and documentation. It compresses time-series data well, which is especially useful for Solana, since its block data is fundamentally time-ordered. </p><p>It’s straightforward to create a database, write SQL against it, iterate on schemas, and optimize for time-to-market, so customers have something in front of them relatively quickly.</p><p>ClickHouse didn’t serve traffic all on its own. Rather, it was the bottom
