RAG on the Decentralized Web: An Empirical Study of Akash and Golem

Abstract

Decentralized compute markets claim to be an alternative to the cloud, yet the systems community lacks hard evidence on whether they can run modern workloads end to end. We put this claim to the test by deploying a Retrieval-Augmented Generation (RAG) pipeline on the Akash and Golem networks. Our analysis reveals a striking paradox: supply is abundant, with over 10,000 CPU cores and 50~TiB of memory available, yet utilization remains minimal (about 0.3% on Akash). When mapped carefully, several RAG pipeline stages execute correctly and up to 3x cheaper than AWS. Others degrade under node churn and network bottlenecks. The limiting factor is not raw capacity but predictability. We also observe that decentralization in practice is already hybrid, with centralized gateways and offchain services masking blockchain complexity. Taken together, our findings suggest that decentralized markets are neither hype nor drop-in cloud replacements, but an underutilized substrate whose viability depends on better scheduling, reliability mechanisms, and trust primitives.