September 20, 2022 Isabel Kurz

Why Decentralized Storage Matters for Web Development

Written by:
Isabel Kurz
Published on:
September 20, 2022

The dawn of the age of Web3 and the metaverse ushers in an exciting new era that promises to solve legacy problems of the internet. Innovations such as self-sovereign digital identity, privacy protocols, and gamification using tokens offer the chance to reclaim our personal data and online personas from big tech firms while distributing value more fairly among participants of applications and services.

However, these benefits depend on a holistic approach to decentralization from the bottom up, which includes infrastructural elements such as data storage. Here, we examine the case for decentralized data storage as an enabler of Web3 and metaverse development, along with some of the solutions available to developers.

Our Global Data Problem

The pace at which we’re collectively generating data is astounding. By 2021, the world was generating over a billion gigabytes of data each day. What’s more, the amount is increasing all the time as the population is growing, more people are using digital devices, and online services have become more sophisticated and prevalent. The advancing adoption of Web3 and the metaverse is set to level up the amount of data we’re producing exponentially.

To date, most of this online data has been stored using centralized enterprise infrastructure providers like AWS or IBM or dedicated cloud storage providers like Dropbox. However, this model has several critical flaws, and even more so when considered from the perspective of building decentralized apps. Firstly, it depends on centralized providers being able to continually scale to meet the ever-increasing demand for more storage. To achieve this, providers need to keep building more and bigger data centers, which both have a substantial environmental footprint to construct and operate.

Furthermore, to be able to guarantee the security and integrity of their storage solutions, providers must build in redundancy. Data is often stored in multiple data centers around the globe, ensuring that if there’s downtime in one location, others can ensure that data is still available. This requirement generates a need for even more data centers.

Centralized Storage – Not Fit for Web3 Purpose

Blockchain, cryptocurrencies, and Web3 are founded on the principles of decentralization, which creates an inherent conflict for dApp developers needing file and data storage. Many everyday users are perhaps unaware that many of the apps they perceive as decentralized are still actually running on centralized services under the hood because, until relatively recently, that’s all that was available. This means that any user or app data held by such a dApp is effectively under centralized ownership. Data can be censored or seized by hackers or government officials or lost completely if sufficient redundancy isn’t built in.

Decentralized data storage solutions aim to overcome many of the issues of centralized data storage and provide a means for Web3 developers to reap the benefits of decentralization under the hood of their blockchain apps. Decentralized storage solutions can take different forms, but typically, they source the storage from a distributed network of participants who have the storage capacity to contribute to the network. There’s also usually a system of incentives that will be familiar to any blockchain user, where those who want to use the available storage pay for the service using tokens, which in turn are used to reward those who contribute their spare storage capacity to the network.

Benefits of Decentralized Storage

These solutions deliver all the benefits one can expect from blockchain-based services. As they’re decentralized, they’re censorship-resistant and with built-in redundancy ensuring no downtime where data may be inaccessible. Privacy protocols may not always be baked in, but where they are, users can be assured that their data isn’t even available to the network participants who may be storing it.

Decentralized data storage solutions are also more sustainable, making efficient use of unused resources that already exist rather than building new data centers. By relying on existing resources and facilitating direct, peer-to-peer exchange without intermediaries, developers are likely to find that they’re substantially cheaper than their centralized counterparts. Furthermore, whereas centralized data storage generally requires signing up to tiered plans that invariably involve purchasing more storage than is needed, decentralized alternatives tend to offer a more flexible, on-demand service.

Decentralized storage of files shared between peers also changes the traditional client-server model used by centralized data storage providers. In the centralized model, typically, the user makes a request via a URL, and the server will deliver the file using HTTP. In the P2P storage model, each file, or fragment of a file, is assigned a cryptographic hash. When the user requests the data, the network searches for an exact match for the hash. This model allows faster retrieval of information while also decreasing network latency, making for a superior Web3 user experience.

Filecoin vs. Storj

Decentralized storage of files shared between peers also changes the traditional client-server model used by centralized data storage providers. In the centralized model, typically, the user makes a request via a URL, and the server will deliver the file using HTTP. In the P2P storage model, each file, or fragment of a file, is assigned a cryptographic hash. When the user requests the data, the network searches for an exact match for the hash. This model allows faster retrieval of information while also decreasing network latency, making for a superior Web3 user experience.

Protocol Labs developed the peer-to-peer file sharing protocol Interplanetary File System, better known as IPFS, in 2015. IPFS is now the protocol that drives Filecoin, the blockchain layer also developed by Protocol Labs. While IPFS provides the means for peers to share and store files, Filecoin provides the incentive. Filecoin launched on mainnet in 2020 and has proven popular among blockchain developers and open-source projects seeking decentralized storage.

For example, a copy of the entire Wikipedia database is now stored on Filecoin, providing a valuable backup in case the main servers go down. Similarly, Filecoin also keeps copies of the digital public domain works of Project Gutenberg, the oldest digital library in the world, offering 60,000 free eBooks.

The other major operator is Storj, a smaller rival to Filecoin. Storj uses its own encryption algorithm to encrypt files and break them into pieces before uploading them to the network. Storj also differs from Filecoin because it only offers storage services, whereas the IPFS protocol underpinning Filecoin also supports retrieval and distribution.

The other key difference is that whereas Storj offers a product designed to serve front-line developers directly, Filecoin is a more infrastructural product. While developers can also use Filecoin’s service, it’s also perfectly possible that a centralized provider like AWS could decide to supplement their server capacity with decentralized storage acquired using Filecoin and resell it.

As we’ve seen, decentralized data storage isn’t just a secure and robust alternative to centralized cloud providers. It’s a more efficient and sustainable answer to the question of how we store an ever-increasing volume of data. In many cases, decentralized storage also presents solutions that are also more cost-effective and user-friendly for developers, making them an optimal solution to further the development of Web3.