Data Availability and Integrity

Almost every Internet user relies on centralized services to store and process data. This has two major shortcomings: users are no longer in full control of their data, and it is hard to verify the integrity of the data. To address these shortcomings, a new generation of protocols and peer-to-peer networks have enabled decentralized storage and cloud services. One prominent protocol of this kind is Filecoin, which provides a cryptoeconomic guarantee that the data exists and can be accessed at any time.

Put simply, Filecoin is a peer-to-peer version of AWS that regularly verifies data availability and integrity. This is made possible by deals negotiated on open markets for data services.

Deals in Open Markets

A storage deal is like a service level agreement (SLA) enforced on the Filecoin blockchain. Storage deals are agreements where users pay fees to storage providers to store their data. The terms of the deal include the fee amount and the duration of the storage deal.

The open-market environment incentivizes storage providers to offer their storage capacity in an efficiently priced way. That is, prices are determined by supply and demand dynamics, rather than a fixed pricing structure. To incentivize storage providers to participate in deals, Filecoin rewards them with the network's native token (FIL).

Filecoin has two functions to keep data safe. First, it uses a cryptoeconomic guarantee that verifies the storage with zero-knowledge proofs. Second, it uses an economic penalty: if the data is unretrievable or the storage fails, storage providers are penalized through locked-up collateral. To enforce this, the Filecoin protocol requires storage providers to submit two ongoing verifiable cryptographic proofs. Proof of Replication first verifies that the data has been fully replicated. Upon success of this initial check, storage providers use Proof of Spacetime every day to prove that a random piece of data is still stored and available for retrieval throughout the deal’s lifetime. Any failure leads to a penalty.

To retrieve data, users pay a retrieval provider to fetch the data for them. Unlike storage deals, which are handled on-chain, retrieval deals may happen off-chain. This results in a faster data retrieval process.

Technology in a Nutshell

Filecoin is built on top of the InterPlanetary File System (IPFS), which serves as the Filecoin network’s distributed data storage and sharing layer. Although some people use Filecoin and IPFS interchangeably, there are important differences between the two.

Put simply, IPFS is like torrents on steroids. Similar to torrent trackers, IPFS uses a fingerprint of the data. Unlike torrents, IPFS leverages a single global peer-to-peer network. IPFS essentially tells users where their data is, but it lacks a built-in mechanism to incentivize data storage, which led to the development of Filecoin in the first place. Filecoin thus complements IPFS and facilitates an open market for storage with a guarantee, essentially making Filecoin the incentive layer of IPFS.

IPFS may be regarded as a peer-to-peer version of HTTP, with an important twist. Traditional URLs and file paths identify a file by its specific geo-location on a server. IPFS uses content addressing to store data decentrally. It references a file by what’s in it, i.e., by its content. A content address is unique and is the result of hashing, which is essentially the cryptographic compression of content into a long string of characters. When retrieving a file, the network only needs the file’s hash to locate the nodes storing that file’s content.

Network Dynamics

Thanks to its open-market policy and incentive mechanisms, Filecoin has been able to attract significant storage capacity. Regardless of market conditions, Filecoin’s committed storage capacity has grown steadily since 2020 to over 16,000 PiB, which is equivalent to approximately 65,000 copies of Wikipedia or 1,600 Netflix movie archives as of March 2022. The network capacity comes from over 3,900 decentralized storage providers distributed around the globe.

What’s the Demand for Filecoin?

Demand for Filecoin comes from both Web2- and Web3-specific storage use cases. Adding up how much storage capacity is bound in deals quantifies the demand for Filecoin. However, when analyzing the underlying content in the storage deals, it is difficult to distinguish between useful storage and generated randomness as a way for storage providers to abuse the network.

To increase the amount of useful storage capacity on Filecoin, the Filecoin Plus incentive program was developed. The program incentivizes storage providers to participate in verified deals by increasing their share of the block rewards over time.

Storage deal inflows grew from 400 TiB in October 2020 to over 7,600 TiB in March 2022, and Filecoin Plus clearly contributed to that growth. Since Q4 2021, there has been a transition from Filecoin Regular deal inflows (unverified data) to predominantly Filecoin Plus verified deal inflows.

Used storage on the network (i.e., active storage deals) has grown steadily from 0.25 PiB (1 PiB = 1024 TiB) in October 2020 to 45 PiB in March 2022. As shown below, over 26 PiB have resulted from Filecoin Plus deals, representing 57% of the total active deals on the network.

The recent flippening of Filecoin Regular deals by Filecoin Plus deals indicates that the verified data incentive mechanism is working as planned. The sudden drop-off in terms of active Filecoin Regular deals in December 2021 was a result of several storage providers going offline. The loss has been compensated by other storage providers picking up and contributing to the growth in Filecoin Plus active deals as of January 2022. As a result, the total deal volume fully recovered.

While supply vastly exceeds demand, storage capacity is only one side of the story. What is equally important is the amount of used storage on a network. With 45 PiB of used storage and 16,000 PiB of storage capacity, Filecoin has a storage utilization ratio of 0.3%. To put these network metrics into perspective, competing decentralized storage providers, like Sia and Storj, have 6 PiB and 14 PiB, respectively, of network capacity. Total storage usage by Sia and Storj stands at 2.2 PiB and 8.6 PiB, respectively. In terms of storage utilization ratio, Sia and Storj surpass Filecoin at 36% and 60%, respectively. Although Arweave has 0.05 PiB of data stored, it has not been included since its data stored is negligible compared to the other storage networks: while it is currently growing from a relatively small base, it does not have network capacity per se.

While the introduction of the Filecoin Plus program is a step forward, there is a concerted effort to incubate new businesses and use cases for building on Filecoin. The Filecoin ecosystem is channeled towards onboarding a large variety of use cases: from decentralized storage for NFTs, to music and video streaming, to metaverse and gaming.

Ecosystem of Developers and Builders

Filecoin is actively supported by several organizations with the common goal of growing the Filecoin ecosystem. These organizations come together to form the Ecosystem Working Group. The responsibilities of the Ecosystem Working Group participants include maintaining and upgrading the underlying protocols, supporting the growth of the ecosystem, and facilitating governance. The most prominent organizations in the group include Protocol Labs, Filecoin Foundation, and Outercore. An overview of the main participating organizations is shown below.

These organizations have been actively developing a funnel of developers and builders through activities such as hackathons, accelerators, grants, mentorship and growth support. The funnel is designed to help early-stage projects and teams get to the point where they are ready to receive funding and investments. As a result, over 330 projects are currently being built on Filecoin and IPFS. Below is a breakdown of the funnel, showing projects’ progression over time, grouped by their maturity stage.

The main objective for 2022 is to enable as many early-stage projects in the ecosystem to translate into full-fledged businesses -- i.e., advancing from initial stages all the way through Series D and beyond.

As a whole, the ecosystem has grown significantly in terms of integrations with applications and clients, tools for developers, and infrastructure. Recently, the Ecosystem Working Group has prioritized making it easier for end-users and developers to work with Filecoin. This has led to the development of services like Textile Powergate, Estuary, and NFT.Storage. All use cases in the Filecoin ecosystem are available here. A map of selected ecosystem players is shown below.

Source: Filecoin

In terms of notable initiatives in 2022, the forthcoming launch of the Filecoin Virtual Machine (FVM) is worth highlighting. FVM is set to unlock programmable storage primitives (e.g., storage bounties, auctions), cross-chain interoperability bridges (e.g., to Ethereum, Solana, NEAR), data-centric DAOs, as well as Layer-2 solutions (such as reputation systems, data availability sampling, computation fabrics, and incentive-aligned Content Delivery Networks).

Main Use Cases Leveraging Filecoin

The Filecoin ecosystem is large and rapidly expanding, positioning itself to become the backbone of the decentralized web. With the rising demand for cloud storage, Filecoin provides a decentralized alternative to the popular centralized platforms. Numerous additional use cases have emerged, and new ones are guaranteed to materialize in the near future. Four major categories have already found high demand: (1) NFT and Web3 storage, (2) permanent storage and Web2 datasets, (3) metaverse and gaming, and (4) audio and video.

NFT Storage

During the summer of 2021, NFTs entered the mainstream spotlight. Driven by network effects, NFTs dominated the crypto landscape in the form of social media profile pictures, fine art, collectibles, music, and more. Interestingly, a large number of NFTs generally point to off-chain storage locations for their metadata since storing it on-chain isn’t feasible. Shortly after the 2021 hype, it became evident that a large portion of NFT metadata was not being stored in a decentralized, immutable manner, putting the content and value of the NFTs at risk. To address this shortcoming, NFT.Storage was developed.

NFT.Storage utilizes Filecoin and IPFS to provide a simple decentralized storage solution specifically for NFT content and metadata. It focuses on perpetual redundant storage through dedicated IPFS servers managed by NFT.Storage and Filecoin. By participating in the Filecoin Plus incentive program, NFT.Storage is able to offer an unlimited amount of storage for NFTs, free of charge.

Launched in April 2021, NFT.Storage has already supported the permanence of over 40 million uploads of NFT data, totaling up to more than 260 TB of data. The simplicity of the service has allowed a wide range of users from individual artists to large marketplaces such as OpenSea, MakersPlace, MagicEden, Holaplex, Jigstack, and Galaxy to take advantage of perpetual NFT storage.

Web3 Storage

Filecoin aims to become the storage layer of Web3. To that end, Filecoin has ongoing collaborations and storage bridges with various major blockchains and smart contract platforms. One of the most recent examples is Polygon. The goal of the partnership is to meet the storage and infrastructure needs of developers for games and NFT-related Web3 projects. Notably, Polygon developers can easily use Filecoin through the storage bridge. Filecoin has also seen growing traction in the Solana ecosystem with a number of protocols and NFT marketplaces offering native support, including Metaplex, Magic Eden, and Holaplex.

To simplify building Web3 apps on Filecoin, the team launched Web3.Storage. It allows developers to store and retrieve data directly on Filecoin instead of dealing with complex decentralized storage networks.

Web3.Storage uses redundancy to store data across multiple Filecoin miners and geographically-distributed nodes on IPFS. These IPFS nodes are run by the Web3.Storage team. In addition, the service offers free storage indefinitely with unlimited data. This is made possible by the economics of the Filecoin Plus mechanism. Overall, Web3.Storage has experienced steady growth since its launch, growing from 3 million uploads in August 2021 to 16 million in March 2022.

Permanent Storage & Web2 Datasets

Similar to Arweave, Lighthouse allows Filecoin users to permanently store files for a fixed price. In addition, the Filecoin Virtual Machine will enable permanent storage to be automated into smart contracts. A prominent use case for permanent storage is data archival. Given Filecoin’s scale, the goal is to make archival storage as cost-effective as possible — essentially becoming a commodity for developers building apps on top of Filecoin. This enables the information layer for humanity to be created and preserved.

The Shoah Foundation is an essential project that archives testimonials from genocide survivors. When it comes to large public datasets, the Internet Archive project stores websites and documents for public access using Filecoin. In addition, Zarr and GainForest store big public datasets with the goal of preventing natural catastrophes related to climate change. The Zarr project makes climate data available to researchers for analysis at scale. GainForest leverages satellite images to combat the effects of climate change.

Metaverse and Gaming

There is a new generation of metaverse and gaming use cases leveraging Filecoin and IPFS. The underlying common aspect of these use cases is user monetization. In this sense, Blockbets and Gala Games support revenue generation and payouts for gamers. When it comes to artist monetization, Mona supports artists displaying their high-end creations in virtual spaces and selling them to collectors. Metaverse AI takes human interactions to the next level: it allows gamers to play and monetize their interactions with other AI-powered virtual humans inside the game.

Audio and Video

Recently, several audio and video applications leveraging Filecoin and IPFS have gained substantial traction. Take, for instance, the Web3 alternatives to Spotify or SoundCloud. Audius is a music streaming service that looks and feels like SoundCloud, but leverages IPFS. Currents.fm and Inflow Music support communities of music creators and curators. Huddle01 could be the next Zoom. It’s a low-bandwidth video conferencing platform tailor-made for remote meetings, with a focus on NFT communities.

Regular users may not be able to tell that Audius, Currents.fm, Inflow Music, or Huddle01 are decentralized or that they specifically leverage Filecoin and IPFS. Nonetheless, there is an ongoing shift towards Web3 streaming applications without users necessarily noticing it.

As the Filecoin ecosystem matures, more use cases are constantly being discovered. The development of the Filecoin Virtual Machine will unlock previously impossible use cases, creating a whole new class of use cases.

Conclusion

While decentralized storage is still in its early days, the Filecoin ecosystem has been thriving over the years. Filecoin and IPFS have been leveraged by a wide range of use cases, including NFTs, Web3, gaming, metaverse, and audio/video. Alongside Filecoin, alternatives like Arweave, Storj, and Sia could use the foundation of IPFS at scale.

The adoption of decentralized storage solutions and the integration of smart contracts built on top may open the door to novel applications. A significant milestone would be getting major browsers to adopt IPFS. Open data initiatives and decentralized compute may enable the next generation of applications beyond storage.

The main challenge for Filecoin is to prove its reliability as a storage provider and as an enabler of data-intensive Web3 services. Filecoin and IPFS operate at the cutting edge of the Web3 paradigm and offer an essential foundation for Web3 development. Their success depends on the Web3 applications building on those protocols.