Exploring Decentralized Storage: Is Permanent Storage Really Possible?

At the dawn of Web 3.0, the revolutionary decentralization powered by blockchain technology has brought profound impacts. In this revolution, data is seen as an important promoter of socio-economic productions and a valuable asset that can be circulated and inherited. The decentralized storage method is also receiving unprecedented attention.

Cutting-edge decentralized applications such as NFT, Metaverse have provided a significant pivot for decentralized storage. Based on the analysis of permanent storage, this article analyzes the similarities and differences between the decentralized storage project MEMO, Filecoin and Arweave from three aspects: storage technology, systems governance foundation, and performance scalability.

Permanent Storage

In the course of human history, many civilization records were accidentally or artificially destroyed. The arrival of the new era of scientific and technological development has perpetuated the incessant human endeavors to unlock permanent storage.

In 2019, GitHub launched a permanent code preservation program, which intends to last for at least a thousand years. GitHub has set up a storage method that incessantly stores multi-copy data across multiple formats and locations, and even in the form of film at a depth of 250 meters in the Arctic permafrost to resolve future data loss risks.

Compared with the Millennium Storage Plan launched by GitHub, decentralized storage attempts to supersede this complex storage method that spans multiple organizations and formats through blockchain and a series of technical mechanisms.

The vision of permanent storage is shared among pioneering decentralized storage solutions like Filecoin, Arweave, and MEMO. Filecoin’s vision is to “store important information of mankind”. Arweave proposes the vision of reviving the Alexandria Library. MEMO aims to “perpetuate human information and data”.

Among the three, Arweave seems to be the best fit for permanent storage, as data are directly stored on-chain using a one-time payment and long-term storage model. By comparison, Filecoin and MEMO adopt a tiered payment model based on storage time and space requests. However, Arweave is a completely open-access network that cannot be tampered with. This model is only suitable for storing public data, but not for private data. There is the possibility of long-term storage resources occupied by low-value data.

Because the spread of information is a natural selection process, low-value data will gradually disappear over time, while high-value data will be continuously passed on across the cycle independently. In a once-and-for-all method, permanent data storage is highly desired, resulting in data over appreciation and an odd definition of permanency measured merely by a 100-year human lifespan.

From the perspective of the hierarchical characteristics of data, permanent data storage is necessary for important data such as technology, history, and culture that influence the progress of human civilization. However, permanent storage is optional for the Internet of Things, smart driving and turnover data that play an important role in the current social and economical operations. The grand vision of permanent storage is not necessary for all data types.

The most salient interpretation of permanent storage is to ensure the safety and reliability of the current data stored, as every moment matters and adds up to permanence. Only by ensuring data safety and reliability at the current stage can the permanent storage be feasible. Therefore, the most reasonable way to organize storage is a tiered payment model, perpetuating data based on the choices of society and time.

Data safety and reliability requires several highly coordinated complex technologies simultaneously. Time-based evaluation criteria seem unrealistic in less than ten years since the birth of decentralized storage. Therefore, this article intends to assess three key metrics: storage technology, systems governance foundation, and expansion performance.

Storage Technology

When it comes to decentralized storage, blockchain is highlighted by most people, not the storage.

As an emerging technology in the last 20 years, blockchain has been highly commended for its decentralized and non-tamperable characteristics. Compared with the world-changing vision of blockchain, time-honored storage technology has a mild yet long-standing role in recording and passing on information.

Contrasting its low-profile, storage technology is never an easy option. Conversely, it is rather technology- and capital-intensive. Amazon, Google, Baidu, and other legacy players have monopolised the conventional cloud storage market, as barriers to technology and capital suffocates small businesses.

The synergy of storage and blockchain creates decentralized storage. For Filecoin, Arweave, and MEMO, decentralization is just a prefix added to the name, while the essence remains the storage and definitely nonnegligible storage technology.

The development of storage technology spans thousands of years. In recent decades, it has been evolved and digitalised. Traditional cloud storage options represented by AWS demonstrate high performance. However, it has been vastly critiqued for privacy concerns and server downtime. Once combined with frontier blockchain technology, conventional cloud storage service providers will witness a major technological breakthrough.

However, most decentralized storage projects currently use relatively primitive storage technologies. In terms of redundancy mechanisms, Filecoin and Arweave both utilize original multi-copy redundancy, while MEMO adopts a combination of erasure coding and multi-copy redundancy mechanisms.

Multi-copy has been a primitive redundancy strategy since the birth of computer storage. The principle is to generate a complete backup of each data copy on each corresponding node. For early small-volume data, multi-copy technology can meet the needs. However, the incremental amount of subsequent data challenges multi-copy technology on the storage space. Therefore, the erasure coding technology is invented and used by cloud storage giants to address large-capacity data storage problems.

Erasure coding technology slices data into small blocks, adds a certain redundancy check code, and stores data on different nodes. Compared with multi-copy technology, erasure coding significantly reduces the storage overhead. For example, for 1G data, 5-copy redundancy mode requires 5G storage overhead that can tolerate up to 4 nodes data loss, whereas, for the 5+5 erasure coding mode, it only requires 2G storage overhead for data loss up to 5 nodes, only 40% of the 5-copy mode for 5 times redundant.

MEMO substantially improves the storage space utilization by the dual redundancy mechanisms and provides users with greater autonomy. For low-frequency access, the default mode is erasure coding, while for high-frequency access, the multi-copy mode is employed. Apart from redundancy mechanisms, MEMO has also developed data recovery features. Its RAFI technology can rapidly identify failed data blocks and enhance efficiency multiple times.

Therefore, regarding redundancy mechanisms, Filecoin and Arweave combine original storage technology with blockchain, while MEMO combines cutting-edge storage technology with blockchain.

Systems Governance Foundations

The operating rules of the system play an important role in operating stability and durability. There are huge differences between Filecoin, Arweave and MEMO based on systems governance.

Considering economic models, Filecoin and Arweave both incentivize storage through block generation. Their user roles fall into two categories, namely storage users and miners. Although the Filecoin system has specified retrieval miner, the role of retrieval miner and storage miner can be performed concurrently. Therefore, there are essentially two types of system roles.

Considering operating principles, Filecoin adopts the Proof-of-Replication and the Proof-of-Spacetime as the basis of consensus as to the incentive layer of IPFS. Its operating mode is to encapsulate blocks; that is, only miners who successfully encapsulate data are eligible to obtain the rights of the block. The storage is highly correlated to the encapsulation success rate. Arweave employs the Proof of Access and PoW as incentives as a blockchain-like technology. Differing from traditional blockchains, it does not require every node to synchronize all the data on the chain. Miners can start verification immediately by downloading only part of the block.

MEMO does not incentivize storage by generating blocks. Instead, an interrelated and mutually restrictive triad of system role profiles is adopted to maintain the system balance. Smart contracts are also utilized for system autonomy. In addition to the User and the Provider, MEMO also designed the role of an intermediate manager, the Keeper. The primary function of the Keeper is to challenge the storage node of the Provider and verify the smart contract for the storage. The Keeper plays a vital role in the stable operation of the system.

Unlike Filecoin, the three roles of MEMO are independent and mutually exclusive. The User is a consumer, the Provider calculates revenue based on storage space and time, and the Keeper obtains a certain commission from the User’s payment. The transactions between roles are automatically executed through smart contracts.

Two roles in the Filecoin and Arweave system are like two dots in a straight line, and the three roles in the MEMO system form a triangle. In geometry, triangles are viewed as the most stable structure, insinuating that the MEMO system’s triad role structure showcases more operational stability.

On data integrity, MEMO has developed a public verification mechanism in which the Keeper assumes critical management and supervision responsibilities. Each node that obtains economic benefits must accept supervision. For example, each Provider must take the challenge from the Keeper, and each Keeper must accept the challenge from other Keepers in the loop. To prevent collisions and attacks between roles, this public verification mechanism also allows any third party to participate in verification apart from the Keeper, ensuring the openness, unpredictability, and randomness of verification.

In addition, MEMO has also designed a user rating system. Users have the right to evaluate the Provider and the Keeper that they have cooperated. Roles with high integrity scores will get more opportunities in the future. The rating system, the Keeper, and smart contracts establish the foundation of the MEMO system governance, improving the system stability.

Performance Scalability

At present, a large amount of data is stored on hard drives with a service life of several decades; the ultimate challenge for permanent storage is performance scalability. Regarding scalability, we evaluate the degree of decentralization by reliability, availability, and energy consumption.

Among three projects, Filecoin and Arweave both use block generation to incentivize storage, but competition for computing power has raised the bar for participation. Filecoin relies on encapsulation to generate blocks. The encapsulation process is a complex encoding and calculation with high technical requirements. Only professional equipment with high configuration and large computing power can participate. This restricts the access from home equipment due to limited computing power, which means that centralization is inevitable for Filecoin.

[Block generation incentive; smart contract transactions]

Arweave adopts a similar blockchain structure, Blockweave. Although it employs on-chain data storage, it does not back up data on the entire network, unlike traditional blockchain. Instead, miners randomly access recall blocks to incentivize storage. This incentive model will inevitably reduce the level of decentralization as data are not backed up on the entire network but only for historical and “rare” blocks.

The innovative design concept of the MEMO system guarantees complete decentralization that does not rely on computing power. MEMO has optimized and innovated the traditional blockchain storage architecture in terms of architecture design. It only records roles and transaction information on-chain and stores a vast amount of data on edge storage devices in a shared economy model. This model using ordinary edge devices does not only ensure complete decentralization but also significantly reduce the barriers to participation.

Moreover, carbon emissions will become one of the significant value propositions of decentralization in the new world order. The Filecoin’s encapsulation process and the Arweave’s PoW consensus process are highly energy-consuming. Although Filecoin stores data off-chain, its verification process is still completed on-chain. And Arweave operates both storage and verification on-chain. MEMO uses the public verification mechanism off-chain through random functions and shard verification mechanisms. This mechanism enables MEMO to perform with lower energy consumption and higher availability without compromising security.

Lower barriers enable MEMO to attain performance scalability. Its complete decentralization, low energy consumption, and high availability further add to its level of scalability. MEMO is also at the forefront of decentralized storage with leading storage technology regarding data reliability. MEMO utilizes several innovative and unique technologies to fulfill the promise of data integrity at every moment.

The genesis of the blockchain catalyzes the separation of computing and storage. The deployment of smart contracts and the design of the Keeper role can separate incentives from the block generation. These differentiations have become the cornerstone of the MEMO concept, a completely decentralized, low-redundant, and low-energy option for cloud storage with high reliability and high availability.