Centralized network operation and data storage methods cannot ensure data security and privacy. Therefore, the need for data security and privacy fuels the development of decentralization projects. In 1993, the concept of decentralization first appeared in Cyberpunk Manifesto. In 2008, the emergence of Bitcoin speeded up the pace of development for decentralized projects. However, it is only in the recent decade that decentralized storage projects have been implemented.
The pioneer of decentralized storage is the InterPlanetary File System (IPFS), developed by Protocol Labs in 2014. MEFS(MEmo File System) is a distributed cloud storage file system developed by Memo Labs in 2017. Both methods are decentralized storage. However, MEFS and IPFS vary significantly in technology, performance, user experience, and vision. This article will provide a comprehensive overview of differences regarding the working mechanism, technical specifications, and storage performance by analysis and comparison.
The Working Mechanism
The design philosophy of MEFS is to use blockchain and smart contracts to build a decentralized storage network by connecting massive edge storage space worldwide. The system devises a data layering mechanism to store critical data (e.g. smart contracts and role portfolio) on blockchain and non-critical data on edge storage nodes. The system involves three roles: the User, the Provider, and the Keeper. Anyone who has spare storage space can register themselves as the Provider. The Keeper matches and verifies storage information between the Provider and the User. The User on MEFS can choose the number of nodes for data storage. The data will be encrypted, redundant, segmented, and stored on different nodes. Once needed, the User must access data from scattered nodes. The core technologies of MEFS include a public verification mechanism and a data recovery mechanism. The public verification mechanism ensures data security and payment, and the RAFI technology provides rapid data recovery. The system also encourages and restricts node activities through reward and penalty measures to maintain the ecosystem.
IPFS constantly proclaims to be a benchmark of HTTP. Its most distinctive feature is content addressing and hash deduplication. When a file is added to IPFS, the file will be segmented into smaller blocks. Each block will generate a unique hash value. Each file will get a unique Content Identifier (CID). In search mode, nodes will query peer nodes referencing the unique CID of these files. Nodes will create a cached file copy in view or download mode and become the content provider until caches are cleared. Each node in the network only stores what it concerns. The IPFS system will generate a different hash value to encrypt the file and, hence, a new CID for adding a new file version. The IPFS system can resist data tampering and censorship. The IPFS system can reuse data blocks to minimize bandwidth and traffic. The underlying technologies in the IPFS include distributed hash table, block exchange protocol, version control, and self-verification file system.
Storage principle (MEFS vs IPFS)
Redundancy and Incentive Mechanisms
1、Redundancy Mechanisms
Modern cloud storage technologies will design corresponding redundancy mechanisms to improve reliability. The most common technologies use multiple copies and erasure codes. Multiple copies are duplicates of the entire file stored in different nodes, and erasure codes refer to data partitioning. Part of the data block is made redundant, and then the redundant data block is expanded, coded, and stored in different nodes.
MEFS redundancy mechanism: MEFS adopts a multi-level fault-tolerant mechanism design. Smart contracts, metadata, and other small volume and high access frequency data use multiple-copy redundancy technology, while large volume and low access frequency data adopt erasure coding technology by default. Erasure coding technology can improve data access and enhance system storage efficiency.
IPFS redundancy mechanism: IPFS does not have a redundancy mechanism. IPFS segments data to transfer and de-duplicate hash, not dispersing data on different nodes since the segmented data is still stored locally. Only in access or download mode, the system will create a duplicate of the entire data. Also, each node in the IPFS network only stores data it concerns, not bounded by permanent storage obligations. In many cases, uploaded data are only stored locally.
2、Verification Mechanisms
In a decentralized network storage system, verification is a crucial technology that determines storage data integrity.
MEFS verification mechanism: MEFS adopts a public verification mechanism. The system assigns a number to each user data segment and generates a verification code for each segment. It then sends the data segment and verification code to the storage node. Only verification code for the segment (e.g. Segment №100–200) is required when it comes to data verification. The storage node will read the corresponding segment and code, and generate a certificate of several hundred bytes to verify the correct storage. Combined with the probability spot check, hundred-byte proof can verify terabyte data.
IPFS verification mechanism: IPFS aims to build a decentralized network but lacks a storage verification mechanism. The user uploads data to the IPFS system and gets a hash table randomly uploaded to neighboring nodes. The original data are stored locally. The user does not know and cannot verify how many nodes are involved in the storage and caching. A self-verification system (SFS) is designed to prevent tampering. However, the SFS is more like encryption than storage verification, as correct keys are required to access, modify, delete or tamper data except for specified anonymous access. The purpose and working mechanism differentiate SFS from MEFS.
3、Recovery Mechanism
The recovery mechanism refers to the self-repair process following data damage. The recovery mechanism and the redundancy mechanism are closely linked. Multiple-copy redundancy repairs lost data by downloading data copies stored on other nodes, whereas erasure code repairs with redundant data segments.
MEFS recovery mechanism: MEFS adopts two redundancy mechanisms, multiple copies and erasure codes. The system also features an original RAFI technology that rapidly identifies invalid data to repair data. As a core technology, RAFI can significantly enhance system reliability.
IPFS recovery mechanism: IPFS does not have a redundant mechanism by design. Copies are disseminated depending on the data concerned by other nodes. The wider the data disseminate, the more copies are. For low-access data, there is no copies generated if there is no access. In case of data errors and damage on local nodes, data loss will be permanent. IPFS data repair mechanism is thus imperfect.
4、 Incentive Mechanism
Incentive mechanisms are technologies and measures that promote safe and long-term data storage nodes. MEFS has a comprehensive incentive mechanism, while IPFS relies more on “community autonomy.”
MEFS incentive mechanism: MEFS incorporates three interactive roles, the User, the Provider, and the Keeper. Smart contracts information is recorded on the blockchain. The Provider provides storage space and charges users on the storage space and duration. The Keeper profits from coordinating and managing services. The system enables a rating and reward-penalty system. High-rating providers and keepers will get more matches, and the system will ban low-rating ones. MEFS also encompasses a series of measures to enhance credibility. For example, a deposit is required during registration as the Provider and Keeper. The system will deduct from the deposit as penalties.
IPFS incentive mechanism: IPFS is a low-layer open network protocol. It mirrors rewards and penalty measures on BT protocol. Uploaders will receive reward points, and long-term inactive nodes will be deducted points. However, the IPFS incentive mechanism only looks at data upload, not secure storage. So the incentive mechanism for data storage is missing on IPFS. There is only one user role in the IPFS system. Like many ordinary users on the network client side, each node user is equal, and there is no division of labor and boundary of obligation. Every node is free. This “freedom” means that no node is obligated to store data. They can freely access data at will and delete data at will without any restriction. In other words, there is no reward for long-term data storage or no penalty for damage or deletion. IPFS is thus more suitable for a file transfer protocol, not a complete decentralized storage system considering its incentive mechanism.
Figure 1: MEFS vs IPFS Technological and Incentive Mechanisms
Features and Fees
1、Data retrieval and access
Retrieval is the technology and mechanism for searching data in the system. In the MEFS system, retrieval is mainly based on searching metadata stored on the Keeper nodes available for other nodes. But anyone can retrieve data as long as they can access metadata. Meantime, anyone can access data but not decrypt as data segments are encrypted and stored in different nodes. Therefore, anyone can access data but cannot decrypt data.
In the IPFS system, the content addressing mechanism makes it easier to retrieve data. Each file generates a unique hash value (a long string of digital codes). As long as the hash value is retrievable, the title, content, and location of the file can be obtained. Therefore, data retrieval is a highlight and competitive edge against HTTP, as hash-value content addressing is highly accurate and rapid. IPFS envisions an open network that allows users to query and access data with a matching hash value. For unencrypted data, anyone can access it. However, each file in the IPFS system has a unique matching hash value. Even with a valid hash value, lost or deleted data cannot be accessed.
2、Data upload and download
In MEFS, the system processes uploaded data by segmenting data and storing these segments in various nodes. The system downloads data by accessing data segments from scattered nodes and piecing data together.
In the IPFS system, data are uploaded rapidly from local data storage. Access and download are parallel. Download starts when accessing data. As long as any node accesses data on a particular node, it will automatically cache a copy on its node. IPFS uses the “block exchange protocol” to access and download faster. The protocol draws on the BT network technology, downloading data segments simultaneously from nearby nodes. The more the storage nodes, the faster the transmission speed. As mentioned earlier, IPFS segments data to make transmission faster, not for decentralized storage. Therefore, the IPFS system showcases relatively better data access, upload, and download performance.
3、Fees and Expenses
MEFS adopts the concept of sharing economy. It aims to link global edge storage devices to provide storage services for those in need. The system charges for services. End-users need to pay for data storage and download. The Provider profits from renting spare storage space, while the Keeper profits from managing storage. Since most facilities are idle infrastructures, repetitive work is reduced. Hence,the price for MEFS storage is relatively low.
In the IPFS system, there is only one user role, and data storage and download are free. Data uploading is essentially local data storage, storage on other nodes depends on the expression of interest and system mechanisms. Without incentives and restraints, storage nodes can delete data at will, and the system will periodically delete less frequently used data. Therefore, the free IPFS system cannot ensure permanent data storage.
Figure 2: MEFS vs IPFS Features and Fees Comparison
Comparison and Analysis
Having compared differences in working principle, technical mechanisms, and performances between MEFS and IPFS, we can summarize that the two storage systems are not comparable as underlying technical framework and visions significantly differ.
1、Different positioning: IPFS focuses on data sharing and dissemination, while MEFS on secure data storage
The vision of IPFS is to create a more open decentralized network. Although many projects use it as the underlying storage network, its goal is to replace HTTP as the next-generation Internet protocol. Drawing upon BT network’s data segment transfer and “block exchange protocol”, data retrieval, access, upload, and download can speed up. Therefore, the core advantage of IPFS is content addressing and fast transmission. This advantage makes data sharing and dissemination easier in line with its vision.
The goal of the MEFS system is secure data storage and data autonomy. MEFS uses decentralized blockchain technology and links global massive edge storage space to improve system scalability. Security is at the center of everything. Safety, reliability and availability are fully considered in the system design. MEFS innovates data layering and redundancy mechanisms and develops a public verification mechanism and RAFI data recovery technology. All these innovation efforts make storage more secure. Therefore, the core advantage of MEFS is high security and reliability in data storage.
Therefore, in terms of positioning, IPFS and MEFS have different focuses: one is open transmission networks, the other is secure data-storage technology.
2、Different security levels: IPFS only uses encryption technology against data tampering, MEFS demonstrate a higher level of security.
In the IPFS system, unique mapping hash value and self-verification system (SFS) are used to ensure data security. This technology can prevent data tampering, but not deletion or loss.
In the MEFS system, MEFS relies on blockchain and ensures data safety and reliability by using a series of technologies (e.g. encryption, redundancy, verification, and recovery mechanisms) that IPFS does not have. Therefore, MEFS has a higher security level.
3、Different operating mechanisms: For a high-quality storage project, IPFS lacks decentralization and incentives, whereas MEFS has a complete decentralization and reward-penalty mechanism.
On data security and reliability, decentralization is the go-to option. IPFS features an open network that focuses on data upload, sharing, and transmission. However, it lacks decentralization and incentive measures. For example, the system will periodically delete low-frequency access data without prior notice. Mis-deleting essential data can cause adverse effects. This system is too powerful and centralized. Also, IPFS nodes store data at will without any incentive mechanism so that each node can delete data without restrictions. For a high-quality storage project, IPFS lacks security and reliability without incentive layers. For instance, Filecoin is a decentralized storage project with the IPFS as an incentive layer.
MEFS demonstrates its decentralization by scattering data storage on multiple different edge storage devices. No one on the MEFS system but the User can modify or delete data, adding new data and copies to the system while uploading. IPFS focuses on data upload and access, whereas MEFS on data storage (the Provider) and verification (the Keeper). The Provider stores data for the User, and the Keeper evaluates storage requests among various available options and verifies data integrity. The system sustains its check and balance on the role portfolio. Complete decentralization and appropriate incentives, rewards, and punishments qualify MEFS for a sophisticated storage option.
4、Different application scenarios: IPFS is more suitable for storing public data, whereas MEFS for diversified data.
The parallel node access and download mechanism greatly benefit popular public data. Only popular data are likely to generate more node accesses and copies for permanent storage. However, encrypted or unpopular data can only be stored locally due to fewer available nodes for access and caching. The security level of encrypted or unpopular data is relatively low. Therefore, IPFS is more suitable for storing publicdata but not for unpopular or encrypted private data.
In the MEFS system, regardless of data type, only the owner can modify data. The User has the autonomy on data redundancy and storage duration, aided by a series of technologies and operating mechanisms to ensure data security. Therefore, MEFS can adapt to multi-scenario data storage regardless of its positioning, security level, or operating mechanism. It is suitable for storing diversified data such as NFT, Dapps, institutional, and personal privacy data.
