Editor’s note: this article is a repost from “Odaily” (official account ID: o-daily, Download App), a blockchain media outlet in strategic cooperation with 36 Kr
Although data in chains cannot be tampered with or backdated, the unknown storage terminals are untrusted by users who save their documents there in the decentralized storage environment. In this case, how is it possible to prevent malicious behaviors of untrusted nodes? In other words, how can we be sure that distributed storage terminals have actually accomplished their storage tasks?
In this regard, the Provable Data Possession (PDP) scheme for storage projects is particularly important. According to the scheme, users send data to the miners for storage, and the miners prove that they have stored the data. Users can double-check whether the miners are still storing their data.
Of course, there are various decentralized storage schemes on the market at present. In terms of PDP, we know that IPFS, Sia, Storj and other decentralized storage schemes all want to ensure the reliability of storage terminals by validating their PDP in an untrusted environment.
He Xiaoyang, founder of Lambda, a decentralized storage project, told Odaily that the existing two kinds of storage schemes all have their own disadvantages:
- IPFS/Filecoin, for example, provides only static file storage and cannot search file addresses through file content. Meanwhile, as to the PDP verification of storage terminals, they want the public to initiate the verification, but specify no specific initiator, nor how to initiate; that means it’s unclear who the data validator is.
- l The storage schemes represented by Sia and Storj intend to challenge storage terminals regularly through smart contracts, requiring them to provide feedback on the verification information of file fragments, hash values, etc. However, due to the limitation of chain ledger sizes, only the Root Hash of the Merkle Tree is stored in chains, which only ensures that the data cannot be tampered with, but cannot guarantee the possession and availability of the data.
Lambda plans to build a decentralized storage platform for enhancing the trust between users and storage terminals through the verification and consensus of distributed nodes. The Lambda platform sets up two network systems (like IPFS/Filecoin): the repository system and the block chain. The former is responsible for storage, while the latter is responsible for the access and control of the repository system. The two network systems are connected by subchains to realize interactions.
Unlike other decentralized storage projects, Lambda intends to provide storage terminals with PDP through the consensus generated by validator nodes, to ensure the integrity and recoverability of the data stored on untrusted storage terminals.
The approach provided by Lambda is to prove a storage terminal’s possession of file “F” in a Permissionless Store in the simplest scenario. The first step is to generate a collection of messages (m1, m2…, mn) of any file “F”, following the sharding of the original file that a user needs to store, and then calculate and cryptographically generate a number of tags based on the shards of the original file and four figures (i.e. the security parameter λ, or metadata).
The validator nodes store metadata for subsequent generation of challenges/puzzles; storage terminals store the original file shards and the corresponding tags which can help storage terminals accurately extract data from the original file shards and solve the puzzles.
The validator generates challenges/puzzles to challenge storage terminals, which then calculate based on the original file shards and tags in hand and generate a corresponding set of digital vectors, feeding them back to the validator. Finally, based on the metadata and vectors, the validator calculates and verifies whether the verification of storage terminals is successful. If it is, the transaction will be packed and embedded in the block chain; if not, the storage nodes will be punished.
When a single point possession of a file can be verified, the characteristics of the file such as dynamics, multiple copies, erasure codes, and deduplication can be supported by corresponding adjustments.
Taking such an approach, the validator needs to save the metadata, and storage terminals need to undertake extra tasks to store the tags, with the storage volume increasing only 1% to 3%. In the process of initiating verification against storage terminals, the validator’s challenge request will be made remotely because the two sides are in different network architectures. When a storage terminal does not accept the challenge or cannot be connected due to network problems, the validator can directly conclude that it fails to store data.
Throughout the system, the role of the validator is Lambda’s innovation point. Lambda also designed the roles of nominator and fisherman.
- Validator: The validator performs transaction packing and block generation across the entire Lambda network, therefore, it needs to pledge some Tokens. Instead of a single validator node, a random set of validator nodes are selected from hundreds of thousands of validator nodes to jointly verify a storage terminal, and the validator node is shifted for every 1024 blocks.
- Nominator: Nominators with capital recommend one or multiple validators to make decisions for them. They have no other function than capital investment.
- Fisherman: Fishermen are not correlated to the block packing process since their roles are similar to “bounty hunters” in the real world. Appealed by a one-time rich reward, the fishermen report the malicious behaviors of validator nodes through entrapment.
If data are maliciously deleted, it is especially necessary to restore the data file. In terms of the recoverability of data, the vision of Lambda is to distribute a file to B persons after the data file is divided into A parts (B < A). As long as the missing data are not greater than a certain preset value, they can be recovered. Since the preparation in 2017, the Lambda project has completed an angel round financing of tens of millions of RMB, with ZhenFund, Metropolis VC and Dfund as the lead-investors. According to He Xiaoyang, the test network of Lambda will go online early next year. Among the 15 team members of Lambda, the major members of the core founding team are from OneAPM, an APM SaaS company, and most team members are front-line programmers of basic software R&D and from open source communities.
Official website: http://www.lambda.im
🔹Telegram Community : https://t.me/HelloLambda
🔹White Paper : http://www.lambda.im/doc/Lambda-WhitePaper-en.pdf