Lambda, a decentralized infrastructure startup that provides unlimited data storage via the blockchain, today announced its a recipient of financial investment from Bitmain Technologies Ltd., the world’s largest producer of application specific integrated circuit miners (ASIC) for cryptocurrency mining, to support secure blockchain-based infrastructure and decentralized applications (DApps).
Responsible for creating leading consumer-grade, power-efficient bitcoin and cryptocurrency miners based on 16nm ASIC chips, Bitmain produces a large number of high quality, efficient computing chips, high density server equipment, and large-scale parallel computing software. The company also operates the largest cryptocurrency mining pools worldwide—Antpool.com, BTC.com and connectBTC.com—as well as hashnest.com, the leading cloud mining platform.
“Bitmain has demonstrated its commitment to expanding strong, DApp blockchain companies such as Lambda,” said Xiaoyang He, CEO of Lambda. “This investment from a global industry leader is a significant endorsement and recognition of Lambda’s longstanding dedication in creating a world-class blockchain-based storage solution.”
“Storage is an interesting development area in blockchain infrastructure and Lambda’s technology vision, pragmatic roadmap, and progress to-date have been key factors for us,” said a spokesperson at Bitmain. “We look forward to working with Lambda.”
Lambda is a fast, safe, and scalable blockchain infrastructure project, which provides decentralized applications (DAPPs), data storage capabilities with unlimited scalability and fulfills services such as multi-chain data co-storage, cross-chain data management, data privacy protection, provable data possession (PDP), and distributed intelligent computing through logic decoupling and independent implementation of Lambda Chain and Lambda DB. For more information, please visit http://www.lambda.im.
Bitmain—founded in 2013 and headquartered in Beijing—transforms computing by building industry-defining technology in cryptocurrency, blockchain, and artificial intelligence (AI). Bitmain leads the global blockchain industry in the production of integrated circuits for cryptocurrency mining, as well as mining hardware under the Antminer brand. The company also operates the largest cryptocurrency mining pools worldwide—Antpool.com, BTC.com and connectBTC.com—as well as hashnest.com, a leading cloud mining platform. Bitmain is headquartered in Beijing with offices throughout China, including Hong Kong, and with global offices worldwide. For more information, please visit https://www.bitmain.com/.
In recent years, the development of blockchain technology has advanced faster than its practical applications. The balance of EOS performance and decentralization has led us to see the possibility of bridging this gap. The EOS Beijing team is composed of experienced digital currency professionals and investors who are eager to participate in this new cutting-edge technology.
Lambda Bridge Pool will collaborate with university research programs and various blockchain community groups, to support EOS-related academic research and promote EOS awareness and adoption. With the support and cooperation of wallets, cryptocurrency media, and other elements of the ecosystem, we will be capable of making significant contributions to the development of EOS.
Location of servers: China, Japan, Singapore (multiple locations)
Type of servers: cloud and bare-metal servers
Node name: eoslambda
Technical Scaling Plan
Before October 5
CPU: Intel Xeon 64 core
RAM: 256 Gb
Disk : 4000 Gb SSD
Network: 200 Mbps Bgp multi-line
CPU: Dual Intel Xeon 32 core
RAM: 256 Gb
Disk: 500Gb hard disk + 10 Tb 7200 hard disk (dynamic increase when needed)
Network: 200 Mbps Bgp multi-line
After October 5
CPU: Intel Xeon 64 core
RAM: 1024 Gb
Disk: 4000 Gb SSD
Network: 200 Mbps Bgp multi-line
CPU: Dual Intel Xeon 32 core
RAM: 1024 Gb
Disk: 500Gb hard disk + 10 Tb 7200 hard disk (dynamic increase when needed)
Network: 200 Mbps Bgp multi-line
CPU: Dual Intel Xeon 32 core
RAM: 1024 Gb
Disk: 512 Gb SSD
Network: 200 Mbps Bgp multi-line
Our core team consists of serial entrepreneur in infrastructure software, and open source community contributors, and seasoned digital marketing professionals.
Our team members mostly are from OneAPM, the business feature of OneAPM is application performance monitoring. Therefore, there will be massive data sources converging from various mobile terminals, browsers, and servers to the OneAPM server in 7X24 hours. The server needs to support the real-time writing, calculation, and query of massive data. At peak time, the OneAPM SaaS system needs to process 100 billion pieces of data every day.
In order to process these data, we built an analytical database druid cluster which is at the forefront in the world. This cluster deeply rewrites the Clickhouse database, creates a Chinese druid user group, compiles a number of technical books, and promotes practical experience of real-time analysis of massive data throughout China.
Since 2017, we have begun to create a completely decentralized and highly available database software using the open source community based on our own experience and business characteristics. In the process we have gained great support from Apache foundation, Akka community, Druid Community, and the clickhouse team, as well as three well-known soft companies in China: OneAPM, YiFang Cloud ( which is positioned as the Dropbox of China ), and Sequoia DB ( which aspired to be the Oracle in China ), have also joined in as partners, all of which form today’s Lambda project.
The background of the establishment of Lambda Bridge Pool
With the end of the EOS crowdsale the EOS community’s attention has shifted from acquiring tokens to launching the mainnet, maintaining and developing the EOS ecosystem.
Block one and BM, for a variety of reasons, took a decentralized attitude and gave the community more power to plan, launch and operate the EOS Mainnet. This has provided a level of uncertainty initially but a fantastic opportunity for members of the community to form block producers that share EOS values, have technical know how and resource strength to launch and maintain the mainnet collectively. Those who have community support and the recognition of the public have the most responsibilities as active BP’s. Block Producers are a vital part of the EOS community and ecosystem.
Lambda Bridge Pool core members have long been focused on the block chain industry and have built a wealth of industry experience and resources. From our perspective, the blockchain industry after 10 years of development is slowly transitioning from being solely concerned about use for digital currency transactions, ICOs and speculation to using blockchain technology to provide true and viable commercial applications. EOS is the key to this great shift and we expect it to provide the real world use that other blockchains promised
Based on this belief we initiated the establishment of Lambda Bridge Pool. We could not miss out on the opportunity to be part of this blockchain experiment and its implementation of the values we share. We also knew that we have the experience and resources to contribute to the EOS ecosystem.
2.The core values of Lambda Bridge Pool
Lambda Bridge Pool follows the following core values:
(1) Open and inclusive.
We have our own independent values to judge, but we do not want an ideological war. We uphold the concept of community values cohesion, the choice of like-minded members and communities to work together to win.
We maintain open communication and dialectical discussion with other communities, even with differing values, because the spirit of EOS is open source autonomy, and even the group that BM thinks values are inconsistent can open new chains and compete.
We believe that true justice and value stand out in the open and inclusive, and EOS will grow in the debate, collision and integration of different values.
(2) service, diligence.
We believe the BP is not the ruler of the EOS kingdom but the night watchman. So our positioning is very clear to serve the EOS ecosystem, the EOS community, the bearer, and the EOS user.
We will never be above the community and the bearer, always reminding ourselves of our service identity and being humble and vigilant.
We feel our responsibility as a BP is to uphold the principle of due diligence, maintain a high sense of responsibility for the EOS block production, community management, ecological construction and continue to contribute to the strength of EOS.
(3) Professional and impartial.
Lambda Bridge Pool will maintain a high level of focus on EOS technology, ecology, community, and DApp, maintain and enhance professionalism in the fields of technical operation, legal governance, ecological philosophy, commercial applications, and maintain the professional image and credibility of Block Producers.
At the same time, in view of the community on bribery, arbitration, governance and other issues of concern and talk about, we will always maintain independence and impartiality, we will not abuse the power of BP, we will actively monitor the behavior of other BPs to maintain a professional and impartial BP community image.
3.Ownership of Lambda Bridge Pool
Lambda Bridge Pool is initiated and owned by independent individuals and institutions.
Lambda Bridge Pool is currently not registered as a commercial entity in China on the grounds that:
(1) The operation of BP does not currently require this.
(2) Meet the block chain and EOS distribution of autonomous spirit of conformity
(3) BP’s hardware and software costs, at present by the founders voluntarily at their own expense, no need commercial financing and other acts therefor reducing the possibilities to be controlled.
We believe that EOS will be a long-term development process, thus taking a regular approach similar to the traditional shareholding lock to ensure that the Lambda Bridge Pool all people can focus on the cause for a long time. Ownership of all members will be unlocked in 4 years.
The first year unlocks 1/4, subsequent monthly unlocks the remaining part of the 1/36.
5.The expectation of EOS is that it will be the first large-scale trans-regional, cross-language, trans-ethnic, and trans-social system of community autonomy experiments in human history. The absolute decentralization and openness contained in the governance style of the Soul man BM was never seen before. We are honored to be part of this experiment and to play the important role of BP. We will do everything that we can do to promote the development of EOS to free, non-violent, ethical and autonomous Communities.
We hope that EOS not only will realize large-scale commercial application, positively transform society, but also provide open development and use by non-violent communities and change the governance structure of human society.
On June 22, in the “Ten Questions with Vitalik Buterin and Wang Feng” Vitalik talks about Casper and Sharding technology from Ethereum’s future to the industry landing scene, and then about programming language evolution and also shares his thought on the common problems faced by the entire blockchain industry.
One of the things we are interested in is the views and answers of Vitalik burtin’s on the challenges of blockchain data availability, which reads as follows:
Fred：We are familiar with the early POW and POS mechanisms; but could you please kindly explain the working principle of the Casper mechanism in a simple way once again?
Vitalik：The challenge is, that it is not enough to just verify that the blockchain is valid, one must also verify that all of the data in the blockchain is available in the p2p network, and anyone can download any piece of the data if they want to. Otherwise, even if the blockchain is valid, publishing blocks that are unavailable can still be used as an attack that prevents other users from taking money out of their accounts, by denying them the ability to update their cryptographic witnesses.
We have solutions, though they are somewhat complex; they essentially involve encoding the data redundantly and allowing users to randomly sample to check that most of it is online; if you can verify that most of it is online, you can use the redundancy to recover the rest of the data.
Randomly distributed throughout the p2p network. The basic idea behind the current Casper implementation is that users can send 32 ETH into a smart contract, and then once they are included in the blockchain they are added to the current validator set. Every block is created by a random member of the current validator set, and every 100 blocks the entire validator set needs to send a message “finalizing” some checkpoint.
In Ethereum’s case, there is this requirement that the blockchain must ensure that absolutely 100% of the data is valid and available; in Filecoin’s case it’s ok if one or two files drop off.
There are a lot of people who can feel that the context of the scene described in the passage above is not very clear, and they are dizzying to read. Why does Wang Feng let Vitalik Buterin introduce Casper and also will focus on data availability challenges and solutions? What is the logical correlation in the middle? To figure that out, lets start with the bitcoin ledger structure.
We say that the essence of blockchain technology is to solve the problem of trust establishment between the decentralized system nodes through the verification and consensus mechanism of distributed nodes, and realize the decentralized and distributed trust establishment mechanism. The process of verification and consensus, for the present, is mainly for two aspects, namely computing and storage. Of course, what we’re seeing is the result after the data is stored, but logically speaking, storage and computation can be validated and agreed separately.
The ledger structure of Bitcoin is shown in the above figure. A basic knowledge that everyone knows is that for a whole node of Bitcoin, all the block records must be saved. For any node, you can verify whether you have the correct record by verifying the Merkle Tree of the block you downloaded. This process can be simply understood as the process of comparing two Merkle Tree files:
So, in a block chain, By simply validating the Merkle Tree, we can get both the validity and consistency of the data, because the data of all nodes are the same, so a few nodes are not online and do not cause the unavailability of the data.
But we also know that because of this design idea, the blockchain has a lot of limitations, So a new technical solution, Sharding, is put forward, the so-called Sharding, which is a technology in the conventional database domain and is now being used in the blockchain field.
Let’s leave aside the pros and cons of Sharding, a special problem that will be introduced after Sharding is the availability of data. Because in the Sharding’s case, its not the same as before. According to the Sharding rules to store the data, the data in that partition is the same, but between the pieces, the data is different. So, we see today that if you want to implement Sharding, then,Smart contracts are largely incompatible with Sharding. Because smart contracts require the same data to be seen.
Let’s imagine that in a P2P network, the previous block-chained network connected to the nodes around it could always be verified with Merkle Tree for data validity.But now, in the case of Sharding, the Nodes we have connected to have the right data may be offline for some reason, and we don’t have the right data here, which can lead to bigger problems.This is also the reason why Vitalik Buterin mentioned about data recoverability at the beginning.
So, from Bitcoin to Sharding, the whole node ledger to Sharding is a somewhat similar process of moving from Sia to IPFS, where the evolution of technology is quite similar. Sia was an early storage project that offered a service similar to an electronic network disk. Sia’s currency is called SC. This project is different from what most people think. It is actually a bitcoin-like project. The mining mechanism using POW is to calculate the hash. Sia’s hash algorithm is called Blake2b, which is slightly different from Bitcoin. Therefore, those so-called hard disk IPFS mining machines can not dig Sia.
Sia uses a special parameter of Bitcoin called Timelock. In other words it delays a fact-accounting transaction, sets a conditional parameter within the delay time, and and a so-called file contract is completed using the script. The contract is to store the value of a specific segment of a Merkle Tree to the client periodically, and both parties verify that the Merkle Tree is holding the data for storage.
The implementation of the Storj project, as well as the proof of storage, is essentially similar to Sia. Both rely on the Client’s to do its own data holding validation. For these two projects, I personally feel that Sia is a “Aura of Light” and Storj is an “Engineering”. The Sia team had a lot of problems with this implementation at the time, but they managed to embellish some concepts and still implement the project. Sia’s problems include confidentiality, integrity, usability, and privacy.
Although Sia claims to implement public validation, it is basically a way of stealing the bell because there is only the Root Hash of the Merkle Tree in the chain. Merkle Tree’s Root Hash only guarantees that the data is not tampered with and cannot guarantee the data is held and available.
The algorithm implementation of integrity verification is dependent on the client and the server for block data and the Merkle Tree leaf node communication verification, and the communication cost is high.
Sia’s excavation work is POW’s, means that the miners simply calculate the hash value. Logically, there is no relationship between POW hash value and storage. There is no block reward for storage node storage. It is not helpful for miners to waste a lot of calculation of hash value to improve the security of storage.
Sia requires the availability that the client itself encodes the file redundantly.
Sia does not deal with privacy and confidentiality.
Sia is the first project that combines distributed validation and consensus with storage systems, and that’s the whole point of Sia. Sia clearly tells us that distributed storage is achievable, although there are many issues that need to be addressed.
Then later developed into IPFS and Filecoin. IPFS is a very good project, it can be simply summarized as using DHT addressing using BT to transfer DAG Object (Blob, List, Tree, Commit).IPFS is a better amalgamation of existing technologies, but IPFS does not address the issue of data integrity. Data integrity should be solved by FileCoin, but FileCoin has a big problem. Many people mistakenly believe that after IPFS hashes the data, it can guarantee that the data cannot be tampered with, which is a misconception.It is impossible to solve the problem of holding data (in non-academic sense) without a simple storage system with digital monetary incentives. on the other hand, the process of claiming the proof of holding data in FileCoin’s paper is logically unreasonable. This is not a technical issue, it is a logical issue.
Logically speaking, the proof of data holding is a game of two characters and four steps. The first role is Challenger, and the second character is the person who completes Proof . Challenger’s first step is to build the file and some puzzles and put them on Server. The second step is to generate information about a Challenger, which requires some data; The third step includes the storage node to complete the Proof and sends it back to Challenger. the fourth step is that Challenger uses some of the information left by itself to generate a Verify.This is to complete the validation with two roles and four steps.
Filecoin strangely changed the four steps into three, eliminating the need to generate Chal. Because the author of the Filecoin white paper clearly knows that there is no role in his system that can logically perform the task of generating Chal. So from the beginning, Filecoin and his POST were an impossible task. In the Filecoin system, the role of Chal cannot be Client, and it is impossible to be a Chain Node at the same time because there was only one consensus algorithm POW, and all computing power of Chain Node can only be used to compute hash value.
Recall that Bitcoin is two equivalent books. The bitcoin scenario is that the bitcoin books of any two machines must be the same. They only need to construct a Merkle Tree to verify the two nodes. In fact, Sia and Storj are the same. The network disk has files and the cloud also has files. In the case where both parties have original files, we can easily construct a certificate. But if it is IPFS, Lambda, and Sharding by V God, how can one complete the proof if one party has data and one party has no data? This is a more common problem. How can we complete the integrity of the data on non-trusted storage without the original file? This is actually an academic problem of the computer. Just as we know that this field is academically Solution.
Since 2007, two algorithms have been proposed, One is the persistence Proofs POR ( POR, Proofs of Data Possession), and the other is the integrity Proofs PDP ( PDP, Provable Data Possession), which is called the integrity of Data. V-god’s data recoverable proof is actually what the academic circles call POR. The principle is simple: first make a (t, n) threshold of the secret share of the data, The so — called tn threshold is the secret you share to n individuals, only t individiual have this data and the data can be restored, this is tn threshold secret sharing, we can realize the data recovery through secret sharing. Because tn threshold secret sharing is an encryption mechanism, a secret message can be inserted, and Challenger request for the message will tell if the data is still there. So it achieves the recoverability on the one hand and the test of possession on the other, this is POR.
PDP does not need to achieve recoverability, it needs to do things very simple, as long as you can see the data is still available. This scheme uses a label method. After the file is split, each block has a label, and the label is placed on the server side. The label can be mathematically operated between the labels. so when the data is requested on the Challenger, the label is also requested. and the service side does the operations on the label,and feedback’s the mathematical results to Challenger for a check
Where is the biggest problem? Who is going to be Challenger is a problem that IPFS does not think about because in this scenario, Client can’t do Challenger because it doesn’t hold data, so Blockchain’s Node can’t do Challenger, because the Note itself is a low-speed chain, and the consensus algorithm is POW. So the IPFS is in a bind. “Prove yourself and test yourself” is a very difficult process to reach a consensus in itself.
I believe that with the continuous development of BlockChain. PDP and POR have developed into so-called trusted third party credential roles. Since third-party trusted verification is academically possible, we can replace the so-called trusted third party through the chain consensus, and also replace the third party to verify through the consensus of the chain and the consensus of the semi-trusted nodes. Putting the results on the chain, and to solve the problem that the original trusted third party cannot solve the problem that the data will not be tampered with, this mechanism is probably the case.
Written in the back
In fact, the challenge for IPFS is that it is similar to AWS S3 when it is a Storage. At this time, it is impossible for the client to hold the original data for verification. Therefore, Filecoin cannot use the same data integrity as Sia and Storj. The biggest problem with Filecoin is that this proof of POST is impossible to implement. In essence, it is equivalent to Sharding’s proof of the availability of the data.
In this Conversation, Vitalik actually gave a general idea of what he thought of the problem. Its essence is similar to Lambda’s idea, but Vitalik is to solve the problem of the recoverability of account data, we are to solve the problem of recoverability plus holding of data. In other words, the Vitalik’s solution is similar to the POR in the field of computer science, but it is worth mentioning that the Vitalik does not mention the integrity of the data (PDP), and from this point of view, Lambda is one step ahead.
In fact, Vitalik Buterin’s POR algorithm is not what he thought of itself, it is that there is research in this field, but the current research progress is just right for the implementation for blockchain technology, which is the origin of Casper consensus algorithm. Lambda’s idea is that, in the case of non-Blockchain, in another verification algorithm that is different from POR, a PDP algorithm can be constructed, assuming a trusted third party to verify the data possession with a certain probability. And store the verification results in an explicit and non-tamperable manner. Then this credible third-party audit, also known as TPA, must be able to pass the verification results through the chain to achieve non-tamperable, and a single-point trusted verification process must also pass a set of semi-trusted Validator nodes. Consensus to complete.
So if you look at Vitalik Buterin’s answer, there are several key terms, namely Validator, Random, Sampling. and through this method, proof of data holding and recoverability can be achieved. In general, Vitalik Buterin discussed only at the POR level, while Lambda is thinking deeply in both POR and PDP dimensions.
Editor’s note: this article is a repost from “Odaily” (official account ID: o-daily, Download App), a blockchain media outlet in strategic cooperation with 36 Kr
Although data in chains cannot be tampered with or backdated, the unknown storage terminals are untrusted by users who save their documents there in the decentralized storage environment. In this case, how is it possible to prevent malicious behaviors of untrusted nodes? In other words, how can we be sure that distributed storage terminals have actually accomplished their storage tasks?
In this regard, the Provable Data Possession (PDP) scheme for storage projects is particularly important. According to the scheme, users send data to the miners for storage, and the miners prove that they have stored the data. Users can double-check whether the miners are still storing their data.
Of course, there are various decentralized storage schemes on the market at present. In terms of PDP, we know that IPFS, Sia, Storj and other decentralized storage schemes all want to ensure the reliability of storage terminals by validating their PDP in an untrusted environment.
He Xiaoyang, founder of Lambda, a decentralized storage project, told Odaily that the existing two kinds of storage schemes all have their own disadvantages:
IPFS/Filecoin, for example, provides only static file storage and cannot search file addresses through file content. Meanwhile, as to the PDP verification of storage terminals, they want the public to initiate the verification, but specify no specific initiator, nor how to initiate; that means it’s unclear who the data validator is.
l The storage schemes represented by Sia and Storj intend to challenge storage terminals regularly through smart contracts, requiring them to provide feedback on the verification information of file fragments, hash values, etc. However, due to the limitation of chain ledger sizes, only the Root Hash of the Merkle Tree is stored in chains, which only ensures that the data cannot be tampered with, but cannot guarantee the possession and availability of the data.
Lambda plans to build a decentralized storage platform for enhancing the trust between users and storage terminals through the verification and consensus of distributed nodes. The Lambda platform sets up two network systems (like IPFS/Filecoin): the repository system and the block chain. The former is responsible for storage, while the latter is responsible for the access and control of the repository system. The two network systems are connected by subchains to realize interactions.
Unlike other decentralized storage projects, Lambda intends to provide storage terminals with PDP through the consensus generated by validator nodes, to ensure the integrity and recoverability of the data stored on untrusted storage terminals.
The approach provided by Lambda is to prove a storage terminal’s possession of file “F” in a Permissionless Store in the simplest scenario. The first step is to generate a collection of messages (m1, m2…, mn) of any file “F”, following the sharding of the original file that a user needs to store, and then calculate and cryptographically generate a number of tags based on the shards of the original file and four figures (i.e. the security parameter λ, or metadata).
The validator nodes store metadata for subsequent generation of challenges/puzzles; storage terminals store the original file shards and the corresponding tags which can help storage terminals accurately extract data from the original file shards and solve the puzzles.
The validator generates challenges/puzzles to challenge storage terminals, which then calculate based on the original file shards and tags in hand and generate a corresponding set of digital vectors, feeding them back to the validator. Finally, based on the metadata and vectors, the validator calculates and verifies whether the verification of storage terminals is successful. If it is, the transaction will be packed and embedded in the block chain; if not, the storage nodes will be punished.
When a single point possession of a file can be verified, the characteristics of the file such as dynamics, multiple copies, erasure codes, and deduplication can be supported by corresponding adjustments.
Taking such an approach, the validator needs to save the metadata, and storage terminals need to undertake extra tasks to store the tags, with the storage volume increasing only 1% to 3%. In the process of initiating verification against storage terminals, the validator’s challenge request will be made remotely because the two sides are in different network architectures. When a storage terminal does not accept the challenge or cannot be connected due to network problems, the validator can directly conclude that it fails to store data.
Throughout the system, the role of the validator is Lambda’s innovation point. Lambda also designed the roles of nominator and fisherman.
Validator: The validator performs transaction packing and block generation across the entire Lambda network, therefore, it needs to pledge some Tokens. Instead of a single validator node, a random set of validator nodes are selected from hundreds of thousands of validator nodes to jointly verify a storage terminal, and the validator node is shifted for every 1024 blocks.
Nominator: Nominators with capital recommend one or multiple validators to make decisions for them. They have no other function than capital investment.
Fisherman: Fishermen are not correlated to the block packing process since their roles are similar to “bounty hunters” in the real world. Appealed by a one-time rich reward, the fishermen report the malicious behaviors of validator nodes through entrapment.
If data are maliciously deleted, it is especially necessary to restore the data file. In terms of the recoverability of data, the vision of Lambda is to distribute a file to B persons after the data file is divided into A parts (B < A). As long as the missing data are not greater than a certain preset value, they can be recovered. Since the preparation in 2017, the Lambda project has completed an angel round financing of tens of millions of RMB, with ZhenFund, Metropolis VC and Dfund as the lead-investors. According to He Xiaoyang, the test network of Lambda will go online early next year. Among the 15 team members of Lambda, the major members of the core founding team are from OneAPM, an APM SaaS company, and most team members are front-line programmers of basic software R&D and from open source communities.