On May 26, WeBank’s blockchain and full-stack technology system debuted at the Guiyang Digital Expo and won the Leading Technology Achievement Award. Since 2015, WeBank Blockchain has launched a number of open source blockchain technologies. Its chief architect Zhang Xiang will talk about the development of blockchain and elaborate on the development of WeBank’s blockchain. Thoughts on technical system research work and talent training "Blockchain can actually be very simple", let's learn together!
A few years ago, blockchain technology was still very primitive, and developers almost built blockchain applications "with bare hands": deploying the underlying platform and writing contracts. In the process, they had to face many technical details and technical problems, and entered various fields. border. It is a pit that no one has stepped on, and it is also very difficult to debug and operate. The experience is like walking on a trail, muddy and unknown.
It has always been our goal to level the technical threshold of the industry, improve application experience, reduce operating costs, expand domain boundaries, allow developers to use technology conveniently and confidently to create value, and accelerate the industry application of blockchain. The beginning of everything is simple. In order to achieve our goals, we firmly believe that we must first have a "big road" that can be opened up and traveled.
The WeBank blockchain team invested in blockchain technology research in 2015 and fully open sourced the results to the industry in 2017. To date, it has officially released more than 10 major blockchain open source projects, participated in the preparation of multiple international and domestic standards, and submitted a large number of patent applications.
In recent years, thanks to my country's strong support for scientific and technological innovation research, including national policies or national initiatives such as new infrastructure, data elements, privacy protection, and carbon neutrality, the improvement of digital technology and industrial innovation has accelerated.
At the same time, with the support and encouragement, optimization suggestions, and co-creation and co-construction of the open source community, the direction of technical research is becoming increasingly clear and iteratively agile. Many technology projects are started after receiving needs and inspiration from the community. The community works together to develop code, maintain peripheral tools, and write technical and user documentation. Together, they created the WeBank blockchain-rich community technology ecosystem.
After years of joint efforts, we believe that the technology in this field has matured and the "highway" for developers has been paved.
Field development trends
As blockchain applications accelerate in recent years, we have seen some clear trends:
Blockchain applications continue to expand in scale and geographical scope. The national blockchain network runs through all provinces and cities. The technical architecture is diversified, the business form is more complex, and the governance model is more changeable... The single-layer, single-chain blockchain network has become unsustainable.
Many blockchain applications running in production environments carry massive requests, and blockchain data will only continue to increase over time. The storage capacity is approaching the hardware limit, and data needs to be split, migrated and backed up. . At the same time, developers also have strong demand for massive data management, analysis, and disaster recovery solutions.
When combined with physical industries, blockchain has different system architectures from existing IT systems, and different industries have different business logic, data dimensions, and security compliance requirements. How to clearly identify the upper and lower boundaries of the chain, let the blockchain and the original IT system perform their respective duties, integrate seamlessly, and solve business pain points is a question that every developer is thinking about.
As blockchain applications enter people's lives, how to define the identities of people, institutions, and the Internet of Things? How is personal data managed? How can people have a good experience when interacting with blockchain while protecting privacy? … Behind these problems are the pain points of an era when data has become a factor of production.
In short, the scale of the network is getting larger and larger, the data is becoming more and more massive, and the scenarios are becoming more and more abundant. Users’ calls for a friendly blockchain application experience will surely become louder and louder. These trends confirm that the depth and breadth of blockchain applications in the digitalization process are increasing, and they also place higher requirements on technology.
First lay a solid technical foundation
Let's go back to the first point. From the beginning, we have firmly chosen the direction of the alliance chain. There is no doubt that blockchain, as an infrastructure, must first be "trustworthy and reliable", that is, it must ensure compliance, security and system stability.
Compliance means that technically we must first remove virtual tokens, adopt a permissioned joining model, support KYC (Know Your: Know Your Customer), and provide regulatory interfaces and audit mechanisms to comply with the regulatory audit requirements of the corresponding industry.
Security is the lifeblood of many industries, including system security, contract security, data security, privacy security, etc. Based on financial-grade security, we have carried out a large number of security reinforcements in network, storage, operating environment and other aspects. For example, we implemented advanced storage encryption in 2016; in terms of contract security, we implemented static/dynamic security both during the development period and during online operation. Detection; In terms of privacy computing, we provide scenario-based and targeted data privacy protection solutions.
When technology is actually deployed in production environments, stability is critical. For example, financial-level stability requires "five nines", that is, 99.999% availability, which is equivalent to 7*24 hours of operation, with almost no downtime, and can be maintained and upgraded online, which can cope with various internal and external faults and meet demanding requirements. RPO (Recovery Point) and RTO (Recovery Time Objective) requirements.
Compliance, security, and stability are "fundamental" and are requirements that cannot be compromised and must be met. This also requires us to concentrate on key core technologies, including cryptography, consensus algorithms, smart contracts, distributed networks, etc., and provide guarantees with strong technical capabilities. Judging from the wide range of applications in recent years, our open source projects have withstood basic tests.
At the same time, FISCO BCOS and related components are fully domestically produced, and the full stack supports domestic hardware, operating systems and national secret algorithms.
On the basis of a solid "foundation", high performance, ease of use, scalability, and sustainability are important "plus points" and are also the key to the popularity of open source software. Only with the recognition and support of the community can the technology be truly implemented and form a healthy and sustainable ecology that is co-constructed and rapidly evolving.
We must run fast and go far
After fruitful optimization, the single-chain TPS underlying FISCO BCOS has reached 10,000 levels and can be infinitely expanded in parallel, which is sufficient to meet the needs of large-scale applications. The “blockchain is slow” narrative is a thing of the past.
But performance is not as simple as "running scores". The system must not only run fast, but also run steadily and go far. To ensure this, the first task is to face up to the issue of data capacity.
The data on the blockchain exists as a chain, only increasing but not decreasing. However, industries such as finance require data to be retained for many years or even permanently, which will inevitably put great pressure on storage. Today, some community developers' applications have been running online for several years, generating hundreds of millions or even tens of billions of data, and the initially configured hard drives are facing capacity bottlenecks.
The problem of data bloat is not as obvious as the functionality and performance issues, but it is like the "elephant in the room" that always has to be faced. We must completely solve this problem from the storage system.
At the same time, the larger the amount of data, the more value contained in the chain and the more worth mining. Developers also hope to have a more convenient, intuitive, efficient and secure method to process this valuable data, such as querying who has deployed which contracts on the chain, how many transaction calls each contract has, multi-dimensional cross analysis, etc., rich Report statistics, business intelligence modeling, supervision and anti-money laundering support, etc. There must be a solution for complex range queries and calculations on data.
When we started analyzing this issue in 2018, we believed that the blockchain nodes themselves should focus on reaching consensus and ensuring the timeliness and transaction consistency of transactions. Even if it is technically feasible to add complex data processing capabilities to the chain, this is not an optimal solution because it will couple the computing and storage overhead with the nodes on the chain, making it difficult to scale.
So, can these problems be solved through the combination of on-chain and off-chain?
First, we have greatly optimized the data structure of the on-chain state, and the on-chain data storage overhead has been significantly reduced by several orders of magnitude. With the same investment in storage hardware, the system can continue to run longer. According to estimates, for general accounting and certificate depository business, under medium-to-high-intensity business traffic, a 4T hard drive is enough to store data for 5 to 10 years.
Secondly, we introduce a distributed storage system to completely synchronize on-chain data to off-chain, and rely on easily scalable technologies such as network storage devices, relational databases, and data warehouses to establish blockchain data mirroring off-chain.
In this way, mature data maintenance tools can be used for data migration, cold backup and other operations. When the blockchain network expands, new nodes can quickly synchronize data. We can also extract data in batches from data mirrors to build snapshots and quickly restore new nodes.
Since the off-chain data is complete, we can consider pruning the historical data in the node and only retain new data that needs frequent access. When an application or on-chain calculation verification process needs to access historical data, it can be directly queried off-chain. This process is completely transparent to data access parties. Hot and cold data are separated, seamless access, and capacity and access issues are completely worry-free.
When performing complex analysis and multi-dimensional joint query on data, we can combine relational databases, professional big data platforms, search servers, AI analysis engines, etc. to perform joint retrieval, modeling and batch processing of off-chain data.
It is worth mentioning that the processing of off-chain data has nothing to do with whether it is "centralized", because all participants on the chain can independently export and process data based on the nodes they have deployed, without relying on other nodes. It is also a form of multi-center distributed collaboration.
By decoupling the architecture and integrating multiple technologies, we have established an on-chain and off-chain distributed storage system, taking into account the distributed real-time and transactional nature of on-chain transactions, while also solving the problems of data capacity and big data processing.
In order to simplify data operations for developers, the community has released a series of open source projects, including data warehouse, data export, data migration, data analysis and other components. Developers can flexibly combine and use investments according to their own needs and resources.
At this point, solving the data capacity problem is as simple as putting the elephant in the refrigerator: download the open source tool, connect it to the blockchain, run it, and go get a cup of coffee. The data is then continuously stored in the database, presenting a complete view of the data. In front of you, it is like hundreds of rivers returning to the sea, and the sea is as vast as the fish leaping.
Respect data and fully protect privacy
When engaging with the community to discuss how blockchain can be introduced into business, privacy concerns are a recurring question: Can personal data be uploaded to the blockchain? Are account transactions between Company A and Company B visible to other companies? If visible, will trade secrets be disclosed?
The distributed ledger model of blockchain emphasizes point-to-point collaboration, data sharing, and full traceability of data. This also means that if a business puts all data on the chain and the data on the chain is accessed at will, it may cause privacy issues. Leakage problem. If key information such as the number of registered users, business volume, and profit margins are leaked due to "on-chain", it is unacceptable in many business scenarios.
There is no contradiction between the openness and transparency of blockchain and privacy protection. In the real world, some information is open to the public, while other information needs to be protected. Private data will only be viewed and used within a controlled scope with the user's authorization and consent.
The dimensions, exchange processes, sensitivities and regulatory requirements of data vary greatly in different scenarios. We believe that no single strategy can solve all problems. What we need to establish is a three-dimensional technology and management system that can be flexibly combined and customized. .
The general idea is divided into isolation, control, and privacy computing.
Isolation refers to whether the data needs to be uploaded to the chain? In what form does it twine? How to give it the upper hand? The isolation strategy is relatively intuitive. What is uploaded to the chain is public data, which should be visible to multiple parties, or only the HASH of the data is uploaded to the chain, and the plain text is stored outside the chain. In addition, FISCO BCOS single-chain multi-group architecture provides developers with a ledger-level isolation strategy. Only participants who join the group can access a specific range of ledger data.
Control refers to the use of complex access mechanisms and role permission models. Network and data access are subject to access review. The entire operation process can be traced, and the results can be audited and supervised to ensure manageability and controllability. This requires presetting control points from the design of network, storage, transaction processing processes, contract engines, and applications, providing customizable strategies for different scenarios and needs, as well as efficient and easy-to-use operation interfaces.
Private computing uses a variety of encryption algorithms. On the one hand, it helps users replace plaintext with proofs, selectively disclose their information, and avoid unauthorized collection of private information. On the other hand, it utilizes cross-institutional federated computing to replace data sharing and achieve data sharing. Available invisibility effects.
WeDPR is the general name for our complete set of data privacy protection solutions for privacy computing. The main purpose of WeDPR is the "," of data and user data sovereignty.
WeDPR provides numerous underlying algorithms, multi-language development kits, and visual privacy computing platform experiences, implementing algorithms such as federated learning, secure multi-party computation, homomorphic encryption, zero-knowledge proof, and selective disclosure. Functionally, it can meet privacy protection needs; in terms of performance, a lot of targeted optimizations have been made for different algorithms, and it has millisecond-level response capabilities during runtime, which is enough to meet the data scale of tens of millions or even billions.
Combining multiple privacy protection strategies, we can meet changing business processes and protect identities, assets, transactions, behavioral data, file sets, etc. in a targeted manner. To facilitate understanding, we provide open source application reference implementations for typical scenarios such as anonymous voting, secure payment, secret bidding, joint marketing, and joint risk control.
In recent years, data has become a factor of production, and relevant legislation has been introduced one after another. Whether it is a personal APP or a B-end back-end service, as long as it is related to data, it faces stricter and more comprehensive privacy protection regulations. The extensive data collection and use processes of the past no longer apply.
It is necessary to respect the rights and interests of users' data, use data reasonably and compliantly, use innovative models to circulate data, meet the needs of safe storage, trusted transmission, and collaborative production, realize property rights that can be defined, stored, evaluated, and circulated, and realize deep-level Mining the value of data and ultimately releasing data productivity.
Expanding the boundaries of blockchain
If the blockchain business is compared to the scenery that users see on the road, and the overall blockchain application architecture is regarded as a smart car, then the underlying platform of the blockchain is equivalent to the "chassis". In addition to a high-performance, stable and safe chassis, many capabilities such as human-computer interaction, intelligent assistance, and cloud-network interconnection are also needed to enable the car to interact smoothly with the driver and lead passengers to gain more experiences. places, and expand technology and application boundaries.
Flexible ledger form: FISCO BCOS has become a widely adopted underlying blockchain platform, equipped with a series of basic tools such as console, SDK, and browser, and has strong usability.
Especially after the release of the group function of FISCO BCOS 2.0, developers can deploy it once like setting up a chat group, and then dynamically create distributed ledgers between different organizations based on business collaboration relationships.
For organizations with many partners and variable cooperative relationships, group characteristics expand the ledger format, accurately solve the pain points of repeated chain building, improve construction efficiency, and reduce human and material costs.
Distributed identity system: At present, distributed identity system has received widespread attention in fields such as finance, government affairs, and public affairs. Using a distributed identity system, cross-institutional and cross-network identity interoperability can be established to solve the problem of distributed mutual verification of trusted identities and trusted credentials on the chain. Only by guarding the door of identity authentication and identification can the connection between the physical world and the digital world be established safely and reliably.
It is a distributed multi-center identity protocol that complies with the W3C DID specification and fully implements the protocol and peripheral support components of the distributed identity system. Combined with the distributed message collaboration system, IoT devices are given distributed identities and carry the instant messages they generate, which can support the integration of IoT, edge computing and blockchain.
Wide-area cross-chain interconnection: Cross-chain solutions are dedicated to the interconnection between heterogeneous blockchains. We see that with the development of business, it is possible to introduce different underlying blockchain technologies in different construction periods of large-scale blockchain networks and applications, or even use multiple underlying blockchain platforms from the beginning of planning. Layering and partitioning. , the business-specific chain network system will not be bound to a certain underlying technology.
Well, in such a complex technology ecosystem, a series of problems will arise due to differences in underlying architecture, data structures, interface protocols, security mechanisms, etc. For example, applications must be developed repeatedly for each chain, data and data on different chains are difficult to seamlessly connect transactions, etc.
Conduct research and abstraction on network addressing, cryptographic algorithms, application interfaces, transaction models, security governance, etc. between different chains, and put forward the vision of "books with the same text, cars with the same track", in line with the spirit of "maximization" consensus, operational The code” “is open-minded and adaptable to various underlying chains.
It realizes a series of functions such as contract intermodulation, mutual status identification, and transaction guarantee between different chain instances, and provides a unified interface for the application layer, allowing applications to easily interconnect WORA (Write Once, Run) and blockchain applications. , expand the network boundary.
Interoperability with the real world: The smart contract engine itself is a "sandbox" that blocks the interference of various uncertainties and ensures the distributed consistency and transactionality of transactions on the chain. For example, when using smart contracts on different nodes to call the local system time, the clocks may be inconsistent according to the time zone and time, resulting in differences in contract calculations and inability to reach an agreement; similarly, random numbers, file systems, network data and stocks outside the chain Real-world information such as prices and football match results cannot be directly called by smart contracts. There is a "gap" between smart contracts and the real world.
Oracles are used to open up information channels on and off the chain. The oracle service responds to invocation requests from on-chain contracts and collects information from the Internet and even the real world. After cryptographic verification, it is anchored on the chain in a trustworthy and reliable manner. On-chain contracts provide a unified view of data, enabling smart contracts to obtain consistent and deterministic off-chain data.
Oracles expand the boundaries of smart contracts and can achieve many interesting and useful on-chain and off-chain interactions, such as distributed draws based on verifiable random numbers, cross-border financial services based on exchange rates, and people's livelihood applications based on blockchain. Weather information.
Smart contract engine evolution: The smart contract engine is the core of the blockchain. Previous smart contract engines have been somewhat unable to cope with complex businesses. For example, the interpretation engine is slower, cannot support more complex data formats and deeper call stacks, does not have strict constraints on asset security, and does not have native cross-chain and privacy protection instructions. Developers also face the pain point of inconvenient development and debugging, and hope to use a programming language they are familiar with to develop smart contracts.
The smart contract project innovatively proposes "SPEC" design specifications for security (), performance (), experience () and customization capabilities () to improve the operational efficiency, asset security, privacy protection native support and cross-chain collaboration language of smart contracts. engine. feature.
At the same time, the introduction of open and versatile technologies will help continuously transplant various popular programming languages into smart contract languages. It also provides friendly development and compilation tools, adjustable IDE, etc., and the expressiveness and development experience of the smart contract language are becoming more and more perfect.
Improve the multi-party collaboration governance system: With the development of applications, collaboration relationships become more complex. We believe that in addition to solving technical problems and meeting various functional requirements, we also need to improve the governance model to achieve long-term stability. Multi-party collaborative governance focuses on "people, things, rules, and processes", defines the various roles of blockchain stakeholders, sorts out matters that require multi-party governance on the chain, and includes collaborative rules covering discussion, formulation, decision-making and execution of the entire process. on the blockchain.
The direction of governance is actually all-encompassing and very complex; the governance process is also full of collisions and games between different ideas and different interest positions. Full on-chain or off-chain governance can be incomplete, inefficient, and prone to forks. It is necessary to seek maximum consensus with a relaxed attitude and implement it pragmatically.
After studying the history and future development of governance, we combined the advantages of on-chain and off-chain and proposed a three-dimensional governance strategy. We start from the "meta-governance" that defines governance rules and clarify the definitions of roles, permissions, affairs, etc. in governance. Describe the overall governance structure and implement and open source a series of multi-party collaborative governance projects.
This series of projects provides intuitive and easy-to-use strategy design and practical tools to implement common functions such as multi-party voting, committee election, account governance, permission management, etc., helping administrators, operators and other participants to participate efficiently and transparently. to on-chain transactions to complete full life cycle management from blockchain construction, business changes, to system upgrades and maintenance.
The above solutions were gradually implemented and improved during our exploration process based on our understanding of the development of the field, as well as discussions and feedback from the community. The release of each solution can often solve pain points in certain fields, fill gaps, effectively broaden the technological boundaries, and continuously promote the development of the industrial ecosystem. It is obviously difficult to make progress behind closed doors, but forward-looking forecasts based on community consensus can help us see a clear direction. We will continue to work with the community to develop more solutions, so stay tuned.
“Cool” is what developers love
As open source projects grow, technical documentation has millions of lines of code and words. Although it gives developers rich choices and a huge learning material library, does this bring additional learning and application costs to developers? Wool? One of the architects on the team had a very useful summary: "If a user doesn't use your open source project within 15 minutes, he may be lost."
Therefore, in order for users to actually use it, it is not enough to just list the seemingly "hardcore" technical components. Open source projects should be well-structured, highly available "products" that deliver core capabilities and have a friendly and smooth interactive experience, allowing developers to use them with almost no cost and no barriers. Only in this way can they gain the attention and recognition of the community and form a community. effect, attracting more people to build together.
We provide high-speed access channels to the code, documents, and distribution packages of each open source project to avoid developers falling into the dilemma of network links; during the installation and deployment process, built-in one-click scripts and visual deployment tools allow developers to perform simple configurations . Set up development and production environments in seconds.
In reality, developer environments vary widely. We will work with developers to study and solve problems reported by the open source community, and continue to cover various environments; the middleware platform integrates one-stop operation management console and developer IDE, rapid deployment and operation and maintenance tools, data reports and audit analysis tools etc., with intuitive web and mobile graphical interfaces, making the blockchain "visible, tangible and tangible". "It's fun to use" and was loved by the community after its release. This series of operations can ensure that developers can start the blockchain "painlessly".
In addition, when users have an on-chain environment and start building blockchain applications, they often not only use the underlying blockchain and specific components, but integrate multiple solutions to jointly meet the diverse needs of the business. .
For example, applications need to manage distributed identities, introduce information such as exchange rates outside the chain, and require cross-chain interconnection. The data generated needs to be maintained and analyzed while protecting privacy... Multiple technologies are integrated with each other, and there are many changes. Collaborative relationships require governance.
Modern software design advocates modularity, customization and scalability. Each component is highly specialized in its own right and can independently solve domain-specific problems. It can also be assembled and integrated with underlying and other components to form a “package” matrix of blockchain solutions.
Around the blockchain bottom layer and middleware, every technology component is continuously integrated through open interfaces. The entire system adopts a layered and loose "microservices" architecture that complies with "cloud native" specifications and is easy to integrate with privatized deployment environments, cloud platforms, and open network architecture systems. .
Based on their own needs, developers can select one or more component blockchain “packages” based on open source code, cloud platforms, and open networks, and use one-click application development scaffolding, contract templates, and application marketplaces to quickly The overall process of building applications and developing them is moving towards a "low-code" model, which significantly speeds up the implementation of the technology.
The breadth of the blockchain field is all-encompassing, going deep into the core of knowledge such as computer theory, distributed networks, and cryptography, and exploring areas of business, philosophy, and game theory at a high level. Blockchain requires professional and comprehensive talents, which are simply in short supply. We have also made some efforts in talent education, including collaborating with competent authorities, industry associations, universities and research institutions to discuss a range of courses and topics; writing blockchain that meets the standards of the Ministry of Industry and Information Technology Talent Exchange and the Ministry of Education Textbooks and courseware; and co-designing and developing a blockchain training platform with industrial institutions to help students gradually complete the course, practice and verify.
We often conduct multi-angle and multi-level scientific popularization of blockchain ideas, technology and learning methods through a series of articles, salons and online and offline communications. A tree cannot grow into a forest. There are many passionate and professional “missionaries” in the community. Community certified training partners are located across the country. Everyone works together to spread useful and beneficial knowledge. The world is full of students and masters of blockchain thinking and blockchain. There are more and more developers of technology.
In previous hackathons and university technology competitions, we have seen participating teams build environments, design and development from scratch and be able to produce highly complete and outstanding works in 3 days; community partners from all walks of life are also increasingly Implement applications quickly. Faster and faster, the experience of blockchain applications themselves is getting better and better, the functions are becoming more and more comprehensive, and user and developer satisfaction is getting higher and higher. The application of "black technology" does not have to be a difficult and painful development experience, but it can also be smooth and enjoyable.
From road to simplicity, hand in hand
Hard-core basic functions, rich functional components, user-friendly ease of use, and thoughtful guarantees on data operations and version compatibility are the essence of the Fisco BCOS open source community and open source projects, and they are also the key to gaining community support. important reason. Since open source, the community has gathered 40,000 followers, over 2,000 institutions, and implemented over 120 applications across various industries. An active ecosystem also promotes technological development.
In cutting-edge technology research, there are many avenues to explore, and each path has intricate details to study. There are also a large number of related technologies, such as the Internet of Things, artificial intelligence, 5G communications, etc. We believe that the future of technological and industrial development depends largely on the openness and growth of ecology. While immersed in the research and development of core technologies for domestic security and controllability, we pay close attention to the current and future needs of the community and work with many communities. Developers work together to build.
Currently, the community has established multiple SIGs (Special Interest Groups) covering blockchain core, peripheral support and application projects. The developers have intense discussions every day and work on fun, challenging, and freeing things together. If we work together and the wisdom of the crowd emerges, we will avoid many detours and pitfalls, run faster, and will have more and more open source projects available and easy to use.
The open source approach can be summarized in three sentences: hard-core technology, friendliness and ease of use, and open ecology. Blockchain is actually very simple!
标签： #Blockchain #WeChat #Data Governance #Distributed Technology #Banking
评论列表