Mongodb blockchain white paper
Sunghyun Yu , Dept. Cheolmin Yeom , Dept. Yoojae Won , Dept. Abstract: With the recent increase in the types of services provided by Internet companies, collection of various types of data has become a necessity. However, the data provider remains unaware of the manner in which the data are collected and used. Furthermore, the data collector of a web service consumes web resources by generating a large amount of web traffic.
We are searching data for your request:
Mongodb blockchain white paper
Upon completion, a link will appear to access the found materials.
Content:
- Get started with MongoDB in windows with .NET
- Performance benchmark: PostgreSQL/ MongoDB
- Developer showcase series: Brian Wu, Coding Bootcamps
- Webinar: Building a Blockchain Database with MongoDB
- New White Paper Published: Smart Data Caching at The Edge
- Table of Contents
- Benchmarks and Research
- Subscribe to RSS
Get started with MongoDB in windows with .NET
Try out PMC Labs and tell us what you think. Learn More. Nowadays, it has been recognized that blockchain can provide the technological infrastructure for developing decentralized, secure, and reliable smart energy grid management systems. However, an open issue that slows the adoption of blockchain technology in the energy sector is the low scalability and high processing overhead when dealing with the real-time energy data collected by smart energy meters.
Thus, in this paper, we propose a scalable second tier solution which combines the blockchain ledger with distributed queuing systems and NoSQL Not Only SQL database databases to allow the registration of energy transactions less frequently on the chain without losing the tamper-evident benefits brought by the blockchain technology.
A prototype was implemented using Ethereum and smart contracts for the on-chain components while for the off-chain components we used Cassandra database and RabbitMQ messaging broker.
The prototype proved to be effective in managing a settlement of energy imbalances use-case and during the evaluation conducted in simulated environment shows promising results in terms of scalability, throughput, and tampering of energy data sampled by smart energy meters. The rising shares of intermittent decentralized renewable energy prosumption sources are completely changing the way in which electricity grids are managed to provide electricity to consumers while preserving continuity and security of supply at affordable costs.
The smart grid management problems can no longer be efficiently addressed by using centralized management solutions, the need for developing decentralized approaches and architectures are widely recognized.
At the same time, the deployment of IoT Internet of Things smart energy sensing devices that measure, collect, and communicate energy data has not only created the opportunity for developing rather sophisticated smart grid management services but also generated problems in managing the big amounts of generated data.
Thanks to the increased efficiency of end-user appliances, low-cost photovoltaics, and disruptive technologies such as virtual financial services, decentralized energy networks are rapidly spreading, contributing to low-carbon, sustainable energy systems [ 1 ]. The integration of decentralized energy systems at neighborhood scale allows to lower peaks in energy demands on the electrical grid, to reduce the overall consumption [ 2 ], and reduce the disparity terms of reliability and cost in rural and remote areas [ 3 ].
Thus, future energy systems should be characterized by the key principles of decarbonization, decentralization, and digitalization [ 4 ]. From our point of view, new designs should be considered for distributed storage and for the validation of energy data as close as possible to the registration point.
When addressing the distributed storage, the new solutions should be scalable in terms of data size as well as data throughput. Furthermore, functionalities regarding the data validation and identification of corrupted energy data should be provided in near real-time fashion, in order to ensure a reliable production and consumption matching and a fast-financial settlement.
Nowadays, it has been recognized that blockchain may provide the needed technological solutions for developing decentralized, secure, and reliable smart grid management systems [ 2 , 5 , 6 ].
However, even if in terms of reliability and control blockchain is a good solution, being able to decentralize the entire management process of the smart grid, the scalability when integrating the real-time energy data collected by the sensors is still a pressing issue. In some grid management cases, the drawbacks limiting blockchain adoption to energy domain are the extremely high costs for storing data, the high hardware requirements for processing data, and the small transaction throughput especially when dealing with many peers.
Smart grid services have diverse requirements in terms of response time which impacts the granularity needed for energy data monitoring and the costs of energy blockchain integration. In the case of ancillary services, as they are usually targeting the safety of the grid, their monitoring requirements are very close to real-time.
For example, in Europe [ 7 ] the manual frequency restoration reserve features an average response time of 15 min, for Automatic Frequency Restoration the average response time is 5 min, while for Frequency Control Reserve average response time should be 30 s. Anyway, studies have concluded that an energy data sampling rate of one minute is enough to determine power states for electrical loads of most of the regular prosumers [ 8 ] and can be successfully used for grid services with more relaxed response time constrains such as, congestion management, unbalances settlement, energy billing, etc.
However, even in this case the cost of storing one-minute energy data as transactions in blockchain can be really high. Considering the gas price of 3 Gwei the safe low gas price [ 9 ] of the Ethereum public transaction promising the transaction confirmation in 12 min and the estimated gas consumed per transaction of ,, the cost for one energy transaction is , Gwei 0.
The trading period for the imbalance settlement process generates a cost of 3. Storing data on the blockchain is very expensive with respect to traditional database systems. For example, in the public Ethereum network to store 1 kB of data one must pay approximately k gas.
With an average price for the gas unit of 3 Gwei we end up paying 0. Also, by increasing the size of the chain, more hardware resources will be used by the full nodes to store and process the blockchain data.
High hardware requirements would lead to a decrease in the number of full nodes available, eventually threatening the reliability of the entire blockchain system which will manage the energy grid. At the same time, nowadays blockchain systems feature a small transaction throughput: Bitcoin allows up to seven transactions [ 10 ] per second, Ethereum allows 15 transactions [ 11 ] per second, etc.
Thus, considering that the smart energy sensors have sampling rates at intervals of a few seconds it is unrealistic and infeasible to register and process energy transactions at similar rates into the blockchain. A scalable second tier solution combining blockchain ledger with distributed queuing systems and NoSQL database to allow the registration of energy transactions less frequently on the chain e. Solution validation considering as use-case the settlement of imbalances in power markets, showing its effectiveness in assessing deviations from energy plans and tracking source of that deviation in near real-time while featuring low computational costs and high transactions throughput.
The rest of the paper is organized as following: Section 2 describes the relevant related work in the area of blockchain and energy and solutions addressing the identified scalability problems; Section 3 presents our second tier solution for storing energy transactions and the digital fingerprinting of relevant data while Section 4 presents evaluation results focusing on the scalability improvements and tamper-evident features of our solution.
Finally, Section 5 presents conclusions and future work. The idea of using blockchain in the energy sector has taken an increasingly large interest in academic research, industry stakeholders, utility companies, and energy decision makers. The most intuitive—and popular—application of blockchain to the electric power sector is to turn the electricity grid into a peer-to-peer network for prosumers which trade electricity with one another e.
In general, the energy surplus is measured by smart meters and then transformed in equivalent energy tokens that can be traded in a marketplace setup at local grid level [ 13 ]. Energy tokens or renewable certificates are defined and serve as proof that the electricity has been generated from renewable energy source, to encourage low-carbon and green energy production [ 14 , 15 , 16 ].
Most state-of-the-art approaches focused on the P2P Peer to peer energy market creation [ 17 , 18 , 19 ] are aiming to demonstrate that blockchain-based intermediary-free energy trading is possible and that is beneficial to both the generators and buyers. Other studies are focused on the optimal management of energy resources by P2P trading in local micro-grids [ 20 ].
The use of blockchain technology for the emerging field of IoT [ 21 ] and, subsequently, energy efficiency in smart grid is another area of active research [ 22 , 23 ]. Blockchain and self-enforcing smart contracts can facilitate data exchange and machine-to-machine M2M communication among energy metering devices. It is estimated that an increasing number of smart devices A similar trend is followed by the energy sector where smart meters and ICT equipment are increasingly being adopted [ 25 ].
When the metering infrastructures are integrated with blockchain technology, it is possible to benefit from automated billing in energy services for consumers and distributed generators, which leads to the reduction of administrative costs [ 28 ].
Similarly, in Stephant et al. Smart meters data is registered into a consortium blockchain and shared with Distribution System Operator DSO and energy suppliers to trace energy generation, having a more accurate billing [ 30 ]. Blockchain may play an important role in the field of grid management, for example, improving the coordination between transmission and distribution system operation or improving the balance of energy supply and demand [ 31 ].
The blockchain-based implementation can: Manage the request of balancing power between Transmission System Operator TSO , Balancing Responsible Party BRP , DSO, aggregators, and generation units and to enable DSOs to interact with the balancing request process in congestion situations well before the delivery period and not just at the stage when the generation load is ramped up by the aggregator [ 32 ].
The advent of Electric Vehicles EV and e-mobility poses serious energy grid management problems. In this context, the blockchain technology brings several advantages, such as elimination of a centrally managed EV charging infrastructure, fault tolerance, elimination of price-setting, collusion between charging stations, or transport providers.
Some researchers propose to integrate blockchain with EV to allow customers to find the cheapest charging station [ 34 ] within a previously defined region while preserving their privacy [ 35 , 36 ]. As previously reported, many energy companies are investing in blockchain and IoT as it clearly benefits energy system operations, markets, and consumers [ 37 , 38 ], offering disintermediation and transparency in the energy grid management processes, but most of all, it offers novel solutions for authorizing consumers and small renewable generators to play a more active role in the energy market and monetize their assets [ 39 ].
Besides the regulatory framework in the energy sector [ 40 , 41 ], the other aspect that can slow the adoption of blockchain, is the low scalability and high processing overhead. To address the scalability challenges of the IoT systems, few solutions have been implemented using different flavors of databases: Centralized databases, NoSQL databases, and distributed databases.
Most of the times the database to be chosen for IoT systems is evaluated among existent solutions like MongoDB or Cassandra [ 42 ].
Custom implementations [ 43 , 44 ] are proposed to address specific problems like scalability or security. While these solutions provide promising results in terms of storage and throughput, and surpass the promises of a blockchain solution, the trade-offs introduced are represented by lower data integrity and higher susceptibility to failures. The state-of-the-art approaches on energy and storage solutions can be roughly classified under three main categories see Table 1 :.
Firstly, use the blockchain for storing data on-chain, which is extremely costly, and at the same time is inadequate in use-cases where high scalability is required;. Secondly, use existing distributed databases solutions that are well known for the high scalability and implement some level of decentralized control. However, their shortcomings in terms of Byzantium tolerance and immutability are rendering them unsuitable for specific use-cases where Byzantine attacks can be expected;.
Thirdly, use custom blockchain-based databases such as the BigchainDB [ 45 ] that relies on the Tendermint [ 46 ] blockchain for transaction broadcasting and consensus, as well as on MongoDB [ 47 ] for off-chain storage. As seen in Table 1 , the blockchain databases, such as BigchainDB, seem to be the most suitable solution because they allow for prosumers-defined energy assets to be stored, while ensuring high scalability and a tamper-evident solution changes cannot be prevented but will be detected.
However, the major disadvantage is that it does not provide smart functionality over the registered values. According to the official website of the BigchainDB solution [ 48 ], to apply any logic or activity evaluation upon the registered values, another chain providing smart contracts capabilities must be considered that can fetch data from BigchainDB through different mechanisms e.
Our novel hybrid solution addresses the identified disadvantages being based on a symbiosis between distributed databases and blockchain solutions benefiting on the advantages of each: The scalability of distributed databases and the immutability and Byzantine tolerance of the blockchain. We propose the implementation of a blockchain-distributed ledger in which energy transactions are generated, registered, and immutably stored in blocks based on the monitored energy data of individual prosumers.
The prosumer is modeled as a node of the P2P distributed energy network i. Other energy players, such as the energy aggregators or the DSO that are interested in micro-grid management, are also registered as network peers. The flow of energy between prosumers will be then represented in the blockchain as transfer energy transactions between two prosumers contracts.
Whenever a new prosumer joins the blockchain network, a new account and associated contract is created, and the corresponding node will be connected to a predefined list of seed nodes.
The seed nodes will provide the new joined node with information about all the prosumer peers they know about, the process being repeated with the new discovered peers, until the new node builds its own list of peers.
To provide better scalability and decrease the length for the chain, multiple transactions denoted as TX1—TX8 are aggregated in a single block see Figure 1. Depending on the solution, different approaches have been used in order to keep track and encode the transactions mined in a block. In Bitcoin the transactions of the block are grouped and encoded using Merkle Trees. To support the protocol enhancement, the Ethereum [ 50 ] solution uses three different modified Merkle Patricia Tries per block that store key value pairs: State and Storage Trie, Transaction Trie, and Receipt Trie.
To build a Merkle Tree, all energy transactions in the block are paired two-by-two and the tree is built from the bottom to the top based on the hashes H of these transactions.
The tree leaves will contain the transactions while the upper levels will be incrementally constructed by pairing and combining the hashes of two elements from an inferior level until the root is reached.
In this way, a binary hash tree is built up to the root. The result is a byte string encoding an entire set of data. Due to its structure, any change that occurs at leaf level, due to tampering with data, will trigger modification up to the root of the tree. The Merkle Tree Root hash encodes the entire collection of energy transactions that are aggregated in the current block. This hash value is very important because it is responsible to ensure the validity and the integrity of the recorded energy transactions in time.
The root hash is added in the header of the block see Figure 1 and, together with all the other fields in the header, is used to generate the hash of the block. The block hash is further used to identify a block in the entire blockchain. The hash of the tree root provides a smaller footprint, an important advantage for the prosumer nodes that do not have enough storage capabilities i.
Thus, the prosumers associated light nodes will store only the header of the blocks while the actual energy transactions will be stored remotely. The Merkle tree root will provide enough information for light prosumer nodes to be able to check the consistency of the chain.
At the same time, light prosumer nodes may interrogate other network full nodes for information to verify if an energy transaction was mined and to identify the block that stores the actual transaction. One of the simplest ways to prove that an energy transaction is stored in a block would be to obtain all the transactions of a block and rehash the entire tree to obtain the root hash.
If the root hash obtained is the same with the one stored in the header of light prosumer node block, due to the collision free and data binding properties of the hash functions, it would lead to the conclusion that the specific energy transaction was successfully mined. However, this process is not optimal. Since a large number of energy transactions can be included in the block, the verification process is significantly improved by providing only a path instead of the entire set of transactions.
For example, consider that the light prosumer node needs to find if energy transaction TX6 was mined or not. It will request a path that proves the membership of this transaction in a specific block as presented in Antonopoulos et al.
Performance benchmark: PostgreSQL/ MongoDB
Mongo shell is the default client for the MongoDB database server. You can use Mongo Shell to connect to DB instances, and query, update, and manage data in databases. This section describes how to use Mongo Shell to connect to a replica set instance over a private network. To improve data transmission security, connect to instances using SSL.
Developer showcase series: Brian Wu, Coding Bootcamps
I was reading a post recently about Red Hat removing MongoDB support from Satellite and yes, some folks say it is because of the license changes. However, in all this time, MongoDB has become a much more mature product. So what happened? Or is the problem that people are blaming MongoDB for their own lack of efforts when evaluating if it was a good fit? Every year there are new trends that pop up in the world of software engineering. Some fizzle quickly, while others fundamentally change the way software development is performed. This process was codified by Gartner with its hype cycle , which while controversial is a decent approximation of what happens with technologies that are eventually found to be valuable. But every once in a while, a new innovation appears or in this case reappears that is driven by one particular implementation of that innovation.
Webinar: Building a Blockchain Database with MongoDB
Try out PMC Labs and tell us what you think. Learn More. Nowadays, it has been recognized that blockchain can provide the technological infrastructure for developing decentralized, secure, and reliable smart energy grid management systems. However, an open issue that slows the adoption of blockchain technology in the energy sector is the low scalability and high processing overhead when dealing with the real-time energy data collected by smart energy meters. Thus, in this paper, we propose a scalable second tier solution which combines the blockchain ledger with distributed queuing systems and NoSQL Not Only SQL database databases to allow the registration of energy transactions less frequently on the chain without losing the tamper-evident benefits brought by the blockchain technology.
New White Paper Published: Smart Data Caching at The Edge
With the development of information technology and network technology, digital archive management systems have been widely used in archive management. Different from the inherent uniqueness and strong tamper-proof modification of traditional paper archives, electronic archives are stored in centralized databases which face more risks of network attacks, data loss, or stealing through malicious software and are more likely to be forged and tampered by internal managers or external attackers. The management of intangible cultural heritage archives is an important part of intangible cultural heritage protection. This study combines the characteristics of blockchain technology with distributed ledgers, consensus mechanisms, encryption algorithms, etc. Optimizing methods, applying blockchain technology to the authenticity protection of electronic archives and designing and developing an archive management system based on blockchain technology, help to solve a series of problems in the process of intangible cultural heritage archives management. With the development of computer technology and the improvement of economic levels across the world, different data intangible cultural heritage management systems have been widely used in the management of intangible cultural heritage.
Table of Contents
Hyperledger Member companies are hiring. Back to our Developer Showcase Series to learn what developers in the real world are doing with Hyperledger technologies. Next up is Brian Wu of Coding Bootcamps. What advice would you offer other technologists or developers interested in getting started working on blockchain? To start your career in this space, you should spend as much time as you can reading blockchain news, white papers and books. Understand the basic blockchain concept from examples like Bitcoin, Ethereum and Hyperledger Fabric.
Benchmarks and Research
The study evaluates the throughput and latency of the databases under four different workloads and three cluster configurations. For consistent results, the comparison was conducted using the Yahoo! Cloud Serving Benchmark. The paper includes four comparative diagrams and seven descriptive tables to support the evaluation results.
Subscribe to RSS
Revised docs based on tips from… compare. Where communities thrive Join over 1. People Repo info. Dec 08
Beyond financial services, blockchain technology has the potential to fundamentally transform interactions across almost every industry — from media and entertainment, to logistics and supply chain, medicines and patient care, energy trading and loyalty programs, through to government and military applications. Form Name. UTM Source. Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously. The cookie is used to store the user consent for the cookies in the category "Analytics".
Find centralized, trusted content and collaborate around the technologies you use most. Connect and share knowledge within a single location that is structured and easy to search. I started to implement bigchainDB.
I think he is wrong. I'm sure. We need to discuss. Write to me in PM, speak.
Exactly, you are right
This seems to do the trick.
I can't take part in the discussion right now - there is no free time. I will be back - I will definitely express my opinion on this issue.