Interpretation of Arweave Version 17 White Paper (5): More Derivative Incentives
This article will continue to interpret the fourth section "Protocol Agency" from the 17th version of the #Arweave white paper.
How to Calculate the Total Number of Partitions from Network Parameters?
By using the equation described earlier and some information provided by the Arweave network, we can calculate the total number of copies that the network is storing. Each time a miner mines a new block, we can determine whether the solution hash of that block comes from the first or second backtracking range of the SPoA challenge. In a network that stores complete copies, this ratio is essentially 1:1. However, if miners store incomplete data partitions or duplicate partitions (thus suffering efficiency penalties), this ratio will be less than 1.
We can calculate the average hash value for each partition by calculating the observed SPoA source ratio. Suppose in the last 1,000 blocks, there are n1 first range SPoAs and n2 second range SPoAs. This means the average copy integrity is n2/n1, so the mining efficiency for each partition is:
Formula Annotation: In this formula, if the ratio of n1 to n2 is 1:1, then e_m will be 1.
Using the above expression, we can accurately estimate the total number of partitions in the network. When the difficulty parameter is d, the expected number of hash attempts is given by:
When the efficiency of each partition is only e_m, the expected number of partitions required to generate this many attempts in 120 seconds is:
Formula Annotation: E[trials] is the total expected number of hash attempts in the network, 800 is the maximum number of hashes per second for one partition, multiplied by e_m is the number of hashes at that mining efficiency, and multiplied by 120 is the total number of hashes at that mining efficiency during a mining cycle (usually around 2 minutes).
Considering that the size of a partition is 3.6 TB, we can derive the network's deployed storage capacity:
All these metrics regarding storage datasets and average copy integrity can be calculated from the observed values in the network.
Incentives for Optimizing Data Routing
The mechanism that incentivizes miners to build complete copies to improve mining efficiency will trigger a series of beneficial incentives for the protocol. This includes the motivation for miners to develop optimized data routing solutions for fast data transmission in a peer-to-peer network, which is a strong driving force for such complex and critical challenges. Because nodes must be able to quickly transmit any data block in the network, this requires maintaining reusable routing capabilities for users and other miners to conveniently access data and enhance data availability.
For miners, this new incentive for optimizing data routing may create a competitive environment similar to that of Bitcoin miners competing to develop more efficient dedicated mining hardware. This competition will promote innovation in routing infrastructure, ultimately leading to a more efficient and powerful distributed network.
Bandwidth Sharing Incentives
Another derivative effect of Arweave's mining incentive mechanism for storage replication is the strong necessity for miners to access data in the network. This creates various market models for data access, including:
- Karma and the Principle of Optimistic Reciprocity: Nodes in the Arweave network participate in a bandwidth-sharing game similar to BitTorrent. In this game, nodes share data with each other. Additionally, nodes occasionally share data randomly, optimistically expecting future returns. Each node maintains its own peer ranking without needing to report how or why these rankings are determined. Such mechanisms have seen significant success in data-sharing platforms like BitTorrent, which once accounted for about 27% of global internet traffic.
- Revenue from Physical Disk Distribution: Node operators can directly buy or sell physical disks storing weave data in exchange for money or other forms of payment. For bandwidth-constrained miners, this may be a more desirable option given the large amount of data required to run an Arweave node. This method of transmission bypasses traditional packet filters and firewalls. In fact, downloading the original data is indeed a threshold that many new miners need to overcome, and as the total data in the network gradually increases, this form of data acquisition will be more convenient and efficient.
- Payment Protocols: Nodes can also participate in protocols and markets that allow them to make payments while accessing data. The Permaweb Payment Protocol (P3) provides such a way, using payment channels to incentivize various services within Arweave (including simple data access).
Scalability
The average time for Arweave to create a block is about 2 minutes, with each block containing up to 1,000 transactions. This limitation ensures that block validation and synchronization remain extremely lightweight, allowing the entire network to be widely decentralized. However, this transaction count limit does not impose any restrictions on the size or number of data stored in a given block, as Arweave uses a mechanism called "bundling." Bundling is a global standard built on the core protocol (standard number #ANS104) for combining many different data entries into a single transaction. These data entries functionally equate to top-level data storage transactions on the network, as bundled transactions can be "unbundled" into their components during retrieval.
Arweave's maximum transaction size is 2\^{256}-1 bytes, which can be divided into any number of individual data entries in potential recursive packaging. This allows the network's throughput to scale without practical limits. This optimization is possible because data uploads on Arweave are not parameterized—every byte on the network is part of the same global Merkleized dataset and is supported by a shared storage endowment. One element of this design is the payment aggregation from a single data item to the uploaded bundle. Users can choose to combine their data item payments in a bundled transaction or transfer the payment entirely off-chain, allowing a bundling service provider to combine their data items with those of other users.
Figure 1: Bundling allows data to be recursively stacked into a top-level transaction.
In Arweave, all transactions are selected to be included in the 1,000 slots of each block based on their total value, as the inclusion fee earned by miners is proportional to the transaction fee. This incentivizes bundling services to recursively combine transactions, increasing the network's scalability. Therefore, regardless of the number of bundlers and users, they can write data to the network at any given time without leading to a block space auction mechanism like in other blockchains. Additionally, the competition among bundlers to build larger transactions will exert downward pressure on the final cost to users. This sharply contrasts with other blockchains, where intense competition for limited block space leads to increasing fees that ultimately force some users to stop using the network due to high costs.
Figure 2: Preference for larger data packets incentivizes bundlers to recursively bundle data, minimizing cost.
Users can also upload data through off-chain bundling service providers, which allows users to pay for Arweave storage using any payment method supported by the bundling service provider, while bundlers settle the grouped data in AR. As of now, the Arweave network supports at least 18 different payment methods through bundling services.