Arweave Permanent Storage + AO Super Parallel Computer: Building Data Consensus Infrastructure

One of the most important features of Web3 is that users control their own data, which is a significant difference from Web2, where internet giants control user data. Now, blockchain technology pioneered by BTC frees us from the control of traditional banks or internet banks as intermediaries, allowing users to control and conduct peer-to-peer transactions with their own electronic cash; ETH and other smart contract public chains enable users to control and conduct peer-to-peer transactions of various contracts and derivative assets.

However, besides financial assets, there are other types of data assets on the internet, and there are currently no more mature solutions that allow users to control and achieve peer-to-peer transactions of these assets. Therefore, Web3 users cannot fully control their data at this time. The reason for this is that we lack the infrastructure for data rights confirmation. For users to master their data, data rights confirmation must be achieved! To achieve data rights confirmation, consensus on the data must be established. Since data states can be dynamic or static, consensus must be reached at both the transmission and storage ends to establish consensus on data assets, thereby enabling data rights confirmation.

Data can only be confirmed when consensus is reached; only with confirmation can data exchange or transactions occur, and data can only reflect value when exchanges or transactions take place, thus generating a value internet. One significant reason for the severe data island phenomenon in Web2 is that data lacks rights confirmation. The emergence of Arweave's permanent storage + AO super parallel computing is expected to change this situation and help us achieve consensus on data storage and transmission. As shown in the figure:

Arweave's permanent storage has achieved consensus on data storage after several years of development, * (Note: There is already a lot of detailed information available online regarding this, so I won't elaborate here) , here we will focus on how the AO super parallel computer achieves consensus at the transmission end ( Note: Many articles studying AO mention that AO stores the holographic state of processes in Arweave, but very few articles clarify the specific implementation details; they just briefly mention it, so I want to clarify the general implementation path here). *

To achieve consensus at the transmission end, it is essential to ensure data integrity, consistency, verifiability, and transmission efficiency. Before diving into the introduction, let me introduce the design principles of the AO economic model, which will help us understand how AO ensures data security from a top-level design perspective. In the AO white paper, there is a passage that roughly states:

The typical economic model of blockchain networks like Bitcoin, Ethereum, and Solana revolves around the concept of purchasing scarce block space, with security being subsidized as a byproduct. Users pay transaction fees to incentivize miners or validators to include their transactions in the blockchain. However, this model fundamentally relies on the scarcity of block space to drive fee revenue, which in turn funds network security. In the context of Bitcoin's security architecture, it is fundamentally based on block rewards and transaction fees. Consider a hypothetical scenario where block rewards are eliminated, and transaction throughput is assumed to be infinitely scalable. In this case, the scarcity of block space would be effectively offset, resulting in minimal transaction fees. Consequently, the economic incentive for network participants to maintain security would be significantly reduced, increasing the vulnerability of transactions to potential security threats. Solana exemplifies this theoretical model in practice, demonstrating that as network scalability increases, fee revenue correspondingly decreases. In the absence of substantial transaction fees, the primary source of security funding is block rewards. These rewards essentially tax token holders, manifested as operational expenses for those who choose to stake tokens personally or as gradual dilution of the proportionate ownership in the network for those who forgo staking. Earlier, we proposed the need for the $AO token as a unified representation of economic value to support security mechanisms within the network.

This passage shows me that the AO economic model is significantly different from those of other mainstream blockchains. The AO economic model focuses on protecting network security, as the characteristics of non-financial data assets require ensuring the safety of the underlying infrastructure while also guaranteeing efficiency.

The data types of non-financial assets are diverse, and the transaction scenarios for each type of data have different requirements for system security, scalability, and timeliness. This requires the AO network security model to be flexible and not adopt a one-size-fits-all consensus mechanism like traditional blockchains to ensure security. If AO were to adopt such a security model, it would lead to significant waste of computational resources on one hand and severely impact the scalability of the AO system on the other.

Therefore, AO can customize security mechanisms based on different data types and values, with the economic model playing an important regulatory role. In simple terms, high-value data can have a high-level security mechanism customized during transmission, while low-value data can have a security model with lower costs, thus saving computational resources and adapting to the security needs of different data categories. When we analyze this, we can see why blockchains like Ethereum, Bitcoin, and Solana are not well-suited for Web3 data transmission. Their security models are uniform rather than flexibly customizable, which does not align with the transmission characteristics of non-financial data assets. Next, we will delve into the details of how the AO economic model and security model interact and adjust each other.

1. Maintaining Data Consistency, Integrity, and Verifiability:

a. Technical Assurance: In the AO super parallel computer, the message transmission mechanism is a core component that ensures effective communication and collaboration between different computing units (such as CU, SU, etc.). The main process of message transmission is as follows:

Message Generation: Users or processes initiate interaction requests by creating messages. These messages must conform to the format specified by the AO protocol to be correctly transmitted and processed within the network.

Messenger Unit (MU) Reception and Forwarding: The Messenger Unit (MU) is responsible for receiving messages generated by users or processes and relaying them to the appropriate SU nodes within the network. The MU manages the routing of messages to ensure they reach the SU accurately: during this process, the MU digitally signs the messages to ensure data integrity.

Scheduler Unit (SU) Processing: When messages arrive at the SU node, the SU assigns a unique incremental nonce to the message to ensure its order within the same process and uploads the message and the assigned results to the Arweave data layer for permanent storage.

Computing Unit (CU) Processing: After receiving the message, the Computing Unit (CU) executes the corresponding computational tasks based on the message content. Upon completion, the CU generates a signature proof with specific message results and returns it to the SU. This signature proof ensures the correctness and verifiability of the computational results. The specific workflow is illustrated in the figure:

💡 (Note: This image is from the AO white paper)

Additionally, the core principle of the AO super parallel computer is to decouple computation and consensus. AO itself does not solve the problem of message verification but ensures that all messages and states are verifiable by storing the holographic states of all processes in Arweave. Anyone can verify the consistency of messages through Arweave, meaning anyone can challenge the correctness of AO messages, and anyone can initiate a challenge through Arweave to verify messages. This not only frees AO from the constraints of traditional blockchains—where all nodes' computations and verifications are parallel, enhancing system security but also consuming significant computational resources and failing to present high scalability (for example, in the Ethereum system, adding more nodes does not significantly increase processing speed). However, because of this characteristic, AO can achieve high scalability; on the other hand, it also ensures that all data is verifiable. This is the cleverness of AO's design, transferring verification costs off-chain while ensuring verifiability.

b. Economic Model Assurance: The above process outlines the general workflow of message transmission in the AO super parallel computer. Additionally, MU, SU, and CU nodes are required to stake $AO, and corresponding solutions are provided for various unexpected situations that may arise with these three types of nodes. For example, if an MU is found not to have signed a message or has signed invalid information, the system will reduce the staked assets of the MU. If an MU discovers that a CU has provided invalid proof, the system will also reduce the staked assets of the CU. The AO white paper sets forth targeted economic model-based solutions for various issues arising from MU, SU, and CU nodes, ensuring that these three types of nodes do not act maliciously. Furthermore, the AO super parallel computer also allows MUs to aggregate proofs from multiple CUs through a stake aggregation mechanism, ensuring the integrity and credibility of information transmission (for specific processes, see AO white paper sections 5.6 AO sec origin process and 5.5.3 stake aggregation).

Additionally, the SIV sub-staking consensus mechanism allows users to reach consensus or partial consensus on results: clients can autonomously set the number of participants or validators, thereby controlling the impact of consensus on costs and delays.

In summary, the AO super parallel computer combines technical and economic models to ensure data integrity, consistency, and verifiability. Moreover, due to the varying security needs of different types of data, AO provides flexible and customizable security models.

2. Preventing Data Leakage:

AO encourages MU/SU/CU nodes to enhance security measures by introducing an economic staking model and ensures data security and flexibility through mechanisms such as security level purchasing, equity exclusivity periods, and equity time value. The general situation is as follows: clients can insure the messages they purchase, and the value of this insurance is related to the message's value, the staker's expected return rate, and the message's security assurance time. This not only ensures the security of data transmission but also motivates stakers to provide higher security guarantees; on the other hand, if a message leakage occurs, it can also protect the interests of the message recipient, thereby enabling consensus between the buyer and seller of the message, promoting data asset transactions.

Additionally, AO has partnered with PADO, allowing users to encrypt their data using PADO's zkFHE technology and securely store it in Arweave. Since Arweave is also decentralized, this helps prevent single points of failure. These mechanisms ensure that data is adequately protected during transmission and storage.

3. Ensuring Data Transmission Efficiency:

Unlike networks like Ethereum, where the base layer and each Rollup effectively operate as a single process, AO supports an arbitrary number of processes running in parallel while ensuring the integrity of computational verifiability. Furthermore, these networks operate under global synchronized states, while AO processes maintain their independent states. This independence allows AO processes to handle a higher number of interactions and maintain computational scalability, making it particularly suitable for applications requiring high performance and reliability.

Moreover, since processes on AO can be holographically projected to Arweave, the message logs on Arweave can trigger the execution of AO processes in reverse. If a single process is interrupted, it can be immediately reactivated through Arweave, thus preventing single points of failure and restoring process states in the shortest possible time, thereby ensuring the efficiency of message transmission.

This article has thoroughly explained how AO ensures the integrity, consistency, verifiability, efficiency, and leak prevention of message transmission at the transmission end. When these aspects can be guaranteed, data consensus can be achieved at the transmission end. At the storage end, Arweave has been operational for several years, achieving permanent storage of data and ensuring consensus at the storage end. Therefore, the combination of Arweave's permanent storage + AO super parallel computing is expected to solve the consensus problem of large amounts of data at both storage and transmission ends.

If this problem can be solved, it will bring revolutionary changes: massive non-financial data assets can achieve consensus, greatly accelerating the confirmation of data asset rights, thus helping to solve the issue of Web3 data asset rights confirmation. Data assets can only generate substantial economic activities once their rights are confirmed, enabling the realization of a true value internet.

Now, BTC has solved the problem of electronic cash rights confirmation and transactions, allowing each of us to control electronic cash. Ethereum has resolved the rights confirmation and transaction issues of various financial assets through smart contracts and blockchain; while Arweave's permanent storage + AO super parallel computing is expected to help solve the rights confirmation and transaction issues of data assets. Of course, this article focuses on explaining this from the perspective of data asset consensus, as it is the key to data asset rights confirmation and transactions. I personally believe that Arweave's permanent storage + AO super parallel computing has the potential to stand alongside BTC and Ethereum, forming a good complementarity to jointly address key issues in Web3, thus helping us advance towards a value internet. As shown in the figure:

4. Project Risks:

• The connectivity between AO and Arweave: AO's super parallel computing may bring significant throughput and challenges to Arweave, potentially preventing messages from achieving holographic projection or leading to instability in other aspects of the system.
• MU/SU/CU are key nodes in the AO system and may exhibit centralized characteristics, leading to corruption and project instability. It is hoped that the AO official website can establish a decentralized reputation assessment system allowing DAO members to evaluate the superiority of the three types of nodes, thus forming a fair and open evaluation and competition mechanism for these nodes.
• Arweave focuses on permanent data storage, and as its scale expands, it may face scrutiny from various governments. It remains to be seen whether the official has corresponding strategies or solutions for this.
• The design of the economic model needs to be validated: AO's security model has a high dependency on the economic model. Although AO has ensured some data security through the aforementioned technical means, if the economic model does not function well, it will reduce the security of each staking link, thus failing to ensure data security. AO's security model differs from traditional blockchain security models: the core principle of blockchain is to make attackers pay a heavy price to ensure security, specifically achieved through a combination of economic models, consensus mechanisms, and the longest chain principle, using mathematical principles to ensure that the attacker's losses exceed their gains, thus providing strong security guarantees. In contrast, AO's security model design follows the same principles as traditional Web2 security models: strengthening defensive measures to resist external attacks, where the strength of these defensive measures is more regulated by the AO economic model to motivate and pressure each staking node to enhance security. Therefore, if the economic model is poorly designed, it could be fatal for the implementation of the AO project.

Of course, the above analysis is only from the perspectives of technical models and economic models. To consider a project's potential, one must also take into account the technical strength of the project team, the overall background, the project's ecological status, and the potential of the sector or direction it belongs to, among other factors. These aspects will also be gradually discussed with everyone in the future.