Technical Details of AO Super Parallel Computer

Introduction

AO is essentially an extension of the Storage-based Consensus Paradigm (SCP), which is similar to sovereign Rollup. The core idea is to decouple computation from DA/storage, publishing/storing data on-chain + computing/verifying data off-chain. Due to Arweave's extremely high data capacity, DApp platforms based on AO and SCP architecture can significantly reduce data publishing and storage costs, making it easier to support scenarios with strong throughput demands.

In most people's existing perceptions, Arweave primarily focuses on the concept of permanent storage and has long been used as a storage layer by various projects. The most well-known narrative of Arweave was to become the never-fading Library of Alexandria, with the ultimate goal of preserving the sparks of human civilization.

Therefore, after the release of Arweave's ao computer, it is quite surprising that Arweave has transformed into a parallel supercomputer. It should be noted that Arweave can still be used as a storage layer; the ao architecture is an addition rather than a replacement of storage functions.

Consistent with SCP, the computation process of ao can be executed in parallel, thus possessing efficient computing capabilities. The processes of ao can interconnect, maintaining a consistent data format, and all data is ultimately stored in the Arweave mainnet in accordance with the ANS-104 Bundle format. All logs of each process are fully stored, and the final process's Holographic State is retained on Arweave.

The computing power of ao, combined with Arweave's permanent storage capability, effectively creates a censorship-resistant, ubiquitous global high-concurrency computer, where any type of DApp, in any language, on any public chain can connect to Arweave and enjoy decentralized computing services that are cheaper yet more efficient than Ethereum.

AOS is slightly different; if ao is understood as a computer architecture, then AOS is an instance of an operating system. Essentially, ao is a structural system, and users need to interact with an operating system similar to AOS to utilize the corresponding functions. For the sake of convenience in discussion, ao will be used uniformly here, but users should be aware of the distinction between the two.

Key Points of This Article:

• Arweave launched the ao architecture, hoping to transform into an all-in-one player for storage and computation, changing the current market perception that it is solely a storage solution;
• Arweave's ao architecture is an addition to its storage capabilities, with the potential to communicate with any public chain and DApp;
• Technologies related to ao include distributed architecture SSI (Single System Image), Actor Model (not to be confused with the Erlang language model), and SU/MU/CU unit construction, where concurrency and asynchrony are key to understanding the ao architecture;
• ao/SCP has enormous potential and is expected to stimulate the development of the Arweave ecosystem, necessitating observation of the ongoing appeal of the "off-chain computation + on-chain storage" model to projects.

Technical Interpretation

Let’s first introduce some prerequisite knowledge as a supplement to ao. After the recent Cancun upgrade and the activation of EIP-4844, Ethereum's data storage issues have become increasingly important. For example, blobs specifically used for storing Layer2 DA data will not be permanently retained on the Ethereum network; nodes can delete blob data that exceeds the time window, and the eliminated data will need to find another storage location.

Although there are Ethereum-based storage platforms like EthStorage that address the issue of blob data expiration, this is not a native solution of Ethereum and relies on additional mechanisms. Furthermore, while EIP-4844 can significantly reduce data publishing costs, it still appears quite expensive compared to Arweave.

Unlike Ethereum, which aimed for the "Library of Alexandria" from the start, Arweave, although appearing rudimentary in computing capabilities, natively supports permanent data storage at a very low cost (storing 1GB of data costs about tens of dollars, comparable to the cost of one Ethereum transaction). In terms of data redundancy storage, Arweave associates the probability of block production with the completeness of the local data set of nodes. If a storage node deletes some data, its probability of successfully producing a block decreases, while nodes that retain the most data have a higher "computational power" for block production and will receive more rewards. Through this method, Arweave's incentive system ensures that any historical data segment is highly likely to be redundantly stored.

It can be said that Arweave is suitable as a decentralized data storage and publishing layer with extremely low storage costs, while ao and SCP are modular blockchain and DApp architectures based on AR. Although the design pattern of SCP theoretically differs significantly from Ethereum Rollup and other security-focused modular solutions, it has high feasibility in terms of ease of implementation and integration with Web2 platforms, as it does not intend to limit itself to a narrow implementation path like Rollup, but rather aims to integrate Web2 platforms with Web3 facilities in a broader and more open framework.

Image Source: Geek Web3 “Interpreting SCP: Breaking Out of the Rollup Paradigm of Trustless Infrastructure”

The above diagram illustrates the principle of everPay using the SCP scheme, where the DA layer uses Arweave, represented by the large circle in the diagram. The brown circle is the Coordinator, which serves as the execution layer, similar to the sorter in Ethereum Layer2. After users submit transactions to the Coordinator, it executes the computations and batches the DA data for submission to AR.

As for the Detector, it is somewhat akin to the challenger/verifier in Ethereum Layer2; they pull the DA data submitted by the Coordinator from Arweave to compute or verify the transaction results. The Detector's client is open-source, allowing anyone to run it. The Watchmen, in fact, are multi-signature nodes managing the cross-chain system, responsible for verifying and executing cross-chain requests. Additionally, the Watchmen are also responsible for signing governance proposals.

It is worth emphasizing that the security requirements for the SCP architecture are not as stringent as those for Ethereum Layer2, but it actually provides projects adopting this architecture with greater freedom, more customization options, and reduced adoption costs, making it a unique and innovative approach.

Image Source: Geek Web3 “Interpreting SCP: Breaking Out of the Rollup Paradigm of Trustless Infrastructure”

To simplify, the framework of ao can be divided into three parts, namely: distributed architecture, parallel computing capabilities, and communication scheduling components. Unifying these three elements constitutes the complete functionality of a supercomputer.

• Distributed architecture: ao uses Single System Image (SSI) to organize the decentralized system of the ao network.
• Parallel computing capabilities: ao employs the Actor Model theory of parallel computing to handle high-concurrency environments and effectively integrates relevant blockchain technologies. The term ao is derived from Actor Orient (focusing on Actor, mimicking the object-oriented term in OOP).
• Communication scheduling components: ao has designed three key parts: Messenger Units (MU) responsible for information transmission, Scheduler Units (SU) responsible for process scheduling, and Compute Units (CU) responsible for parallel computing processes.

Let’s explain each of the above parts. First, the SSI (Single System Image) is essentially a type of distributed architecture. For instance, the server systems behind major Web2 applications are typically composed of many server nodes forming a distributed system. These servers communicate through special messaging and communication protocols to ensure that their states and views on new data are consistent.

However, at the client/front-end level, users are unaware that the servers behind the front end are distributed; to users, even a massive computer cluster appears as a single computer. This is what is often referred to as "abstraction" in computer engineering, where complex underlying components are unified and merged into a single module. The external world does not need to know the internal structure of this module; it only needs to pass input information to the module to obtain output results.

The aforementioned SSI (Single System Image) utilizes Arweave's "cheap decentralized storage" feature, which can be said to be the narrative foundation of ao/SCP, primarily established on Arweave's storage price advantage compared to other public chains, as well as its advantages in censorship resistance and data transparency compared to traditional Web2 platforms. In the narratives of ao and SCP, AR is treated as a massive data bulletin board and log recorder, where data sent from the DApp front end is transmitted to the Arweave network and stored by numerous Arweave nodes in a distributed blockchain network.

Compared to mainstream public chain networks like Ethereum, which have a higher degree of trustlessness, Arweave's extremely low storage costs can better support applications with high data throughput demands. In contrast to traditional Web2 platforms and consortium chains, the openness of the Arweave network is more conducive to censorship resistance and data transparency, making DApps reliant on AR more trustless than Web2 applications.

For example, traditional Alipay could also be Web3-enabled; as long as Alipay designs its interface to be compatible with the ao protocol, the interaction data of Alipay would automatically upload to the Arweave network, becoming a trustless "Alipay" in Web3. If it is an Ethereum or EVM-based DApp, it can also interface with the ao protocol by converting the information format to ANS-104 format for uploading to Arweave.

Unlike traditional cloud services and closed consortium chains, anyone or DApp project can run nodes of third-party public chains like Ethereum or Arweave and request and read data from multiple nodes via P2P. As long as one of the N nodes is willing to provide data, you can obtain what you need, fundamentally relying on the openness of the network.

From these two perspectives, the DApp architecture solutions based on Arweave, such as ao and SCP, resemble a transition between Web2 and Web3. Traditional Web3 platforms like Ethereum and Bitcoin achieve a high degree of censorship resistance and trustlessness at the cost of scalability and efficiency, making large-scale adoption difficult; Web2 platforms sacrifice data transparency and censorship resistance for high efficiency and low cost, but cannot be trustless. ao appears to be an intermediate form between the two.

The difference between SSI and other distributed architectures like client-server architecture, three-tier architecture, N-tier architecture, and peer-to-peer architecture lies in transparency. SSI can significantly enhance system abstraction and user experience. However, it is important to note that SSI relies on optimistic synchronization control, which requires the system to have a high synchronization control capability to ensure data consistency and reliability. If synchronization control fails, it may lead to data loss, thus affecting the usability of the ao architecture.

Another benefit of SSI is deployment speed; SSI can run multiple instances on a single server without overly relying on cloud services or containerization tools like microservices architecture, effectively reducing system complexity and deployment costs.

In the practice of ao, data synchronization and backup of the distributed architecture rely on the Arweave network. Due to Arweave's permanent storage characteristics, theoretically, the data state at any point in time is retained, and data loss or damage does not occur.

However, it should be noted that SSI can also incur new additional overhead, particularly concerning network communication and effective data synchronization between distributed nodes. For instance, when an SSI architecture fails, in extreme cases, as long as there is one normal node, the entire network can operate normally. However, this can lead to serious node security crises and system robustness issues.

Actor Orient

After briefly introducing the SSI architecture, we need to delve into the implementation of ao's parallel computing mechanism. Unlike the simple "stacking" of centralized servers, ao uses the Actor model to achieve decentralized high concurrency, and users are largely unaware that this is a distributed system.

The efficient concurrent computing capability of the ao architecture comes from the Actor model, which was defined by Carl Hewitt in 1973 as a theoretical framework, with Actors serving as primitives for concurrent computation. Interestingly, it was originally designed for computing in artificial intelligence.

However, in practice, people may be more familiar with OOP and other models. In fact, according to Oracle's research, OOP is an improved version of Actor, but the two have diverged significantly in their later developments.

The Actor model defines a set of general rules for how system components should act and interact. Each Actor is an independent entity capable of making local decisions and communicating with other Actors. However, it is important to note that the Actor model emphasizes asynchronous, parallel, and distributed characteristics.

Especially with asynchrony and parallelism, this means that the states of each component are not synchronized, which may lead to conflicts. Therefore, it must heavily rely on a messaging mechanism, which is why MU and SU are emphasized in ao. Execution is not difficult; the challenge lies in arranging and scheduling to leverage the powerful computing capabilities of parallelism.

Each Actor is an independent execution unit that can handle assigned tasks. If utilized properly, such as ensuring the atomicity and consistency of messages, it becomes a very powerful and flexible concurrent model.

The consideration here is the special cross-node communication needs in blockchain. For example, common microservice architectures often use node-to-node communication patterns, but RPC-based implementations can lead to various complexities and delays in data transmission. In contrast, the ao architecture employs a unified messaging mechanism (MU) to standardize message formats for easier final storage in Arweave.

Compared to the synchronous execution task model of CSP (Concurrent Semantics), the most typical feature of Actor is asynchronous execution. To this end, ao does not use common shared memory mechanisms, ensuring the independence of each Actor and facilitating more flexible cross-node communication.

Asynchrony and parallelism constitute the efficient source of the Actor model in the ao architecture. To ensure this efficiency, MU/SU/CU have been proposed and put into use.

In summary, the combination of the Actor model and the Arweave blockchain builds an asynchronous high-concurrency computing model under efficient information transmission.

Three Components

Under the ao architecture, whether it is SSI or the Actor model, higher requirements are placed on information transmission, leading to the emergence of SU, MU, and CU.

First, understanding the processes of ao refers to requesting the corresponding computing resources, such as virtual machines and memory, when initializing tasks. The flow of any task is essentially achieved through the transmission of processes.

Whether it is SSI or the Actor Model, the messages flowing between them must comply with the ANS-104 data standards and formats, allowing any type of DApp to understand each other.

After producing data that meets the requirements, MU sends messages to the online SU. This process continues until all messages are processed. Next, the SU needs to receive the data and upload it to Arweave to link with Arweave's verification capabilities.

Moreover, the processing process of MU can also establish a payment mechanism for customized message processing, such as sending messages without subsequent actions.

Once SU receives the message, CU will begin to engage. CU consists of multiple units responsible for computation. It is important to note that CU is also a decentralized computing power market; similar to Akash, various CU clusters will compete with each other, and the winning competitor has the right to perform computations. CU will respond to requests and submit a computation result, with all response results stored on Arweave, verifiable by the original data on Arweave.

It can be observed that in this model, ao provides an efficient and competitive computing network, where users do not need to establish consensus for computation; they only need to ensure that the transmitted messages comply with the corresponding processes, fundamentally freeing them from the high costs of computation validity incurred by Ethereum and others.

The Combination of SCP and ao

First, let’s discuss verifiability. Keep in mind that ao's function is to present verifiable data states, and ultimately, the verifiability issue is guaranteed by the consensus data on Arweave. ao is essentially an SCP application; at this point, both querying and returning states are recorded on-chain to Arweave. The ao/SCP program will load these two actions and compute the Mint and Slash results based on them.

Specifically, based on the SCP paradigm, the rules for Mint and Slash need to be written into the index. Thus, nodes calling the indexed data will naturally compute the results of Slash and Mint (for data models, see ao spec).

After discussing the technical architecture of ao, the following will focus on its applications. Although native cross-chain protocols like aox, decentralized stablecoin protocols like astro, and EVM-compatible projects like AOVM on Arweave have emerged after the ao launch, they are still in experimental stages. It is worth mentioning that many applications based on ao are currently in development and testing, including ao versions of Twitter and games.

At the same time, some mature SCP projects already existing in the Arweave ecosystem, such as everPay and Permaswap under everVision, will undergo corresponding adaptations and modifications to ao. Theoretically and practically, SCP theory and ao are also of the same origin.

SCP is rooted in Arweave's storage capabilities. One can imagine a Turing tape machine that never stops, where SCP is responsible for uploading data to the blockchain, serving as the tape recording function of the Turing machine, while the state machine can be provided by ao. Each state change can be stored on Arweave.

The problem here is state explosion, which has troubled Ethereum for many years. Since Arweave does not store state, it inherently avoids the state explosion issue and can permanently retain all data produced by ao processes.

It should be noted that the data in ao nodes does not need to be computed to reach a consensus state. Theoretically, as long as the relevant data is stored on the Arweave network, every step of state change must be recorded. Therefore, more accurately, the data on ao does not need computation to be written to the network; computation is merely a part of data change.

Secondly, in the face of the enduring blockchain trilemma, where no blockchain can simultaneously solve the issues of security, decentralization, and scalability, the combination of SCP and ao can essentially resolve this dilemma.

• Security: The data provided by Arweave is the greatest consensus, and the consensus data stored on Arweave provides verifiability for applications;
• Decentralization: ao brings about the decentralization of computing power, allowing any individual, institution, or NGO to join and leave the ao computing network. Thanks to data consensus, restoring the state after leaving is also incredibly simple;
• Scalability: Unlike Ethereum's vertical layering, ao and SCP are generally horizontally partitioned, allowing for infinite expansion of computing and storage capabilities.

Based on the ao/SCP architecture, Arweave is no longer just a pure storage public chain. The combination of AR and AO creates a decentralized supercomputer that possesses both storage and computing functions, where any DApp can be deployed and call upon each other.

Currently, ao is also a unique modular architecture, specifically horizontally scalable modularity, which not only allows smart contracts like Warp on the Arweave network but also enables EVM-based smart contracts to be integrated into the ao network, as long as data format consistency is maintained.

In other words, SCP builds a full-chain Layer2 based on Arweave, capable of connecting to any public chain and DApp, while ao is a super version of SCP, capable of turning any public chain, smart contract, and DApp into part of ao.

Another Possibility for DeFi

In the existing EVM system, smart contracts are at the core of everything, and on-chain behavior essentially involves scheduling and using smart contracts. Taking the most common DEX trading as an example, smart contracts execute corresponding operations according to specified conditions, such as adding liquidity or finding the contract address of a token.

However, it must be noted that at this point, the contract is merely a single-threaded process and cannot perform concurrent calls. All transactions will be sorted by Ethereum to determine their success or be subject to MEV front-running attacks.

If the ao architecture were used to modify Uniswap, it could create a parallel, non-stop on-chain trading bot. The ao version of Uniswap can set different triggering mechanisms for each process without interfering with each other, utilizing all computing resources. This can be understood as a Web2-level quantitative on-chain exchange. The largest DEX project in the Arweave ecosystem, Permaswap, has already adopted this method to adapt to the ao native environment.

Each process in ao has the capability to issue tokens. For example, in Ethereum, the issuance process of each ERC-20 token is a Token Process. By simply setting the price range, orders can be generated for trading users, completing the token exchange process.

In reality, transferring and trading tokens on Ethereum is incredibly challenging, essentially involving addition and subtraction of different account balances. This ultimately leads to each transaction needing to be recalculated based on the latest demands of Ethereum, resulting in a large amount of redundant data accumulating on-chain.

ao changes the computation method for token transfers. The transfer of tokens between different accounts essentially involves synchronizing different states, relying on the message-passing process of MU. Each process only needs to send data to the Arweave network for storage, meaning consensus occurs before computation. The final combinatorial computation can then transfer assets without the entire network participating in the computation. Each token on ao is concurrent and can even establish countless sub-ledgers for a single token, each providing independent parallel computing capabilities.

In the current design of ao, it allows for the automatic awakening of specific contracts. Users only need to pay the nodes to use that process, which will then compute and execute at a set frequency. Thanks to Arweave's low costs and ao's high speed, the execution frequency can be set very densely.

For users, ao executes a program similar to a computer, rather than an abstract smart contract. For instance, after integrating EverID, everPay connects to the ao network, where EverID operates the ao interface. The type of program behind the ao interface is irrelevant; users can simultaneously operate DApp applications across multiple public chains, highly resembling the usage logic of existing internet terminals, which call upon different servers in the network to provide users with a simple and unified application interface.

Essentially, this represents a transformation of existing DeFi, unifying users' operational logic within a single interactive system while preserving the underlying degree of decentralization.

Global Supercomputer

In addition to traditional crypto needs like DeFi, ao also opens up the capability to feed back into traditional Web2, with one key focus being trustworthy computing for ML (Machine Learning). As mentioned earlier, Carl Hewitt designed the Actor Model with the original intention of AI computing, which naturally gives ao and AI a combined capability.

In previous integrations of AI and Crypto, machine learning models struggled to smoothly go on-chain, especially large models like LLMs with massive parameters. However, ao is different; since users can choose and customize the resources allocated by ao and can use computing services without admission, and since computing resources are essentially infinitely scalable and cooperative, putting ML on-chain seems possible.

Compared to decentralized computing markets like Akash, ao's advantage does not lie in the number of GPU clusters but in its massive parallel computing capabilities. Akash requires a strong trust mechanism to build a computing power market.

ao does not need to sacrifice its no-admission feature, and it is important to note that all of this is still implemented based on smart contracts, meaning it runs on-chain and is stored in Arweave for state proof. Thanks to its strong compatibility features, users can use the on-chain environment in their preferred ways, such as running LLM models, with their data stored on the Arweave network, simultaneously addressing the decentralization needs of AI large models in terms of computing power and data.

ao is distinct from existing decentralized computing platforms and cloud computing providers; it is the first decentralized high-concurrency network. One can understand it as cloud providers equipped with smart contract functionality. In the face of the computational and storage crises encountered by Ethereum, it can be said that Arweave has completed the marvelous loop of "decentralized computing equals decentralized data."

In simpler terms, the once expensive and inaccessible supercomputers are now within reach of everyone, and no one can control their start, operation, or end. As long as a process begins, it will only end under the conditions set by the smart contract; otherwise, it will run indefinitely.

Conclusion: The Future of Arweave

With the arrival of ao, combined with the capabilities of the SCP paradigm, Arweave has the potential to become a network of permanent storage and infinite computation. However, it should be noted that the current ao operating nodes are still in a testnet state and rely on the penalty slash mechanism coded into the system for operation.

Theoretical optimality does not equate to practical feasibility. ao aims to become a computing network that can infinitely extend and scale in real-time, with users having complete control. However, the various ecosystems on Arweave are not yet active, especially mainstream DeFi applications, which remain relatively scarce, not only fewer than Ethereum but even lagging behind Filecoin's launch of FVM.

Overall, the combination of ao + SCP + Arweave indeed opens up another possibility for blockchain, but this possibility still requires validation over time.