The past, present, and future of AO: A transcript of the conversation between Arweave founder Sam and EverVision founder Outprog

Author: PermaDAO

The term "narrative" is quite popular in the crypto industry. It originates from the sociological concept of "metanarrative" proposed by French philosopher Jean-François Lyotard, used to describe stories that serve as universal ideals with foundational legitimacy in political propaganda.

However, the crypto industry is increasingly misled by the notion that it is the "narrative" that determines a project's value, rather than the production efficiency improvements or enhancements in production relations that the project itself can provide.

Since the release of the hyper-parallel computing network AO on February 28, it has largely been interpreted as a narrative upgrade for Arweave. It has evolved Arweave from mere decentralized storage to "storage + computation," becoming a part of the Ethereum killer lineup.

If the understanding of AO remains at the "narrative" level, it would be a profound regret. When you try to get closer to understanding AO, you will find it has tremendous potential and the opportunity to bring about profound changes in the crypto industry.

Unlike the blockchain networks we are familiar with, AO seems to have no boundaries or limitations; anything is possible. We have noticed that in less than a month, a series of exciting applications based on AO have emerged, including instant messaging software, social platforms, games, and more. Perhaps in a few years, the entire crypto industry will realize that our current position is not merely a crossroads of narrative upgrades, but the beginning of a true paradigm shift!

To help developers, researchers, and even investors in the crypto industry gain a deeper understanding of AO, PermaDAO invited two of AO's founders for an in-depth conversation about the origins, evolution, and future vision of AO. They are Sam Williams, founder of Arweave and CEO of Forward Research, and Outprog, founder of EverVision, proponent of the SCP paradigm, and early concept designer of AO.

The Development History of AO

First, Sam recalled the development of AO from his memory. During the lockdown at the beginning of the COVID-19 pandemic in early 2020, the Arweave team discussed and formed a concept through Zoom meetings: to create a neutral decentralized computing log system capable of inferring the state of any program. Based on this idea, they quickly developed a basic smart contract system called SmartWeave, which was released as a proof of concept. SmartWeave quickly gained success, attracting numerous teams to join the Arweave ecosystem in the summer of 2020 to build projects based on SmartWeave, which became the core of the Arweave ecosystem. Subsequently, Outprog proposed the Storage Consensus Paradigm (SCP), extending the concept of computing logs to all types of data logs and writing a series of articles that laid the foundation for understanding the core theory of AO.

The concept of AO further proposed implementing SmartWeave within bundled interactions, where these data entries could have their own labels, unrestricted by scalability, but with different data availability properties, hinting at the possibility of an arbitrarily scalable computing system. The core design of AO is messaging, which discards the interactive locks in traditional SmartWeave and adopts message logs that can come from users or other processes. This design began development by the Forward Research team in the summer of 2021. By the end of 2022, the architecture of AO was proven to be feasible and capable of horizontal scaling. An internal testnet was launched in January 2023, quickly attracting 30 active developers. The public testnet was released at the end of February, and within just three and a half weeks, about 3,000 developers joined and built projects, demonstrating the community's broad acceptance and interest in AO.

Outprog also described the birth history of AO based on his experience with Arweave. Since July 2020, he joined the Arweave ecosystem and began researching, attracted by its potential as a consensus data storage solution, believing it opened up infinite possibilities for computation. His goal was to promote the widespread adoption of blockchain technology, not just for cryptocurrencies and NFTs, but to become a part of everyday life. He developed everPay, the first multi-chain payment network on Arweave, serving as a bridge between the Arweave ecosystem and other blockchains, and received investment from Arweave in May 2023. everPay operates with zero gas fees and requires no waiting; all data is uploaded to Arweave to ensure verifiability. His concept for AO originated from the Arweave Day in Asia in 2022 in Singapore, and later, during the Berlin Arweave ecosystem gathering in the summer of 2023, he discussed parallel computing issues with other developers. To address the issue of trusted communication between SCP applications, he designed the Message Protocol, which can be seen as a precursor to AO.

The Design and Improvement Process of AO

Sam mentioned that in the early stages of the project, the team envisioned building AO as an extension layer on top of Warp, but soon realized that this approach required fundamental changes to the SmartWeave architecture, which was impractical in practice. As the team continued to focus on the development of AO in the latter part of last year, they reached a breakthrough "almost discovery" moment by the end of the year, confirming that the designed architecture was not only reasonable and feasible but also capable of horizontal scaling. Upon first using AO S, the team experienced a new way of operating: they could log in and start any number of processes within the system, all running in a boundless flat space. Users could easily log into processes, issue commands, and seamlessly communicate with other users' processes, which was not only highly composable but also possessed strong scalability.

Why AO is Called a Global Parallel Computer

Sam said that AO provides a single system image, allowing users to experience an underlying structure composed of numerous different computers within a unified and singular computing environment. This system supports the parallel execution of an unlimited number of processes in a decentralized environment, achieving true parallel computing, as if creating a powerful decentralized supercomputer. Its design inspiration comes from the concept of distributed operating systems, which aim to enhance capabilities by adding more computers at home, in the office, or at the workplace, while users feel as if they are logging into a single computer. The AO test network runs on over 220,250 different nodes, and anyone can connect their computing unit to this system, thereby enhancing overall computing power, making AO a democratized supercomputer where users can log in and perform computing tasks through various devices, such as laptops or smartphones. This design not only promotes widespread user participation but also enhances the accessibility of decentralized computing, whether through running nodes or using services for computation.

Outprog recalled that initially, the concept of AO was an omnipotent state machine, a platform for trusted computing where the clients, operating systems, and all running programs in users' hands originated from Arweave. AO itself does not solve the issue of verifiability; through the SCP paradigm, verifiability is provided by consensus data on Arweave.

The role of AO is essentially to replace the traditional HTTP communication protocol with the Message Protocol, providing a decentralized messaging method for SCP applications, thereby facilitating communication between decentralized applications. According to the vision from July 2020, if both the user's requests and the server's responses go through AO, it would be possible to reconstruct a truly decentralized internet, migrating the entire internet to Arweave. The value of AO goes beyond storage; it can also record the stories of the present and distribute future value in a decentralized manner, transforming Arweave's "Library of Alexandria" into a platform for recording and distributing value.

Differences Between AO and Traditional Blockchains like Ethereum, Solana, and ICP, which also aims to be a Global Computer

Sam:

1. Comparison with Ethereum

   Ethereum is described as a "world computer," but in reality, it resembles more of a "world calculator." While it has enabled many innovative applications, it has not achieved the original vision of a global computer. AO adopts a different architecture, not optimizing global shared addresses or memory spaces, but allowing each contract to have local private memory and interact through message responses, which is fundamentally different from Ethereum's model.

2. Comparison with Solana and Other Blockchains

   Solana and some other blockchains (like Sui and Aptos) adopt widely different architectures, but the key distinction of AO lies in its messaging approach. AO believes that to achieve scale, one should not optimize for global shared memory space, based on practical experiences outside of distributed systems and traditional networking.

3. Comparison with ICP

   ICP handles computation itself and the consensus through consensus. In contrast, AO employs a lazy evaluation computing method. AO has a dedicated subnet, known as the computing unit, for executing these computations. This approach differs from ICP's handling of computation and consensus. The ICP community seems to face challenges in achieving instant consensus, while AO, through its computing unit subnet and staking mechanism, can provide more reliable and economically guaranteed computational outputs.

4. Uniqueness of AO

   The design of AO draws lessons from the history of distributed operating systems, particularly the limitations of shared memory methods learned in the 1980s and 1990s in distributed operating systems. AO adopts a messaging approach, which is the foundation of the internet, transmitting messages through TCP/IP and computing across a wide network. AO brings this concept into blockchain, achieving equally good scalability both on-chain and off-chain. AO does not directly reach consensus on the results of interactions (i.e., computational outputs), but rather on the inputs of interactions and the order of those inputs. This means AO focuses on the order and availability of data, rather than directly reaching consensus on computational results. Although AO's consensus does not directly target computational results, the outputs of computations remain trustworthy and verifiable. This is because AO has a dedicated computing unit subnet, and even on the testnet, there are numerous computing nodes waiting to execute computations and provide staked, verifiable responses. Since AO uses a deterministic virtual machine and reaches consensus on the order of interactions, there is also consensus on computational results. This ensures that even if not instantaneous, the computational results are consistent and predictable.

Outprog:

1. AO as a New Interaction Layer

   Traditional blockchains like Bitcoin and Ethereum primarily address the Byzantine Generals Problem, which is achieving consensus on a single event in a distributed network. AO and SCP (Storage Consensus Paradigm) no longer focus on the issues of blockchain and objective truth, as these problems have been well addressed through Proof of Work (PoW) or Proof of Stake (PoS) in blockchain systems. AO is building a brand new interaction layer on top of the existing blockchain infrastructure.

2. Separation of Consensus and Computation

   SCP emphasizes that consensus occurs at the storage layer, using Arweave for immutable storage, ensuring security and verifiability. On Arweave, there is consensus on the Merkle tree and the order of data, rather than on the state of data computation. Essentially, AO does not solve the issue of verifiability; the functions of AO and AR are completely separate.

3. Computational Handling of AO

   AO is responsible for sequentially organizing data on Arweave to generate states. Since AO cannot change the order of data on Arweave, it cannot change consensus either. Ethereum, Bitcoin, Solana, and ICP all adopt on-chain computation, which is distinctly different from the design philosophy of SCP. SCP allows AO to compute on top of Arweave's storage layer, while the storage layer is responsible for maintaining data immutability and security.

Use Cases for Developers That Were Previously Impossible on Traditional Blockchains but Can Be Achieved on AO

Sam provided some reference directions for developers:

1. Decentralized Computing Infrastructure

   AO offers a new form of network space with the potential to decentralize global computing infrastructure, making the cost of running computations comparable to running on cloud services (like Amazon EC2), while also providing verifiability for smart contracts.

2. Large-Scale Computation as Smart Contracts

   AO allows developers to build large-scale computational smart contracts, which is impossible in other existing smart contract systems. These smart contracts can execute financial interactions, such as automated algorithmic funds that can automatically adjust buy and sell orders based on market data and news.

3. Trust-Minimized Financial Services

   Developers can create trust-minimized financial services, such as decentralized algorithmic funds, where users can deposit tokens without trusting fund managers, with all operations automatically executed on-chain.

4. Decentralized Data and Information Processing

   By using Oracle services like 0rbit, developers can process and verify large amounts of data from the internet on AO without trusting centralized Oracle providers. This service can access and process HTTPS-encrypted data from the entire internet in a trust-minimized manner.

5. Autonomous AI Agents

   In AO, it is conceivable in the long term to have a network space composed of autonomous AI agents that live and interact in their own virtual universes, unbound by the physical laws of the real world.

6. A Brand New Financial System

   AO enables developers to build a completely new financial system that includes autonomous financial agents capable of making decisions based on market information, allowing users to deposit or withdraw funds at any time.

What Are the Next Developments and Roadmap for AO? When Will the Mainnet Launch?

Sam outlined the future roadmap for AO:

Current Development Stage

The mainnet version 1.0 has been built and is currently operating as a testnet. The core data protocol has been completed and is not expected to undergo major changes. The data protocol itself does not provide security but has formed the foundation of AO.

Second Phase Goals

A staking mechanism will operate using a set of nodes and three different subnets, with each subnet responsible for executing specific tasks, requiring staking to ensure these nodes and subnets operate as expected and enhance system security. The staking mechanism not only ensures the normal operation of nodes but also provides security guarantees for computation, messaging, and process scheduling. Although the related smart contract designs should remain simple, they must undergo thorough testing to verify their reliability. Nodes will also take on the role of "gatekeepers," responsible for monitoring processes and sub-nodes, and voting on the network when security policy violations are detected. The core protocol infrastructure of AO consists of the underlying data protocol, off-chain implementations of nodes, and a smart contract layer that allows for staking and penalties, thereby building a decentralized and trustworthy computing environment.

Third Phase Goals

The long-term development goal of AO is to gradually implement strict security protocol execution for computing units, messaging units, and scheduling units by combining hybrid authority proof systems and staking proof mechanisms. This process will involve creating software to enforce security rules and smoothly upgrading the system after running for a period, allowing scheduling units to stake and users to autonomously run schedulers. AO aims to gradually increase its degree of decentralization, ultimately becoming a permissionless decentralized staking proof system. It is expected to transition through a series of testnet versions to mainnet previews and candidate versions, which may take several years until the data protocol and smart contract layer reach 99.5% stability, at which point the system will officially become the mainnet.

Future Development of AO

The AO team hopes to adopt an approach similar to Solana's "mainnet beta" in the early stages until reaching version 2.6. Over time, the team can improve in this regard. It is almost certain that a solid foundation has been established to provide protocol guarantees to the world, meaning that the core architecture of the system will not undergo large-scale changes. Although it has taken about six and a half years to develop AO, the hope is that future development processes can be shorter and more mature. Before starting to build AO, the team ensured that the architecture was nearly optimal from a scalability perspective, recognizing that improvements could be made in early naming and version control. The team believes they will take a more mature approach to optimize the development process and timeline for AO.

What Actions Will EverVision Take Next? What Contributions Has EverVision Made to the AO Ecosystem and How Will It Participate in AO's Development in the Future?

Outprog:

everVision is a lab aimed at exploring large-scale applications of blockchain. AO may currently be one of the best practices for large-scale blockchain applications. The team will reconstruct the results of their work over the past few years according to AO's specifications, meaning EverID can become a universal identity standard on AO, making it easier for ordinary users to use these applications. Meanwhile, everPay's cross-chain capabilities will also transform into native cross-chain capabilities on AO. AO's "sub-ledger" can significantly address the scalability issues of payments, which are key points we need to integrate technically.

Permaswap has already launched the AO test token CRED, and now you can establish liquidity for CRED/AR, and you can conveniently trade CRED on Permaswap or provide liquidity to earn transaction fees. Permaswap is a very mature DEX product and can be seen as the Uniswap of the AO ecosystem. Next, Permaswap will also be highly integrated with AO, allowing tokens on AO to easily create trading pairs on Permaswap, supporting various market-making curves.

everPay will also deeply integrate with AO, becoming a bridge connecting the AO ecosystem with other ecosystems. EverID aims to become the universal identity layer of the AO ecosystem, addressing the challenges posed by high barriers of public and private key account systems for users entering Web3.

Sam States That Protocol Change Losses Experienced on Other Blockchains Will Not Be Repeated on AO

Changes on Ethereum have previously harmed projects like Aragon, and protocol changes can pose risks to users. There are concerns about building applications on AO, especially when the system is labeled as a testnet, as they worry about the system's stability and integrity. Despite the potential communication challenges this may bring, the team has chosen to clearly express the current state of the system and possible future changes. When building the protocol, the team emphasizes the importance of providing immutable and guaranteed rights to users, which should not be controlled by any individual or organization. To better manage user expectations, the team uses terms like "mainnet preview" and "mainnet candidate version" to describe the current state of the system and to set expectations for possible future changes. When the system reaches a nearly fully determined state, it will be referred to as the mainnet, at which point it will possess appropriate permissionless upgrade and forking capabilities, ensuring the long-term stability of the system and the protection of user rights.

How to Understand the Concept of Sub-Ledger in AO and Its Role?

Sam:

The sub-ledger provides a mechanism that allows the system to increase additional computational threads by creating new processes (i.e., sub-ledgers) when the single-threaded computational capacity of a single process reaches saturation. This approach is similar to creating new tables or shards in traditional databases to distribute the load and enhance overall processing capacity. Sub-ledgers allow computational capacity to be naturally distributed across a decentralized network, meaning that computational capacity can be dynamically adjusted and scaled based on the available computing resources in the network.

The interactions and data transfers between sub-ledgers are designed to be flexible enough so that users and application developers do not need to be aware of the underlying complexities. Users can seamlessly transact and interact across different sub-ledgers as if they were operating on a unified ledger. Tokens between sub-ledgers can be interoperable, meaning that tokens on one sub-ledger can be seamlessly transferred to another sub-ledger, and both can be merged into a unified balance. This design enhances the liquidity of tokens and simplifies the complexity of users managing multiple accounts.

The concept of sub-ledgers makes it more feasible and efficient to build payment networks on AO. Theoretically, this can support the creation of a global payment system where users can easily make payments in any currency from anywhere using their phones. Through the design of sub-ledgers, AO achieves unlimited scalability, meaning the system can expand infinitely with growing demand without encountering the common scalability bottlenecks found in traditional blockchain systems.

Conclusion

Through this in-depth conversation about AO, we learned that the SCP theory is one of the theoretical foundations of AO, SCP applications have unlimited scalability but lack interoperability, while AO originated from efforts to solve interoperability issues between different SCP applications. Later, AO was designed as a parallel computing network with a single image, achieving almost unlimited scalability through multithreaded parallel computing, while AO adopts a single system image, hiding the underlying complexities, allowing users and developers to still interact as if with a single network. In the future, AO will steadily iterate through different mainnet versions, seeking a balance between iteration speed and system stability.