Understanding StarkWare in One Article: The L2 Solutions Behind dYdX and Immutable

Author: msfew, Foresight Ventures

Introduction to StarkWare

StarkWare primarily addresses blockchain scalability and privacy. StarkWare has developed a complete solution using STARK technology, generating and verifying proofs of computational integrity through zk-Rollups and Validium modes that compose Volition. StarkWare's cryptographic proofs can achieve zero-knowledge, succinctness, transparency, and post-quantum security. The main products developed by StarkWare include: StarkNet, StarkEx, and Cairo.

Key Features

• Scalability: StarkWare allows blockchains to scale massively by relying on cryptographic proofs generated by off-chain provers in the cloud, which are then verified through on-chain smart contracts.
• Privacy: zk-STARK proofs do not reveal private information about the inputs. Combined with other cryptographic tools, zk-STARK can completely and provably protect user data.
• User Experience: Through examples like Immutable and dYdX, we can experience the strong scalability provided by StarkWare's solutions. DiversiFi's payment TPS can reach 18k, Immutable's NFT minting costs only 0.2 cents, and dYdX's trading fees have been reduced to 1/50. Instant confirmations, with fees almost at 0, provide an excellent user experience.

Milestones

• 2018: STARK white paper published, Ethereum Foundation Grant

• 2019: First demo released (200 times Ethereum efficiency), StarkEx Alpha launched testnet, second demo released (700 times Ethereum efficiency)

• 2020: DeversiFi (StarkEx 1.0) launched mainnet, VeeDo (Stark-based VDF) launched mainnet, StarkEx Rollup launched mainnet, ethSTARK released, Cairo (Turing-complete language for STARK) and its Playground launched, Ziggy STARK (post-quantum secure signature) released, StarkEx 2.0 launched mainnet

• 2021: StarkNet announced, dYdX and Immutable X (both StarkWare clients) projects launched mainnet

Immutable and StarkWare

Immutable is the first Layer 2 NFT trading platform. TikTok partnered with Immutable in September to launch a series of NFTs.

Due to Ethereum's native throughput limit of 4 transactions per second, any DApp hoping to scale is constrained. For NFTs, the throughput bottleneck is even greater than for regular tokens: using NFTs, a thousand tokens actually consume blockchain resources a thousand times more than a single token. Immutable aims to enjoy Ethereum's security, development tools, and network effects with a native Ethereum solution, thus adopting the solution provided by StarkWare.

One of the unique features of the solution provided by StarkWare is its ability to be deployed in multiple data availability modes: zk-Rollups or Validium. Both are based on validity proofs, but in zk-Rollups, all data is submitted on-chain, while in Validium, data is stored off-chain, and only the submission of the latest state and the on-chain submission of the validity proof of that state are made. Immutable started with the Validium mode to provide lower gas fees. Due to regulatory requirements, the Data Availability Committee (DAC) ensures that users always have access to their data. Therefore, Immutable switched to the new data availability mode of Volition, which allows users to choose the storage location of data at a single transaction granularity—on-chain (zk-Rollups) or off-chain (Validium). This approach also maximizes performance scalability.

In the blog post about TikTok and Immutable's collaboration, TikTok specifically mentioned that StarkWare is the first carbon-neutral L2 scaling solution. From the perspective of traditional internet companies, environmental protection is very important, and being environmentally unfriendly can easily lead to criticism. Therefore, the high performance and resource-saving characteristics of L2 can also attract the attention of traditional internet companies, paving the way for them to enter the cryptocurrency space openly.

The solution StarkWare provided to Immutable ultimately led TikTok, currently the hottest company, to collaborate with Immutable. The two deployment modes offered by StarkWare give clients flexibility in data storage methods, complying with regulations while also allowing for performance scalability. More importantly, the scalability of performance also addresses Ethereum's controversial energy consumption issue, marking a victory for Layer 2 and a victory for StarkWare. In the future, we will certainly see more traditional enterprises choosing Ethereum and StarkWare's approach to enter the blockchain space.

Why do StarkWare applications have such high performance?

StarkWare's Prover incorporates various mathematical optimizations and some optimization algorithms first proposed by StarkWare, while the Cairo language used for development has specific mathematical optimizations. Additionally, the interactive data is coordinated by the StarkEx engine before being sent to the Prover, processing the data to be proven and batching it. The entire operational process is fully optimized. Specific details will be discussed in detail later.

What is the degree of decentralization of StarkWare applications?

The consensus on StarkNet is zk-STARK. zk-Rollups are not necessarily decentralized and permissionless. However, the zk-STARK used by StarkWare is permissionless, just like public chains such as Ethereum. In the intermediate processes of the applications made by StarkWare, there will be some centralized servers providing certain services. But this is necessary and cannot be eliminated in the development of a complete application. Just like Uniswap must have a centralized domain name and frontend. Therefore, StarkWare's dYdX, StarkNet, etc., are still decentralized.

Comparison of zkSync and StarkWare

Currently, StarkWare is leading zkSync in terms of performance and current operational status. The biggest difference between zkSync and StarkWare is still their operational philosophy. zkSync's projects are all open-source, and the team claims that it doesn't matter if they are replaced, as long as it promotes the development of the community and Ethereum. StarkWare operates in a centralized manner focused on B2B, and the STARK prover can currently only be used by StarkWare, and developing the Cairo language is not necessarily beneficial for the Ethereum ecosystem (a developer-friendly approach should be to make it EVM compatible like zkSync or Optimistic).

zk-Rollups and STARK Proofs

• The technology used by StarkWare includes zk-Rollups and zk-STARK. These two are not the same thing. STARK refers to zk-STARK, which is a type of zero-knowledge proof.
• StarkWare prefers to call its solutions Validity Rollups, as the solutions do not necessarily use zk-Rollups. Validity Rollups include Validium and zk-Rollups. The difference between the two is whether user data is on-chain.

STARK improves scalability by allowing developers to move computation and storage off-chain. Off-chain services will be able to generate STARK proofs, proving the integrity of off-chain computations. These proofs are then put back on-chain for any stakeholders to verify the computation. Using STARK moves most of the computational workload off-chain, allowing existing blockchain infrastructure to scale exponentially while maintaining computational integrity.

The general process of on-chain data for Ethereum: 1000 transactions sent to each Ethereum node → each node reads 1000 transactions → nodes update 1000 transactions

The on-chain process for zk-STARK data: 1000 transactions sent to zk-STARK → zk-STARK acts as a prover, generating 1 proof (*generation phase) → nodes as verifiers read the proof and update

• Generation phase: Typically, zk-STARK proofs require the following process to generate the proof. The proof process for the verifier is divided into two steps: the first step is to generate an algebraic intermediate code representation after the computational integrity statement undergoes arithmetic, and the second step is to generate FRI (an optimization method for proofs described in a paper by StarkWare Co-Founder in 2017) after testing, followed by outputting a scalable, publicly verifiable knowledge argument (i.e., STARK) through cryptographic algorithms. In short, it generates a trustworthy and high-performance proof through a secure and reliable environment and optimization algorithms.

Comparison of zk-Rollups and Other L2 Solutions

• Strictly speaking, the terminology used in many of StarkWare's documents regarding zk-Rollups does not necessarily adopt zero-knowledge proofs; the correct description is Validity Proofs (which include zk-Rollups and Validium, as StarkWare provides two modes). To avoid confusion from terminology changes, this article will refer to it as zk-Rollups in accordance with StarkWare's documentation.

Sidechains (Polygon or xDai)

• Advantages: Even if users make a large number of transactions, only two occur on the main chain, namely deposits and withdrawals. Since transaction costs on the sidechain are lower than those on Ethereum itself, scalability can be achieved.
• Disadvantages: Granting the power to create new blocks can lead to the power to stop block production. Sidechains can have the power to prevent users from withdrawing funds.

State Channels

• Advantages: An example of a state channel is the Lightning Network. State channels are very useful when participants will exchange a large number of state updates over a long period. They have strong privacy since the states only occur within the channel. They have instant finality, ending immediately once concluded.
• Disadvantages: State channels are highly dependent on validity and are more suitable for applications with a defined set of participants.

Plasma

• Advantages: Plasma is very suitable for handling NFTs, as each NFT has a unique ID. One of the issues with sidechains is that the consensus mechanism can stop block production or even lock user funds; Plasma users can call the Block root, so if the consensus mechanism stops creating blocks, users can still claim funds from Ethereum.
• Disadvantages: If many users exit their Plasma chains simultaneously, they may overwhelm the root chain and congest the network. Issues such as fraudulent activities or network attacks could lead to such a massive outflow. Another disadvantage is the lack of complexity; users cannot perform the same types of complex operations as on sidechains and cannot simulate a complete Ethereum environment.

zk-Rollups

• Advantages: The fees for each transfer are very low. Faster than Plasma and Optimistic Rollups. Less data leads to higher throughput and scalability.
• Disadvantages: Initial setup is not decentralized (there is no initial setup in STARK), quantum computing may pose a future hacking threat (STARK can resist quantum computers).

After comparing various L2 solutions, we can find that zk-Rollups may be the closest to a perfect solution for blockchain upgrades and scalability. At the same time, the few shortcomings of zk-Rollups will be addressed in the zk-STARK adopted by StarkWare.

Comparison of STARK with Other ZK Performances

There are three main types of ZK proofs: SNARK, STARK, and Bulletproof. Bulletproofs and STARK do not require trusted setups, and STARK uses publicly verifiable randomness to create an untrusted verifiable computing system. Among them, STARK has a much larger proof size compared to the other two, but has significant advantages in computation size and proof speed. At the same time, STARK's security is higher and can resist quantum attacks.

The above image was presented by StarkWare's Co-Founder during a presentation. The STARK proofs used during their testing ran on the server configuration shown in the image, outperforming several others, including SNARK algorithms.

StarkNet

StarkNet Alpha will launch on the Ethereum mainnet in November. (Blockchain Explorer)

Introduction

StarkNet is a decentralized, permissionless, and censorship-resistant L2 zk-Rollups that supports general computation on Ethereum. It is based on the Turing-complete Cairo language. Developers can build applications to implement business logic and deploy them on StarkNet; users can send transactions to StarkNet in the same way as normal transactions on Ethereum. StarkNet nodes and participants will be economically incentivized to ensure the network operates efficiently and fairly.

All StarkNet transactions will be regularly batched, with transactions proven in STARK proofs, ultimately verified on Ethereum. Since the computational workload required to verify STARK proofs is exponentially smaller compared to Ethereum's proof workload, StarkNet can scale Ethereum by several orders of magnitude. Since all StarkNet state transitions will undergo STARK verification, Ethereum will only accept ultimately valid state transitions.

Main features: performance scalability, development convenience, and composability for the Ethereum ecosystem.

Network Ecosystem

• Programming Language Cairo: Cairo is a Turing-complete high-level programming language and framework that can be used to generate STARK proofs. Developers can use Cairo to define any business logic, send transactions to the Shared Prover Service for off-chain verification, and then verify on-chain. Cairo is now suitable for formal project development on the mainnet and is available for developers to use.
• Off-chain Verification Service Shared Prover Service: Before transactions are on-chain, they are sent to Sharp for verification, determining the integrity of application execution and calculating the gas costs for proof verification.
• Open-source Prover ethSTARK: ethSTARK provides zero-knowledge and post-quantum secure signatures. In one proof on the mainnet, ethSTARK handled 300,000 transactions, achieving a TPS of 3000. Rollup gas efficiency also broke the world record, reaching 315 gas/tx, making it several orders of magnitude cheaper than transactions on Ethereum L1. This technology is the cornerstone of StarkNet's decentralized proof layer.
• Third-party Projects: Currently, the main third-party library is StarkNet.js. StarkNet.js serves a similar purpose to Web3.js, used for frontend access to contract content. The Ethereum client team is working on a complete node implementation for StarkNet. Currently, StarkNet network auditing software includes CryptoExperts and LEAN proof.

Use Cases

1. DeversiFi: Trading TPS 9k, Payment TPS 18k.
2. Immutable: NFT minting gas 0.2 cents, daily L2 transactions > weekly L1 transactions.
3. dYdX: Transaction fees reduced by 50 times.

StarkEx

Introduction

StarkEx is like an underlying engine that constitutes components to support applications using STARK.

StarkEx is an L2 scalability engine suitable for production environments, deployed on the Ethereum mainnet since June 2020. StarkEx leverages STARK technology to provide scalable self-custodial trading for applications such as DeFi and gaming. StarkEx supports a wide range of use cases: spot trading, perpetual trading, and NFT minting and trading. Before mainnet deployment, over 50 million StarkEx transactions were settled on public and private Ethereum testnets.

StarkEx currently supports ETH tokens, ERC-20, ERC-721 tokens, and synthetic assets. Additionally, it can support tokens on other EVM-compatible blockchains.

Features

• Scalability and computational integrity: Applications can rely on StarkEx and the underlying STARK infrastructure to scale, while StarkEx ensures computational integrity. Through SHARP (Shared Prover) technology, gas costs are shared across all StarkEx-based applications.
• Diverse deployment modes: StarkEx can be deployed in zk-Rollups mode (publishing data on-chain) or Validium mode (validating data availability through sidechains). There is also an upcoming mode: Volition, which allows users to choose their data availability mode.
• Wallet integration: StarkWare provides native support through Ledger, Authereum, and WalletConnect. StarkWare also offers MetaMask integration through Web3-compatible key generation (suitable for many other existing wallets).

Architecture and Business Logic

The above image is a typical application architecture diagram using StarkNet and StarkEx. Assume this application is a DEX called Starkswap.

• The application in the leftmost yellow box ------ operates the entire application: this off-chain component receives user transactions and defines business logic and execution order. It passes transactions to the StarkEx service. This component is the centralized frontend of the application, typically a web or client interface.
• The light blue box containing the StarkEx service ------ batching and coordination: an off-chain component responsible for batching a set of operations and updating the system state based on those operations. For each batch, it sends this batch of operations (executed through the Cairo language, first converting Solidity bytecode into Cairo language smart contracts, then running in the Cairo language environment) to SHARP to prove its validity. Once the proof is verified, it publishes the new state on-chain. The state is represented by a Merkle tree. The structure and content of each Vault vary based on the specific business logic implemented. The Merkle tree root represents the state to be submitted to the chain.
• SHARP ------ Shared Prover (customized by StarkWare for applications): SHARP is a shared proof service for the Cairo language. It receives proof requests from different applications and outputs proofs to demonstrate the validity of Cairo code execution. The output proofs can be shared among multiple proof requests. This component was previously named Generic Prover Service. Therefore, the component is essentially a backend server containing specific services. It is speculated to be deployed on StarkWare's centralized servers.
• StarkEx contract (Verifier contract) ------ state updates, deposits, and withdrawals: This contract has two main functions. The first function is to update the system's state after verifying that validity conditions are met, and the second function is to manage deposits and withdrawals into and out of StarkEx in a non-custodial manner (in any case, users can withdraw their funds). This component is a smart contract on Ethereum.

The business logic of Starkswap is as follows:

1. Processing requests:

The StarkWare application in the light blue box processes customer requests, converting these transactions into provable statements and sending them to the dark blue box SHARP. All requests sent and processed are transmitted through the API in the first step of the diagram and the batching statements in the second step (one batch can compile multiple requests).

2. SHARP proof:

The dark blue SHARP constructs the proof.

3. Registering the proof:

SHARP transmits the proof to the pink box's on-chain verifier smart contract in the third step.

4. Registering state transitions:

In the fourth step, the StarkWare application performs state transitions on the Starkswap smart contract.
In the fifth step, to ensure the validity of the transition (SHARP's verification), Starkswap's smart contract confirms that the transition statement has been registered in the Verifier smart contract.

Functions and Cryptographic Algorithms

https://docs.starkware.co/starkex-v3/crypto/stark-curve

Function details:

• Off-chain state: balance tree and order tree. Through the Merkle tree, the security and trustworthiness of fund states are guaranteed, as well as the avoidance of double-spending issues.
• Flow Description:
• On-chain accounts (the balances of accounts are stored and managed off-chain unless funds are moved in/out of on-chain accounts. From the perspective of L1, the funds of these accounts are stored under the StarkEx smart contract.)
• Off-chain accounts (all balance instructions arrive as L1 transactions. A special property of on-chain accounts is that they do not require a private key (stark's private key), so the smart contract is the "owner" of such accounts. Therefore, L1 users and applications can trade through StarkEx and match with any orders. The on-chain transaction model achieves interoperability between StarkEx applications and L1 applications, usable for DeFi Pooling or by liquidity aggregators).
• StarkEx specific concepts: There are many concepts in StarkEx that differ from Ethereum, so additional learning is required for development.
• Additionally, StarkEx has specific operations for information encryption, batch flash loans, DeFi pooling, etc.
• Cryptographic algorithms: STARK Elliptic Curve, Pedersen Hash Function, KDF. By using the above cryptographic algorithms and references, third-party wallets can integrate StarkEx.

Cairo

Introduction

Cairo is the first production-grade platform for generating STARK proofs. It is Turing-complete and highly efficient. To deploy high-performance and low gas fee Layer 2 application components on StarkNet, Cairo must be used to write applications. The Cairo language is not a specialized contract language but can and is primarily used to write contract-like content.

The current drawback of zk-Rollups is general computation: directly porting smart contract logic into rollups is more challenging, so only limited functionality is available, such as transfers and transactions. However, Cairo fills this gap by allowing smart contract logic to be directly ported into rollups. zkSync and Optimistic's approach is to make rollups EVM compatible and EVM equivalent (more comprehensive and further EVM compatible).

Features

1. Computational Integrity

Computational integrity represents the assurance of fairness in computation without regulation. There are five methods to ensure Computational Integrity: bank delegation accountability (based on reputation, audits, etc.), computational replayability (e.g., Bitcoin and other L1s publicly disclose all transactions), trusted execution environments (e.g., Intel's SGX), fraud proofs (Optimistic rollups, which lie between delegation accountability and computational replayability), and cryptographic proofs (zk-STARK). The Cairo language has been validated through multiple papers and mathematical proofs to securely and reliably generate zk-STARK proofs to ensure computational integrity.

2. AIR (Algebraic Intermediate Representation)

In the STARK proof process, many operations and symbols for designing algebraic proofs are needed. Cairo will convert the programming language into Cairo bytecode through a compiler, generating an AIR that contains all mathematical proof steps, ultimately producing results. The Cairo language has accompanying AIR visualization tools to view details within proofs.

3. Language Design

The design of the Cairo language follows the principle of Minimal Variable Language, emphasizing just the right amount. It achieves expressiveness (easy for developers to read) and efficiency in generating STARK proofs. The design of the Cairo language is not overly complex, with only three registers (PC, allocation pointer, frame pointer) and a concise instruction set. The Cairo language also supports necessary features, including functions, recursion, branching, and conditionals. The design of the Cairo language feels more aligned with the logic of mathematical proofs, making it more coherent.

4. Toolchain

Compiler, virtual machine, code editing environment plugins (code suggestions plugin on VSCode), tracing programs, code examples.

Language Ecosystem

Currently, the Cairo language ecosystem can be described as very comprehensive but young. In terms of toolchain, it is fully equipped, but based on the GitHub star count of these tools, it seems to have not garnered much attention. However, the repository developers are continuously updating, with new commits generally occurring every one to two weeks. In the future, the Cairo language will become more open, and after cultivating more developers, the work of writing Cairo code on the StarkEx engine is likely to be transferred from StarkWare to open-source developers, which will have significant implications for the decentralization of StarkWare applications.

More importantly, OpenZeppelin is implementing basic templates for Cairo language ERC-20, ERC-721, etc. implementation. The significance of this implementation is that it allows developers to build on this foundation, enabling them to confidently and boldly copy this implementation, rapidly develop, and not worry too much about security. From the repository's commits, it appears that OpenZeppelin currently has dedicated personnel supporting the Cairo language, demonstrating OpenZeppelin's emphasis on the StarkWare ecosystem.

Conclusion

As a Layer 2 solution provider, StarkWare has produced blockbuster products like dYdX and Immutable. Among various solutions, StarkWare's zk-STARK technology supports the largest transaction volume on Layer 2. From the mathematical optimizations of the Prover, the mathematical optimizations of the Cairo language, to the proof coordination of the StarkEx engine, StarkNet, as a decentralized zk-STARK network, provides extremely high performance and achieves complete optimization of the operational process. StarkWare, through cutting-edge technology and comprehensive technical services, supports the massive transaction volume of Layer 2 applications and will undoubtedly embrace open-source in the future, achieving greater decentralization and aiding in the construction of Ethereum 2.0.