Data Interoperability: The Core Experience of Web3

Author: Chen Li, Jims Young, Li Gong, Ivo Entchev, Youbi Capital

1. The Dilemma of Web2

We live in various apps every day: chatting with friends on WeChat, using review software to choose restaurants, exploring life's diversity on Xiaohongshu, and paying daily expenses with Alipay… Our traces of life are captured by various apps and stored in their respective databases.

At the same time, with increasing demands for work and life, we have more and more specialized apps: from simple videos to long videos, short videos, and medium videos, from blogs to Weibo, long articles, and graphic information streams, the increasingly refined user needs have created more segmented app categories, which also leads to more dispersed data.

This dispersion between apps and the lack of data interoperability creates "data islands." We repeatedly register accounts and create content across different apps, needing to publish on each platform again and again to achieve "cross-platform synchronization."

Imagine if a video we posted on Douyin could be seamlessly synchronized to Xiaohongshu. Similarly, a comment we made on a video on Bilibili could also be synchronized to other platforms. If apps could share user data, data would no longer exist in isolation, and users could leverage more flexible platform interoperability to achieve more creative content output. This is also the frequently mentioned ultimate experience of Web3—an interoperable application matrix.

Faced with the major trend of an application matrix based on integrated data platforms, how will this trend further materialize among the data islands on different platforms? How can Web3 technology serve this major trend? These are the questions we need to discuss.

2. Why Web3

The meaning of Web3 includes concepts such as blockchain, cryptography, and decentralized economic models. Representative products include cross-sovereign currency Bitcoin, decentralized finance based on smart contracts, and decentralized computing and storage networks.

Taking decentralized finance as an example, the main feature of Web3 applications is trustless asset management and trading (signatures). User data exists in a public database, that is, on the public chain. There is no data isolation between applications, and even on most chains, there is completely permissionless interoperability of data between applications. This level of freedom far exceeds that of Web2 platforms, leading to an explosion of decentralized finance applications, with the number of applications in the EVM ecosystem growing 1000 times between 2020 and 2022.

However, in terms of consumer applications, Web3 has encountered significant bottlenecks. The data storage capacity of public chains is very limited and extremely expensive (the storage cost on Ethereum is 1 million times that of AWS), only able to support the data volume generated by decentralized finance-type applications, and cannot accommodate the large-scale data required by other C-end applications such as content and user behavior tracking. The solution for Web3 is to meet these storage needs off-chain and to organize and abstract resources through cryptography to provide them to developers. We will discuss the specific implementation later.

This off-chain storage solution, such as Filecoin and Arweave, distributes data in a segmented and encrypted manner across various servers in their networks, at a lower cost than centralized storage. Unlike the completely open data on public chains, these data require user authorization to be accessed.

Here, we need to mention a key concept proposed in the early days of Web3 applications: the model that breaks the ceiling of application platform development must be users owning the data rather than the platform, and users should also own a part of the application platform, so that the platform can maximize the value of the data while sharing benefits with users. When users own the data, the data isolation between applications can be bridged through user authorization, so the Web3 data platform must be user-centric. Next, we will discuss how to combine the Web3 technology layer, account management, storage, execution, etc., into a user-centric technology stack that is most convenient for developers to use.

3. The Web3 Technology Stack - Starting with DeFi

DeFi applications are relatively slow compared to traditional C-end applications; a transaction usually takes several seconds to tens of seconds or even longer to complete. Applications based on IPFS may take at least a minute, sometimes up to several hours, to synchronize a content update. The speed of these applications is determined by the update speed of their backend databases.

The speed of DeFi is based on the consensus of public chains and is limited by the degree of decentralization of the network; for non-financial applications, content generally only exists on one or a few nodes, and IPFS synchronizes addressing information, which results in longer synchronization times due to higher decentralization of nodes.

To solve this problem, different blockchains have developed their own Layer 2 solutions to store and update data that needs to be processed quickly, and then periodically transfer it to the chain. A similar situation has occurred on IPFS, which is what Ceramic does. Ceramic nodes can be used like centralized clouds to record events occurring on a certain application, and then after a while, update the results to IPFS, so it can be seen as a second layer of IPFS.

With this second layer, the experience of dapps can be very close to that of apps. In addition to dynamic storage, Ceramic also proposes the concept of data models, establishing data standards between applications, making cross-application data interoperability possible.

Dataverse-OS builds on Ceramic to further abstract and isolate resources, creating a kernel that can manage storage resources and identities, similar to the kernel of an operating system, allowing applications and users to authenticate using their public keys to obtain resources without infringing on each other. The benefit of this approach is that all applications can run within the same kernel, rather than as independent systems. Applications do not need to connect with each other directly; they can apply through the kernel and obtain permissions for any data table with the authorization of the user's private key.

From the perspective of data interoperability, Dataverse-OS acts as a cloud operating system, enabling large-scale data interoperability between applications, and is a prototype or MVP of the future Web3 data platform. There are several other projects in this field with similar visions, but we will not list them all here.

4. Outlook and Challenges

Since the birth of Web3, enabling free flow of data between applications has always been one of our major visions. With the continuous maturation of various technology stacks, we are finally almost there.

For users, this is the first time that data interoperability can be achieved across all categories of Web3 applications. We can share comments across various video platforms and discuss the same topics with netizens from around the world without platform restrictions; personal assets can be paid across platforms, and personal social credits and reputations will be widely recognized by all apps; we can even freely manage our own data, allowing it to flow freely between applications and users, achieving more breakthroughs in composability.

For developers, the barrier to entry for traffic has never been so low, making it easy to access and utilize the information users open up. Data is no longer a competitive barrier between applications; good products are what matter. At the same time, the development threshold has also been further lowered. With the continuous popularization of no-code tools, the front-end barrier is constantly decreasing, and as long as it connects with the Web3 backend, an application can be born, and all of this is within reach.

In Web2.0, cloud-based operating systems are becoming the most sticky user entry points, with more and more functions integrated into cloud operating systems, and more user experiences being migrated to the cloud. This is an unstoppable trend. User-centric cloud services will inevitably compete with centralized platforms for developers and users; we believe that in terms of content and social applications, user-centric cloud service platforms have undeniable advantages.

The design of Dataverse-OS is simple yet powerful, providing developers with all the core functions to achieve data interoperability. We are eager to recommend their SDK to all dapp developers. However, we face a challenge: users are the main factor for platforms to attract developers. How to attract early developers through a win-win approach is the biggest challenge for the data platform. Our answer here is a project that serves as a public data platform, Glitter, which we will introduce in detail in the next article, "Glitter: The Locomotive of Web3 Traffic."