Title: Into the Dataverse

Source: Ceramic Blog

Compiled by: Evelyn, W3.Hitchhiker

There has been a lot of hype around the Metaverse, which is a vision for our digital future that is increasingly merging with the digital future of Web3. In this article, let’s steer clear of the claims about the Metaverse made by Facebook, Meta, or the NFT tribes. We believe the Metaverse is the whole of all composable and interoperable resources, identities, applications, platforms, services, and protocols that exist in cyberspace.

Our vision for the Metaverse will run on the Dataverse: a composable, network-scale data ecosystem owned by everyone but not exclusively owned by anyone. Developers will build interchangeable interfaces and online experiences that interact directly with the Dataverse and the composable data running within it. It can be said that the Dataverse plays the most crucial role in the Metaverse: a shared, high-performance, always-available graph of all the data created and used by all applications.

Most functionalities on the web are merely data rather than currency value transactions. For example, DeFiverse refers to the universe of interoperable financial protocols, assets, and applications; however, just think about the applications you use every day. How many times do you click "send" or "like" compared to "buy" or "transfer"? In the Dataverse, your data follows you from one application to another, placing you at the center, allowing you to control your own digital universe.

In the next five years, we will see a massive surge in the developer market, along with the full integration of permissionless protocols with mainstream development stacks, resulting in a billion network applications running and interoperating on shared, composable backend infrastructure. This equates to one application for every ten people on Earth.

We want to take you on a tour of the Dataverse ------ to see how we move from today’s Web2 application data silos and today’s Web3 application data deserts towards a future where everyone is building applications on the same networked, composable data layer.

The Golden Age of Application Development

The road to an interoperable Metaverse depends on supporting developers. The next five years will be the golden age of application development.

Everyone is a "Developer" of Applications

Developing applications will become so easy that in five years, anyone in the world, regardless of their location, technical ability, or education level, will be able to build and launch an application on the internet. Essentially, everyone will be able to become a "developer." For example, even a non-technical journalism student could launch a fully functional crowdsourced intelligence forum in a few hours.

Goodbye, Backend.

The trend of everything as a service will abstract away most or all of the backend application development logic, enabling frontend developers to launch applications with zero infrastructure. This infrastructure can be provided by decentralized networks or managed service providers.

Open, Shared Platforms

By this time, decentralized permissionless networks will be widely adopted by application developers, with some forward-thinking Web2 companies joining in. The main benefit of building on these platforms is that, in addition to providing zero-infrastructure deployment, it also makes collaboration and interoperability within the Metaverse feasible. This addresses many challenges of traditional application development, including risks of centralized platforms (read: Twitter shutting down APIs, Facebook shutting down game developer platforms), onboarding new users, content curation, building network effects, creating delightful user experiences, and more.

Copy / Paste Frontend

Two main trends are driving the minimization of frontend development. First, low-code/no-code solutions are enabling non-technical individuals to create applications more effectively and easily. Second, due to the nature of building on open platforms and users wanting increased transparency, more application code is being open-sourced at a faster rate. Combined with zero-infrastructure backends, this means that essentially every application can be fully inspected, sampled, forked, cloned, remixed, modified, and redeployed. Creating applications becomes so easy that it looks more like copying/pasting someone else's frontend code or throwing modular snippets that work together onto a canvas.

Global Access, Equal Opportunity

By launching satellite-based 3G+ internet transmission, connectivity around the world will improve, allowing a new generation of developers to go online for the first time. This connects more people through cyberspace, encouraging those inclined to create something in the global market. The internet will be filled with free educational materials ------ all that is needed is a bit of curiosity and initiative.

Strong Incentives and Direct Rewards

If there are two ways to build the same thing, all else being equal, people will want to work on the one with the most promise. Web3 provides strong native participation and builder incentive mechanisms that are compelling and direct for developers from all backgrounds. Large tech companies will struggle to compete for talent, as the potential for rising from 0 to 1 (or joining the Web3 rocket) will far outweigh the allure of working at companies like Amazon. The Metaverse as a whole will compete for talent.

Power to the People

The organizing principle of the Metaverse will be the individual. When developers build on this shared application architecture, as it better suits their applications (user-centric composability), it opens up a new world for end users, where they can regain control of their digital identities and start experiencing the web as if every application were built specifically for them, with their data magically appearing where they want it to be.

Self-Agency and Determinism

In the Metaverse, individuals are choosable. Based on public key cryptography, users in the Metaverse do not need permission; they can start their lives with a simple, self-sovereign, pseudonymous key pair. From there, they can decide who they want to be. They can let others control their identity for them. They can become their own bank and practice self-custody. They can start attaching as much or as little data as they want. They can set permissions dictating who can do what with their data under what conditions. This level of digital ownership and self-agency is how the Metaverse remains independent and secure.

But self-sovereignty is not limited to a single account. Users will also own trusted independent infrastructure to maintain their data long-term, so they never have to rely on others.

The Allure of Interoperability

If one of the main attractions of building in the Metaverse is interoperability, then applications will naturally be incentivized to adopt the minimum foundational standards required to achieve that interoperability. The simplest and most intuitive way to achieve interoperability is through adopting user-centric architectures. In this model, users act as the link for data interoperability between different applications; wherever they go, their data goes.

For end users, the experience of interoperability feels magical, just as the internet should work. No more passwords, forms, verifications, re-posting content, or re-following the same person across platforms.

The Explosion of Knowledge, Science, and Human Progress

As a public resource, the Dataverse will unleash an explosion of shared knowledge, accelerating science and human progress. The initial use case of the internet 1.0 was sharing scientific databases between California and Massachusetts. Web3.0 will adopt this concept and supercharge it.

So far, the implementation of the semantic web and linked data has been great for infographics, but has only been deployed in the internal systems of the world’s largest companies and institutions (like Google) in production. They are hard to build and require many of the smartest minds.

The Dataverse will provide access to a simple, open, semantically defined structured data graph that leverages cutting-edge research and allows everyone to build and collaborate on it. When we can connect the entire scientific community on a unified verifiable data platform, the significance of the progress we can achieve cannot be overstated, but we do not need to make unnecessary forward-looking predictions about what the end result might be.

The Creativity of a Billion Networked Applications

It is known that when everyone in the world is building applications, and they are all building on the same networked, composable data layer, it will unleash diversity, creativity, originality, and innovation ------ let’s call this network supporting the Dataverse a platform.

As more developers join the platform and more data exists on it, it will become easier for developers to create applications. The startup costs will drop to nearly zero. This allows for diversity in implementation and experimentation, increases resilience, and lets Darwinism chart the path forward. It allows developers to piece together scalable micro-apps in minutes and focus on product-market fit rather than spending weeks delivering a cumbersome monolith. It provides a new vehicle for everyone to compete with Facebook or Google. In this case, the speed and originality of the marketplace will surpass that of the cathedral.

High Stakes

Defining several trends that will shape the Dataverse and the future of the network.

Explosive Growth of Data Functionality in Web3 Applications

The Web3 developer market contains a collaborative group of value-aligned innovators/early adopters (some of whom are already building in other parts of the Metaverse) who will be the first to adopt the Dataverse. So far, Web3 applications have been data deserts. This makes sense, as there is only so much data functionality you can build with smart contracts.

This trend is changing and will continue to accelerate as the market becomes more crowded/mature, with applications seeking new innovative horizons. As the market demand for building data functionality increases, applications will begin to collaborate on large datasets, the network effects of the Dataverse will rapidly accelerate, and the Web3 application market will be filled with data-driven functionalities. Others outside this market will take notice of this ecosystem and be compelled to join.

Interoperable Ecosystems Will Outperform Isolated Platforms

The perfect cocktail is brewing to make the Metaverse even surpass the largest, most entrenched institutions. No one wants to build their applications on islands or for big companies anymore. The simplicity, creativity, pace of innovation, development speed, open data, and rewards in the Metaverse will be too powerful. Those who have built their businesses on a completely different incentive structure will find it difficult to pivot or compete.

Identity Arises from Community Needs, the Organizing Principle of the Dataverse

Identity is inherently multi-person. This is true not only in the physical world, where identity is initially a structure that enables interaction between people, but also in the digital world, where having a profile on social media applications is more important than having one on your personal note-taking application. Identity is a tool that provides context, trust, and efficiency for transactions. Ceramic is a platform that, by providing an inherent multi-person data ecosystem, we can expect to generate a widely adopted identity system that can provide greater interoperability for the Metaverse.

The Effectiveness and Sustainability of Digital Communities

Irrespective of technology, related to sustainability, we are betting on the fact that strangers on the internet can effectively organize and create high-impact work through community-owned protocols and their inherent incentive mechanisms. While there are positive signs for this trend, whether these communities can sustain themselves long-term or maintain innovation and high velocity remains to be seen. The design and implementation of effective digital communities are things we are closely monitoring, but we also believe that many elements that can make communities successful long-term are yet to be discovered.

Some individuals have fully entered the state of "working for the internet," contributing to protocols and DAOs, taking on one-off tasks or participating in long-term contracts. We already have DAO participants lobbying in front of Congress. DAOs are recognized as material world entities under Wyoming law. Digital communities have long been used for OSINT (open-source intelligence), where a group of highly motivated strangers gathers on topic-based platforms like subreddits or Facebook groups to do things that no single person (or government agency) could do ------ like crowdsource villains or solve investigations. Wikipedia runs very well. All of this makes me optimistic about how this fluid work and incentive system will evolve over time.

There are positive signs that these communities are beginning to cross the chasm into the mainstream, which will only deepen their tentacles in society, making them more resilient. The Bored Ape Yacht Club (BAYC), where basically every celebrity has an ape as their Twitter profile picture; Alexis Ohanian and Serena Williams wearing their CryptoPunks badges at the Met Gala. These digital-first communities will spread across all aspects of society, with influential people and celebrities leveraging their talents to represent their tribes.

To understand where Web3 (relative to NFTs) will cross into mainstream culture, we should look at where there are highly engaged multi-user community networks, what it means to be part of that community, and how people fly their flags.

Ceramic: Decentralized Infrastructure Built on the Dataverse

While the market may be ready for the Dataverse, that does not necessarily mean it will emerge naturally. The best developer platform for building applications in the Dataverse requires some key design considerations.

Universal Data Protocols

Data is highly diverse, and developers need freedom and flexibility to build their applications and data use cases. Rigid standards will not work or will lead to fragmentation. We need a highly universal protocol capable of supporting the diverse use cases that will emerge in the Dataverse. Importantly, all these use cases should interact seamlessly, and the simplest way to achieve that is to have them built on the same protocol. It should not matter which wallet users log in with, what functionalities you build on the frontend, or what type of data you want to store; they should all work seamlessly.

Fast, Decentralized, High-Capacity Processing

This universal data protocol needs to support the variability of fast, decentralized, high-capacity transaction processing, simply because so many data events occur in each application every day. Current blockchain protocols cannot scale to support this level of processing, which is a powerful area of differentiation. Data consensus properties, like those of Ceramic, can be treated separately from value consensus properties (like Bitcoin’s).

Globally Consistent, Always-Available State

Next, it is important that the system has a way to provide participants with a highly available state that can be verified and reconstructed from the underlying event stream without trust. This is crucial because it is what allows developers to build permissionless, composable applications on shared data. Anyone inserting into the underlying event stream can ensure the availability of the state, as updates are published through this system, and all participants can immediately update their state databases.

Related to global state availability is state representation. Ceramic currently supports state outputs in the form of simple JSON objects and local indexing systems, which works well for graph-based application use cases. The protocol should be able to adapt and support different state representations, optimized based on the data structures that are more suitable for those use cases. This can be achieved by inserting Ceramic’s event stream layer and building custom databases on top of it.

Object-Oriented Permissions

Every piece of data in the Dataverse should be verifiable, secure, and useful ------ not spam or data bits that clog the network. By using decentralized, encrypted, object-oriented permission protocols, we can allow decentralized identities to have provable agency over their data while allowing access permissions for each state (even nested/recursive/chain permissions for groups of states) to be fully customizable.

For example, a user could grant their therapist, their school counselor, and their best friend permanent access to their diary. Or someone could empower an application to update their status on their behalf during a session. It can also natively integrate with different cryptographic authentication systems used within developers’ applications (for example, logging in with Ethereum). We must collaborate with other Dataverse protocols and services to adopt the same permissioning standards to achieve maximum interoperability. In many ways, the more powerful, developed, and flexible the permission protocol, the less important it becomes to bind an identity to any single key.

We should also expect a lot of innovation in read/write permissions within the developer ecosystem on the network. We have already seen experiments with programmatic permission protocols along these lines. Examples in this area include hosted regulatory agents or "personal data stewards" or something akin to "credit bureaus" to manage user data permissions. We should encourage a wide variety of diversity, especially in the early stages.

Fast, Expressive, Full-Network Queries

Whenever a user logs into an application with their wallet, that application needs to collect data about that user from the Dataverse. This is easy when the application wants to retrieve a single state for an identifier they already know. However, application developers need more data retrieval capabilities than just using their specific machine-readable identifiers to get content.

When this application is multi-user, such as a social network, developers need to collect datasets from different users and display them in one interface. This type of data retrieval functionality is at the core of almost any application, especially for those applications that are most valuable to the entire data ecosystem. When a user’s identifier is known, it can be achieved by looking at the data index for each user present in the Dataverse and then pulling in that data. This assumes the existence of a user-centric indexing protocol, whether as a single list physically stored somewhere in the Dataverse or using metadata and indexing services to maintain.

However, applications do not always have the full picture about users ------ just having a wallet address does not tell the whole story. Broader social graphs of users participating in games or ranking in games can provide contextual information.

Developers building on Ceramic will have the ability to use indexing and queries to extract selective data from users of other applications within the Ceramic ecosystem.

Ceramic will soon provide a graph database that allows developers to explore the connections from any user’s account to data streams, such as all blog posts. But in further iterations, Ceramic will allow relationships between any accounts, models, and data streams associated with those models. This will allow for infinitely complex queries across the entire Dataverse.

Open Data Model Marketplace

The Dataverse needs interoperability. This is achieved through urgent consensus on data models. Data models on Ceramic are similar to Ethereum’s ERC contract standards ------ for example, ERC20 for tokens, ERC721 for NFTs ------ they are essentially equivalent to Ceramic’s functionalities. This allows any developer to easily select from a pre-existing set of functional templates, add a few lines of code on their frontend, and have an application that can access all datasets in the Dataverse that conform to the same data model.

Communities can build and curate the world’s largest data model universe. The most powerful way to create good enough standards to make it a vibrant marketplace is to give the community ownership and upward mobility.

Developer Community

The Dataverse does not belong to anyone, nor does it belong to everyone. It is a product of a community where everyone is working to build creative, networked applications, thus creating valuable resources for the world. This large project can only succeed with a massive, participatory community behind it.

Community members can help build developer tools and stack ecosystems, grow and influence, create educational content, network operations, and more. We must carefully consider the incentives for all ecosystem participants so that we can maximize the deployment of this community and make it effective.

The Path Forward

We will first focus on providing rich data application capabilities for advanced Web3 developers, using data models to build within a highly networked application ecosystem ------ the market and use cases that are best suited for Ceramic. Each stage will seek signals on what infrastructure to use and how to build from the previous stage, which is why it is so important to fully focus on today’s innovators.

Unmet Needs of Web3 Developers

The Metaverse is emerging from the blockchain foundation; it has all the right conditions mentioned earlier to become a thriving, vibrant interoperable application ecosystem. Web3 developers are already building applications on the blockchain, and many of them want to differentiate and mature their products by expanding into the data space.

Reputation Functionality

Reputation functionality is needed across many different Web3 multiplayer applications, decentralized protocols, and social communities, covering everything from profiles, assets I can prove ownership of (NFTs, domain names, social handles), experiences (self-asserted and verified skill trees), contribution tracking, KYC VCs (verifiable credentials), and more.

Some of these use cases include venture capital, but many do not. Within this group, Ceramic can serve public and ephemeral private data use cases. Fully private use cases should be encrypted and stored on an access-controlled server, ideally using IPLD and CACAO, so that it can interoperate natively with data on Ceramic ------ like Spruce Systems’ Kepler.

In the long run, the functionalities of these datasets will act as a person’s identity and passport in the Metaverse. The stakes are high. For example, the future of DeFi is very unstable until reputation and credentials are clarified.

Social Functionality

Social is very hot in Web3, with a lot of momentum building around decentralized social graphs and decentralized social media. We see the appeal and expansion of Orbis, Convo, and CyberConnect, entering different iterations and experiments. These may emerge as fully decentralized social networks or as social functionalities embedded in other applications. Expect social graphs, feeds and posts (blogs, microblogs, etc.), reactions, private and public groups, DMs, forums, and address books.

Account Features

Many applications and protocols are asking themselves how to identify a user when they have many different wallets or accounts. This becomes increasingly important for Web3 applications that will integrate more and more Metaverse protocols into a single application. Maintaining that user object with all mappings becomes cumbersome and prone to execution errors. Different models for creating aggregated identities for users in multi-account, cross-chain (MACC) metaverses can explore a specific set of data features.

The Best is Yet to Come

The best times of the internet are yet to come. The Metaverse will be the biggest breakthrough in application development in the past 25 years, replacing the cloud as the preferred application environment.

To fully unleash the potential of the Dataverse, interoperability is needed. This composable, interoperable data network will be a catalyst for a new wave of innovation, reshaping not only how we build applications but also how we share knowledge, invent science, and advance society. This is the world we are building with Ceramic.