Decentralized S3, CESS pioneers decentralized object storage DeOSS

Web3's application and development across various fields has become a focal point of discussion. Although the industry's definitions of Web3 vary, a common consensus is that Web3 empowers users with ownership and autonomy over their data. This shift is crucial as our lives and work increasingly intertwine with digitalization, meaning that all human activities will soon be presented in the form of data streams. In other words, our various behaviors will be recorded as data, and a vast and complex data ecosystem will serve as the underlying form of digital life for the Web3 industry and humanity as a whole.

How to effectively store, manage, utilize, and share data resources, and how to truly unlock and enjoy the value of data, has become a common challenge faced by businesses and individuals. Web3 data solutions, with their return of ownership, have become a means to realize a new data distribution relationship. Web3 data infrastructure, represented by CESS, provides an open, decentralized, permissionless data economy where users hold data sovereignty.

However, the Web3 data field still faces issues such as high data loss rates, low storage space utilization, restricted data usage (e.g., slow retrieval speeds), limited data circulation and exchange, and a lack of data privacy management. Among these, the most critical issue is that large-scale commercial application storage in the Web3 industry has not been effectively addressed, which will limit the infrastructure's ability to truly enable large-scale applications and hinder the complete arrival of the next era of the internet.

This article will introduce CESS's pioneering decentralized object storage service, DeOSS, from the following perspectives, and how it addresses the challenges faced in the data storage field.

• Analysis of the technical principles of decentralized object storage service DeOSS
• Comparative analysis of the performance, security, and cost of "DeOSS" versus "OSS" services
• Application scenarios and product usage guide for DeOSS

Why Decentralized Object Storage Services Are Needed

From the perspective of large-scale commercial storage demands, the massive unstructured data generated by each user in their digital life on the internet, such as images, videos, and log files, due to its diverse formats and large sizes, has become the primary data type generated and consumed by businesses and individuals. Object Storage Service (OSS) has emerged in response to this need. Object storage is a storage and management technology for unstructured data that can provide large-scale, scalable, and cost-effective storage services, such as Amazon S3, which is a well-known object storage service in the industry. However, the current OSS in the industry, due to its centralized nature, exposes users to issues such as loss of data sovereignty, vulnerability to data breaches, and the sale of digital identities and data to third parties.

At the same time, with the rapid development of Web3 applications, especially Dapps that rely on real-time data streams, there is a need for fast and effective processing and storage of high-frequency dynamic data to ensure application responsiveness and user experience. Furthermore, the new generation of the internet has higher requirements for the security, privacy, and ownership of data storage, prompting the exploration of more efficient, secure, and scalable solutions in the storage field.

In response to the aforementioned storage issues and demands, CESS has pioneered the decentralized object storage service DeOSS. CESS aims to provide reliable support for unstructured data, high-frequency dynamic data, and large-scale commercial storage through decentralized technology and innovative storage protocols. This not only promotes the healthy development of the Web3 ecosystem but also provides users with higher quality services, driving the advancement of a data-driven new era. DeOSS can meet the enterprise-level unstructured data storage needs and provide high-performance storage services with high security, scalability, and privacy management.

Understanding "Decentralized Object Storage Service DeOSS"

Decentralized Object Storage Service (DeOSS) is a decentralized, object-based massive storage service that provides low-cost, secure, and scalable distributed data storage services for the Web3 field.

Both individuals and businesses can use DeOSS to store, manage, and protect data, such as images and videos of NFTs, user data of DApps, and programs of DeFi. Based on blockchain technology, DeOSS effectively utilizes idle online storage resources to build a vast distributed storage network.

The mission of DeOSS is to return data rights to the data producers themselves, rather than to centralized institutions; DeOSS follows the Apache 2.0 open-source protocol, allowing anyone to deploy it at any time and place, and to freely access files stored around the world.

Product Introduction

• Product Principles

DeOSS will provide a standard storage method to achieve basic storage functions, such as purchasing storage space, creating, deleting, and restricting access to storage containers, uploading, downloading, deleting, sharing files, and limiting user operations.

DeOSS Product Architecture

Specifically, user data is stored in the form of objects within containers (Buckets). During the process of using DeOSS to store data, you need to first create a storage container and specify attributes such as region, access permissions, and storage type. Data will be uploaded to the storage container in object form, with each object containing user data (Data), data tags (Metadata), and file addresses (Key) as its unique identifier.

The internal storage space of DeOSS is flat, without a hierarchical structure of file directories. Therefore, data objects are directly associated with their corresponding storage space. Businesses can choose to build their own gateways for external access or use existing gateways. When users access via a domain name, the DeOSS gateway will forward user requests to the corresponding storage service.

• Product Features

- Decentralized: DeOSS is an object-based massive storage service within the CESS decentralized network, supporting encrypted online data transmission via HTTPS, ensuring security and privacy during data transmission and storage.

- High Scalability: Each account can manage up to 1000 buckets, with no space limitations for each bucket, providing massive storage capacity.

- Data Ownership: You have control over your data and its ownership, with the ability to choose private storage, share with specific individuals, or make it completely public, as well as the ability to delete data without interference.

- Anti-Censorship: Any user can control their own data according to the protocol, avoiding interference from other entities.

- Millisecond-level Data Retrieval: Supports online real-time sharing of data and flexible file management.

• What is DeOSS Suitable for Storing?

DeOSS is suitable for storing high-definition images, audio, video, backup files, and various files from DApp applications and cloud storage applications.

• Who Can Use DeOSS?

DApp, NFT, and Web3 application developers, SocialFi developers, and users with large-scale data storage needs.

If you are a non-developer user, you can experience DeOSS by visiting the CESS official website.

If you are a developer, you can experience DeOSS using HTTP requests.

(Participation link: https://www.cess.cloud/deoss.html)
System Architecture

Unlike mainstream distributed system architectures, DeOSS has achieved a high degree of integration with the CESS blockchain at the underlying level, forming a unique open, decentralized object storage system. The system architecture is divided into four layers, as shown in the diagram below: application layer, service interface layer, blockchain layer, and storage service layer. The application layer provides users with an interactive and user-friendly file object management service. The interface layer acts as middleware between the application layer and the blockchain layer, enabling DeOSS to provide object storage services for various custom applications and achieving service functionality openness.

• Blockchain Layer

The blockchain layer is the core component of the DeOSS architecture, connecting the scheduler, storage miners, and user applications, providing important functions such as file metadata management, user storage space management, and scheduling and storage miner account management. The blockchain layer is based on the CESS chain as the infrastructure and adopts a self-developed efficient consensus algorithm, R²S. (For a detailed explanation of the mechanism, see “CESS Mechanism Explanation (2): Blockchain Layer and Random Selection Rotation (R²S) Consensus Node Mechanism”). DeOSS also has flexible storage expansion capabilities, allowing any qualified account to apply to become a scheduling node or storage miner, providing services for DeOSS and earning rewards.

• Service Interface Layer

The DeOSS service interface layer implements modular encapsulation of commonly used code libraries, so the application layer only needs to implement a small number of interfaces to provide complete object storage services. The DeOSS application layer service architecture is shown in the diagram below. Currently, DeOSS provides two application modes: a command-line-based Go client mode and a server mode based on the Gin framework. Both can be quickly launched with simple configuration. Users can also freely add functional module interfaces according to their needs without worrying about the internal code implementation details.

DeOSS Application Layer Service Architecture

• Application Layer

The DeOSS application service has a clear three-layer architecture.

The network connection layer provides basic services such as user interaction, data object transmission, and on-chain function calls to the upper layer.

The internal service layer processes user data files from the interaction layer, providing preprocessing programs such as data rights confirmation, file fragmentation and redundancy, and file declaration, and hands the processed content over to the scheduler for scheduling. On the other hand, it is also responsible for processing data from the chain end and storage miner end.

The interaction layer is responsible for the interaction between users and DeOSS, forming a complete service closed loop;

In addition, DeOSS also provides additional service support such as data encryption and decryption, and system logs.

• File Transfer Protocol

DeOSS uses the TCP protocol for reliable transmission as the underlying communication protocol, and on this basis, establishes a secure and reliable file transfer mechanism. The file transfer protocol is shown in the diagram below. When the application service layer initiates a file send or receive request, it first establishes a TCP connection with the receiving/sending party, and then establishes a file fragment byte stream transmission channel through the handshake mechanism of the DeOSS file transfer protocol.

DeOSS vs OSS

In addition to possessing the advantages of existing object storage products on the market (such as high reliability, easy scalability, low cost, and high security), DeOSS also features anti-censorship and data ownership belonging to users. Current Web2 object storage services, such as AWS and Alibaba Cloud's OSS, are centralized storage solutions. As mentioned earlier, users using OSS storage services face the risk of data being regulated and monopolized. However, the data uploaded by users through DeOSS is decentralized storage based on CESS blockchain technology, and the ownership of the data belongs to the user. Users can authorize certain applications to use the data through DeOSS, but these applications cannot control the data that users have authorized for use.

Below is a detailed comparison between DeOSS and OSS.

From the comparison in the table above, it can be seen that DeOSS not only has advantages in storage performance and cost compared to existing object storage (OSS) products on the market, but it is particularly noteworthy that the decentralized nature of DeOSS breaks through the constraints of traditional storage services in terms of security, data rights confirmation, and benefits. This is precisely the issue that enterprise-level big data storage urgently needs to address.

DeOSS Application Scenarios

In traditional application fields, DeOSS decentralized object cloud storage can enter the commercial cloud business domain just like Amazon Cloud and Google Cloud. Through DeOSS, it can provide all categories of cloud storage services for businesses and individuals, meeting a variety of use cases, including but not limited to the following:

Distributed File System --- Distributed file storage can be used not only for personal user backups and sharing but also to provide large-scale commercial storage services for enterprises.

Social Media --- With its efficient, secure, scalable, and large-scale storage capabilities, DeOSS meets the needs of social applications for secure storage, real-time updates, and user data sovereignty of massive user data.

Streaming Media Applications --- DeOSS provides the best solutions for the creation, storage, and streaming of large files such as high-definition images, audio, and video.

Data Trading Market --- DeOSS supports the monetization and circulation of data value, providing a foundation for the establishment of data trading markets.

Blockchain DA Layer --- DeOSS can provide low-cost, trustless, and highly available storage for block data and state data of Layer 1, as well as transaction data in Layer 2 rollup solutions.

Creator Platforms --- DeOSS integrates decentralized storage networks and content delivery networks, meeting the needs of creator platforms for data copyright establishment and traceability.

Data Lakes --- DeOSS is suitable for storing, processing, and protecting large amounts of data, helping enterprises reduce costs, manage data, and utilize AI.

Biological Data --- DeOSS supports multi-type data rights confirmation and traceability, providing massive storage and efficient retrieval, addressing the challenges of biological data, which include vast quantities, rapid growth, diverse sources, and difficulties in standardization and structuring.

Experience DeOSS Now

In the revolutionary construction and innovation process of Web3, a new type of data infrastructure architecture and data circulation mechanism is gradually forming, creating an economic system focused on data rights confirmation, circulation, and trading, which is expected to drive changes in internet business models and value distribution models, accelerating the development and innovation of the digital economy.

CESS's innovative decentralized object storage service, DeOSS, will become one of the important products and services driving this process. In addition to having the advantages of existing object storage on the market, such as high reliability, easy scalability, low cost, and high security, it also possesses the advantages of decentralization, data sovereignty belonging to users, and anti-censorship.

Dear partners, we sincerely invite you to experience and use the DeOSS product! The DeOSS functionality can be used after source code editing and deployment. For the getting started guide, please refer to the link below:

https://docs.cess.cloud/deoss/

Finally, if you encounter any issues during use, please feel free to contact us. We eagerly hope to contribute to the construction of the next generation of value internet with global partners by providing the foundational power of decentralized cloud storage!