Alibaba Cloud Launches Dual-mode SSD to Optimize Hyper-scale Infrastructure Performance

简介: Alibaba has announced today the launch of a new system that aims to optimize the storage performance of hyper-scale infrastructure in addressing incre.

Alibaba Cloud has announced today the launch of a new system that aims to optimize the storage performance of hyper-scale infrastructure in addressing increasing demands from usage of artificial intelligence (AI) and cloud computing.

A new standard for cloud storage

A Dual-mode SSD (Solid State Drive), a storage device which supports both Open-Channel Mode and native NVMe mode, has been developed by the Alibaba Infrastructure Services team, and an optimal software/hardware integrated solution based on the Dual-mode SSD is currently being deployed to Alibaba's internal servers. It is expected this novel storage system will lead to a 75% reduction in read latency and enhance the overall storage performance of data centers by as many as five times.

dual_mode_SSD.

"The increasing proliferation of AI and cloud computing has led to more sophisticated demands in data centers, while traditional storage systems face severe limitation in meeting such demands.In light of these challenges, Alibaba has pioneered the research and development of a new storage system, the Dual-mode SSD infrastructure. This underscores our commitment to driving the innovation and optimization of technology infrastructure in a new AI and cloud era," said Shu Li, Senior Staff Engineer at Alibaba Infrastructure Services.

"By creating and sharing the Dual-mode SSD specification, we are also working with different manufacturers on related firmware and hardware products, leading to the fast development of SSD-centered infrastructure and ecosystems."

AliFlash

Limitations of traditional hardware

With the proliferation of new applications like artificial intelligence, Internet of Things (IoT), big data, and cloud computing, today’s hyper-scale data centers run far more diversified and complex workloads than ever before. These different applications can have drastically different I/O patterns, performance or Quality of Service (QoS) targets, usage models, and sometimes require different set of features from storage devices.

Traditional storage architecture is designed around standardized hardware that provides generic block I/O interface, and software stack that is built on top of such abstract, generic block device. While this architecture has advantages of portability and backward compatibility,it is, however, facing serious challenges in today’s hyper-scale data centers.

  • Standard hardware (e.g. NVMe SSD) must conform to certain specs and has limited room for customization. It is difficult to adapt to many different I/O patterns or use.
  • In traditional architecture, hardware and software are designed and optimized separately without knowing each other. This separation creates a major obstacle for further optimization.
  • Standard hardware is mostly a black box to host software – it conceals most of its internal mechanisms in order to create an illusion of “generic block device” to host software. The drawback of such encapsulation is that software has no control on performance once I/O reaches the device.

Leading the edge in cloud storage

Taking Alibaba Cloud as an example, we have numerous different applications serving our business units and customers, such as E-Commerce, Search, Online Promotion, Multimedia, Financial Service, Logistic Service, and Cloud Computing. Some of our applications demand features that are not available in standard SSDs. Our application requirements also change frequently, therefore storage system must be agile and quick-responding.

Alibaba Cloud tackled these challenges with hardware/software co-design approach using the Dual-Mode SSD. We combine in-house SSD hardware with first-hand understanding of applications and use cases, and work closely with business teams to design and optimize the entire I/O stack. The result is a set of hardware/software integrated solutions that are highly optimized for applications in our data center.

architecture

The dual-mode SSD demonstrates Alibaba Cloud's consistent effort to pursue performance improvements in hyperscale data centers with hard-ware/software co-design approach. We develop the in-house dual-mode SSD that supports both NVMe device-based mode and Open Channel mode. On software side, we develop User-Space Open Channel I/O stack that closely integrates SSD hardware, firmware, driver, operation system together with our applications.

Furthermore, dual-mode SSD demonstrates the promising potential of hardware/software jointed optimization with the use case of advanced I/O scheduling. Evaluation results show the proposed dual-mode SSD deployed in hyperscale infrastructure reduces access latency by 75%, and improves 99th percentile latency by 5.8 times.

For more information about the Dual-mode SSD infrastructure and products, please refer to the link below: Alibaba Dual-Mode SSD

目录
相关文章
Query Performance Optimization at Alibaba Cloud Log Analytics Service
PrestoCon Day 2023,链接:https://prestoconday2023.sched.com/event/1Mjdc?iframe=no首页自我介绍,分享题目概要各个性能优化项能够优化的资源类别limit快速短路有什么优点?有啥特征?进一步的优化空间?避免不必要块的生成逻辑单元分布式执行,global 阶段的算子哪些字段无需输出?公共子表达式结合FilterNode和Proje
Query Performance Optimization at Alibaba Cloud Log Analytics Service
|
缓存 前端开发 安全
译|High-Performance Server Architecture(上)
译|High-Performance Server Architecture
77 0
|
存储 缓存 网络协议
译|High-Performance Server Architecture(下)
译|High-Performance Server Architecture(下)
85 0
|
存储 缓存 应用服务中间件
Network & Shell & Operation & Automation Topic | Cloud computing (FREE)
云计算 Network & Shell & Operation & Automation 习题(试读)
109 0
|
固态存储
《In Pursuit of Optimal Storage PerformanceHardwareSoftware Co-Design with Dual-Mode SSD》电子版地址
In Pursuit of Optimal Storage Performance:Hardware/Software Co-Design with Dual-Mode SSD
82 0
《In Pursuit of Optimal Storage PerformanceHardwareSoftware Co-Design with Dual-Mode SSD》电子版地址
|
存储 监控 文件存储
|
SQL 存储 算法
《Optimization of Common Table Expressions in MPP Database Systems》论文导读
Optimization of Common Table Expressions in MPP Database Systems
《Optimization of Common Table Expressions in MPP Database Systems》论文导读
|
弹性计算 安全 关系型数据库
Deploy Web Apps with High Availability, Fault Tolerance, and Load Balancing on Alibaba Cloud
High Availability (HA), Fault Tolerance (FT), and Horizontal Scale Friendly (HSF) are as equally important as to functionality for web applications to run and succeed today.
4022 0
|
SQL 关系型数据库 RDS
Troubleshooting High CPU Usage on Alibaba Cloud SQL Server
A primary issue with SQL Server is its sensitivity to latency, often resulting in performance issues.
1362 0
Troubleshooting High CPU Usage on Alibaba Cloud SQL Server
|
固态存储 关系型数据库 分布式数据库
100TB Capacity and 6x Performance Improvement with Alibaba Cloud PolarDB
This article focuses on the optimizations of Alibaba Cloud PolarDB's compute and storage engines to offer an unparalleled performance.
5940 0
100TB Capacity and 6x Performance Improvement with Alibaba Cloud PolarDB