AI 助理
文档备案控制台
开发者社区 安全 文章 正文

Ceph搭建

2026-07-15 20
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： ceph

Ceph搭建

一、系统环境初始化

时间同步

关闭selinux和防火墙(Centos)

配置主机域名解析或通过DNS解析

所有节点安装podman

apt install -y podman lvm2
主机名 ip
ceph1 192.168.11.10 \ 192.168.75.10
ceph2 192.168.11.11 \ 192.168.75.11
ceph3 192.168.11.12 \ 192.168.75.12 
root@ceph3:~# cat /etc/hosts
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.11.10 ceph1
192.168.11.11 ceph2
192.168.11.12 ceph3

配置时间同步 使用自带的timedate

timedatectl set-timezone Asia/Shanghai

二、部署RADOS集群

https://mirrors.aliyun.com/ceph/
https://mirrors.163.com/ceph/
https://mirrors.tuna.tsinghua.edu.cn/ceph/

2.1仓库准备

各节点配置ceph 仓库

aliyun默认带了ceph

root@ceph3:~# apt-cache madison ceph
      ceph | 19.2.3-0ubuntu0.24.04.3 | http://mirrors.aliyun.com/ubuntu noble-updates/main amd64 Packages
      ceph | 19.2.3-0ubuntu0.24.04.3 | http://mirrors.aliyun.com/ubuntu noble-security/main amd64 Packages
      ceph | 19.2.0~git20240301.4c76c50-0ubuntu6 | http://mirrors.aliyun.com/ubuntu noble/main amd64 Packages
root@ceph3:~#

2. 2安装 Cephadm（仅在主节点 ceph1）

apt update
apt install -y podman lvm2
apt install -y cephadm

初始化主节点

root@ceph1:~# cephadm bootstrap --mon-ip 192.168.11.10
Creating directory /etc/ceph for ceph.conf
Verifying podman|docker is present...
Verifying lvm2 is present...
Verifying time synchronization is in place...
Unit systemd-timesyncd.service is enabled and running
Repeating the final host check...
podman (/usr/bin/podman) version 4.9.3 is present
systemctl is present
lvcreate is present
Unit systemd-timesyncd.service is enabled and running
Host looks OK
Cluster fsid: 24f2c24c-75e6-11f1-b8f1-000c291acd7f
Verifying IP 192.168.11.10 port 3300 ...
Verifying IP 192.168.11.10 port 6789 ...
Mon IP `192.168.11.10` is in CIDR network `192.168.11.0/24`
Mon IP `192.168.11.10` is in CIDR network `192.168.11.0/24`
Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network
Pulling container image quay.io/ceph/ceph:v19...
Ceph version: ceph version 19.2.4 (7cc0193a616a3333d22b7762037fa5b73a4d06a6) squid (stable)
Extracting ceph user uid/gid from container image...
Creating initial keys...
Creating initial monmap...
Creating mon...
Waiting for mon to start...
Waiting for mon...
mon is available
Assimilating anything we can from ceph.conf...
Generating new minimal ceph.conf...
Restarting the monitor...
Setting public_network to 192.168.11.0/24 in mon config section
Wrote config to /etc/ceph/ceph.conf
Wrote keyring to /etc/ceph/ceph.client.admin.keyring
Creating mgr...
Verifying port 0.0.0.0:9283 ...
Verifying port 0.0.0.0:8765 ...
Verifying port 0.0.0.0:8443 ...
Waiting for mgr to start...
Waiting for mgr...
mgr not available, waiting (1/15)...
mgr not available, waiting (2/15)...
mgr not available, waiting (3/15)...
mgr is available
Enabling cephadm module...
Waiting for the mgr to restart...
Waiting for mgr epoch 5...
mgr epoch 5 is available
Setting orchestrator backend to cephadm...
Generating ssh key...
Wrote public SSH key to /etc/ceph/ceph.pub
Adding key to root@localhost authorized_keys...
Adding host ceph1...
Deploying mon service with default placement...
Deploying mgr service with default placement...
Deploying crash service with default placement...
Deploying ceph-exporter service with default placement...
Deploying prometheus service with default placement...
Deploying grafana service with default placement...
Deploying node-exporter service with default placement...
Deploying alertmanager service with default placement...
Enabling the dashboard module...
Waiting for the mgr to restart...
Waiting for mgr epoch 9...
mgr epoch 9 is available
Generating a dashboard self-signed certificate...
Creating initial admin user...
Fetching dashboard port number...
Ceph Dashboard is now available at:

             URL: https://ceph1:8443/
            User: admin
        Password: 5vcur62785

Enabling client.admin keyring and conf on hosts with "admin" label
Saving cluster configuration to /var/lib/ceph/24f2c24c-75e6-11f1-b8f1-000c291acd7f/config directory
You can access the Ceph CLI as following in case of multi-cluster or non-default config:

        sudo /usr/sbin/cephadm shell --fsid 24f2c24c-75e6-11f1-b8f1-000c291acd7f -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring

Or, if you are only running a single cluster on this host:

        sudo /usr/sbin/cephadm shell

Please consider enabling telemetry to help improve Ceph:

        ceph telemetry on

For more information see:

        https://docs.ceph.com/en/latest/mgr/telemetry/

Bootstrap complete.

此命令将

  • 在本地主机上为新集群创建一个 Monitor 和一个 Manager 守护程序。
  • 为 Ceph 集群生成一个新的 SSH 密钥，并将其添加到 root 用户的 /root/.ssh/authorized_keys 文件中。
  • 将公钥副本写入 /etc/ceph/ceph.pub。
  • 将最小配置文件写入 /etc/ceph/ceph.conf。此文件是与 Ceph 守护程序通信所必需的。
  • 将 client.admin 管理（特权！）密钥副本写入 /etc/ceph/ceph.client.admin.keyring。
  • 将 _admin 标签添加到引导主机。默认情况下，任何具有此标签的主机都将（也）获得 /etc/ceph/ceph.conf 和 /etc/ceph/ceph.client.admin.keyring 的副本。

免密失败了 /etc/ceph/ceph.pub

ssh-keygen

重新初始化

root@ceph1:~# cephadm bootstrap \
  --mon-ip 192.168.11.10 \
  --ssh-private-key /root/.ssh/id_ed25519 \
  --ssh-public-key /root/.ssh/id_ed25519.pub \
  --allow-overwrite
Verifying podman|docker is present...
Verifying lvm2 is present...
Verifying time synchronization is in place...
Unit systemd-timesyncd.service is enabled and running
Repeating the final host check...
podman (/usr/bin/podman) version 4.9.3 is present
systemctl is present
lvcreate is present
Unit systemd-timesyncd.service is enabled and running
Host looks OK
Cluster fsid: 10b97a6a-75ef-11f1-8612-000c291acd7f
Verifying IP 192.168.11.10 port 3300 ...
Verifying IP 192.168.11.10 port 6789 ...
Mon IP `192.168.11.10` is in CIDR network `192.168.11.0/24`
Mon IP `192.168.11.10` is in CIDR network `192.168.11.0/24`
Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network
Pulling container image quay.io/ceph/ceph:v19...
Ceph version: ceph version 19.2.4 (7cc0193a616a3333d22b7762037fa5b73a4d06a6) squid (stable)
Extracting ceph user uid/gid from container image...
Creating initial keys...
Creating initial monmap...
Creating mon...
Waiting for mon to start...
Waiting for mon...
mon is available
Assimilating anything we can from ceph.conf...
Generating new minimal ceph.conf...
Restarting the monitor...
Setting public_network to 192.168.11.0/24 in mon config section
Wrote config to /etc/ceph/ceph.conf
Wrote keyring to /etc/ceph/ceph.client.admin.keyring
Creating mgr...
Verifying port 0.0.0.0:9283 ...
Verifying port 0.0.0.0:8765 ...
Verifying port 0.0.0.0:8443 ...
Waiting for mgr to start...
Waiting for mgr...
mgr not available, waiting (1/15)...
mgr not available, waiting (2/15)...
mgr not available, waiting (3/15)...
mgr is available
Enabling cephadm module...
Waiting for the mgr to restart...
Waiting for mgr epoch 5...
mgr epoch 5 is available
Setting orchestrator backend to cephadm...
Using provided ssh keys...
Adding key to root@localhost authorized_keys...
Adding host ceph1...
Deploying mon service with default placement...
Deploying mgr service with default placement...
Deploying crash service with default placement...
Deploying ceph-exporter service with default placement...
Deploying prometheus service with default placement...
Deploying grafana service with default placement...
Deploying node-exporter service with default placement...
Deploying alertmanager service with default placement...
Enabling the dashboard module...
Waiting for the mgr to restart...
Waiting for mgr epoch 9...
mgr epoch 9 is available
Generating a dashboard self-signed certificate...
Creating initial admin user...
Fetching dashboard port number...
Ceph Dashboard is now available at:

             URL: https://ceph1:8443/
            User: admin
        Password: tvhp9krki9

Enabling client.admin keyring and conf on hosts with "admin" label
Saving cluster configuration to /var/lib/ceph/10b97a6a-75ef-11f1-8612-000c291acd7f/config directory
You can access the Ceph CLI as following in case of multi-cluster or non-default config:

        sudo /usr/sbin/cephadm shell --fsid 10b97a6a-75ef-11f1-8612-000c291acd7f -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring

Or, if you are only running a single cluster on this host:

        sudo /usr/sbin/cephadm shell

Please consider enabling telemetry to help improve Ceph:

        ceph telemetry on

For more information see:

        https://docs.ceph.com/en/latest/mgr/telemetry/

Bootstrap complete.

2.3 安装 Ceph 命令行工具

为了方便直接在终端输入 ceph 命令，在 ceph1 上安装客户端：

root@ceph1:~# apt install -y ceph-common

三、编排三节点集群

3.1 允许主节点免密登录其他节点

Cephadm 需要通过 SSH 管理 ceph2 和 ceph3。

3.2 将主机加入集群

在 node1 上执行命令，把其他两个 Ubuntu 节点拉进集群：

Bash

root@ceph1:~# ceph orch host add ceph2 192.168.11.11
Added host 'ceph2' with addr '192.168.11.11'
root@ceph1:~# ceph orch host add ceph3 192.168.11.12
Added host 'ceph3' with addr '192.168.11.12'

验证节点状态：ceph orch host ls

root@ceph1:~# ceph orch host ls
HOST   ADDR           LABELS  STATUS
ceph1  192.168.11.10  _admin
ceph2  192.168.11.11
ceph3  192.168.11.12
3 hosts in cluster

4. 部署高可用组件（Mon & Mgr）

为了让 3 个节点对等且具备高可用性，将 Monitor 和 Manager 拓扑应用到所有节点：

# 让 3 个节点都运行 mon
root@ceph1:~# ceph orch apply mon --placement="3 ceph1 ceph2 ceph3"
Scheduled mon update...

# 让 3 个节点都运行 mgr
root@ceph1:~# ceph orch apply mgr --placement="3 ceph1 ceph2 ceph3"
Scheduled mgr update...

5. 添加 OSD（存储裸盘）

确保你准备用于存储数据的硬盘没有分区表，没有被挂载过。

5.1 检查可用硬盘

root@ceph1:~# ceph orch device ls
HOST   PATH      TYPE  DEVICE ID                                             SIZE  AVAILABLE  REFRESHED  REJECT REASONS
ceph1  /dev/sdb  hdd                                                        40.0G  Yes        15m ago
ceph1  /dev/sr0  hdd   VMware_Virtual_SATA_CDRW_Drive_01000000000000000001  1023M  No         15m ago    Failed to determine if device is BlueStore, Insufficient space (<5GB)
ceph2  /dev/sdb  hdd                                                        40.0G  Yes        4m ago
ceph2  /dev/sr0  hdd   VMware_Virtual_SATA_CDRW_Drive_01000000000000000001  1023M  No         4m ago     Failed to determine if device is BlueStore, Insufficient space (<5GB)
ceph3  /dev/sdb  hdd                                                        40.0G  Yes        5s ago
ceph3  /dev/sr0  hdd   VMware_Virtual_SATA_CDRW_Drive_01000000000000000001  1023M  No         5s ago     Failed to determine if device is BlueStore, Insufficient space (<5GB)
root@ceph1:~#

如果看到各节点的硬盘（如 /dev/sdb）状态为 Available: Insufficient space (0B) 或 False，说明盘不干净，需要用 wipefs -a /dev/sdb 清理。

5.2 自动挂载所有可用裸盘

ceph orch apply osd --all-available-devices

6. 完工检查

在 node1 上输入：

root@ceph1:~# ceph -s
  cluster:
    id:     10b97a6a-75ef-11f1-8612-000c291acd7f
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum ceph1,ceph3,ceph2 (age 7m)
    mgr: ceph1.uwzqip(active, since 25m), standbys: ceph3.ftoasw, ceph2.wevbmh
    osd: 3 osds: 3 up (since 5m), 3 in (since 5m)

  data:
    pools:   1 pools, 1 pgs
    objects: 2 objects, 449 KiB
    usage:   81 MiB used, 120 GiB / 120 GiB avail
    pgs:     1 active+clean

只要看到 health: HEALTH_OK，且 mon 和 mgr 的数量都显示为 3，三节点 Ceph 集群就已经搭建成功了！

image-20260702165445177

文章标签：
网络安全
存储
Ubuntu
网络协议
域名解析
给我一碗炒粉
目录
相关文章
游客37x5chh37o32g
|
5天前
|
缓存 人工智能 安全
GPT-5.6 Terra与GPT-5.5性能实测：成本减半后的跑分对比与快速迁移指南
GPT-5.6 Terra 的定价为每百万 token 输入 2.50/输出 15。GPT-5.5 则是 5/ 30。Terra 的每一项费率，包括 $0.25/M 的缓存读取，都恰好是 GPT-5.5 的一半，因此在任何工作负载组合下，Terra 都固定 便宜 2.0x。以每天 10 万次请求、3K token 提示词计算，大约是 Terra 每天 2,000，GPT−5.5每天 4,000，即每月约 60,000对 120,000。问题在于：OpenAI 没有发布任何针对 Terra 的编码基准。那个著名的 91.9% Terminal-Bench 数字是 Sol 在 Ul
游客37x5chh37o32g
677 1
阿里云安全_
|
13天前
|
云安全 人工智能 安全
阿里云入选Forrester报告七大全场景，引领中国AI原生云安全
喜报～
阿里云安全_
1302 7
AI未闻花名
|
4天前
|
SQL 人工智能 自然语言处理
大模型内容安全实时防护：恶意Prompt注入拦截、越权阻断与熔断机制方案.166
本文系统阐述大模型输入安全防护体系，涵盖提示词注入、恶意Prompt拦截、越权阻断与输入熔断四大核心风险及应对方案。提出四层防护架构（预处理、检测、鉴权、熔断），结合规则引擎、语义识别与RBAC权限控制，实现全链路实时防护，保障业务合规、数据安全与服务稳定。
AI未闻花名
242 1
数据cool
|
12天前
|
存储 关系型数据库 MySQL
云数据库自研存储引擎：阿里云 PolarDB 相比开源 MySQL 性能提升数倍
云数据库自研存储引擎已成为云原生数据库竞争的核心技术高地。阿里云 PolarDB 通过 X-Engine 高压缩存储、PolarStore 用户态 IO + RDMA、IMCI 列存 HTAP 三大自研引擎组合，配合物理日志、跨节点 Buffer Pool 共享、Parallel Query 五大优化，实现写性能 6 倍、压缩 3-5 倍、AP 加速 100 倍、IO 延迟 -80% 的全面超越，让客户存储成本下降 65%、DBA 人力节省 80%。
数据cool
453 114
程序员鱼皮
|
5天前
|
人工智能 缓存 JSON
刚刚 GPT-5.6 发布，吊打 Claude 5 和 Grok 4.5？一手实测来啦！
GPT-5.6 刚发布，跑分号称超越 Fable 5，真的假的？附一手实测，让 3 个最强 AI 编程模型 GPT-5.6 Sol、Claude Fable 5、Grok 4.5 在 Cursor 里同时一把梭开发网页游戏，看看最新模型到底谁更能打。
程序员鱼皮
307 0
aliyun9804940036-19635
|
5天前
|
存储 人工智能 JSON
Qwen 本地部署搭配 ComfyUI 生成 AI 漫剧完整实操指南（小白零基础可落地，零成本无限生成+角色一致性天花板）
2026全网最优本地漫剧流水线：零成本、离线运行、角色统一、低配（8G显卡）可跑。融合Qwen本地大模型+ComfyUI双引擎，实现剧本生成→分镜绘图→动态成片全自动，隐私安全、无审核限流，新手30分钟上手，日更无忧。（239字）
aliyun9804940036-19635
389 2
aliyun9804940036-19635
|
12天前
|
人工智能 编解码 物联网
2026 最新Stable Diffusion 本地部署教程 下载安装使用详细图解（含官网安装包）
Stable Diffusion（SD）是2022年发布的开源文生图模型，由Stability AI等联合开发。支持文生图、图生图、局部重绘等，依托VAE降低算力需求，可在消费级显卡运行。本文提供秋葉aaaki制作的Windows整合包（含图形界面与插件），开箱即用，零配置启动。
aliyun9804940036-19635
874 1
游客usiy5oatinu3g
|
28天前
|
Linux 程序员 数据格式
【2026最新】Notepad++下载、安装和使用一篇搞定（附中文版安装包）
Notepad++ 是一款免费开源、轻量高效的 Windows 文本编辑器，支持 C/Python/HTML 等 80+ 语言语法高亮、代码折叠、正则替换、编码转换及插件扩展，专为程序员与文本处理用户打造，完美替代系统记事本。（239字）
游客usiy5oatinu3g
6365 8
老金带你玩AI
|
1天前
|
缓存 UED 开发者
Codex109天重置23次，明天还要再送一次
Codex近109天完成23次额度重置，7月14日将迎来第24次。Tibo高频响应用户反馈：优化GPT-5.6高消耗问题、补发失效福利、调整重置时间——形成“反馈→回应→修复→补偿”正向闭环，彰显以用户为中心的产品哲学。（239字）
老金带你玩AI
405 11