centos7使用ceph-deploy部署ceph

时间：2021-07-08 02:43:57

准备阶段

准备yum源

删除默认的源，国外的比较慢

yum clean all
rm -rf /etc/yum.repos.d/*.repo

下载阿里云的base源

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

下载阿里云的epel源

wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

修改里面的系统版本为7.3.1611,当前用的centos的版本的的yum源可能已经清空了

sed -i '/aliyuncs/d' /etc/yum.repos.d/CentOS-Base.repo
sed -i '/aliyuncs/d' /etc/yum.repos.d/epel.repo
sed -i 's/$releasever/7.3.1611/g' /etc/yum.repos.d/CentOS-Base.repo

添加ceph源

vim /etc/yum.repos.d/ceph.repo

[ceph]

 
 
name=ceph
 
 
 
 
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/x86_64/
 
 
 
 
gpgcheck=0
 
 
 
 
priority =1
 
 
 
 
[ceph-noarch]
 
 
 
 
name=cephnoarch
 
 
 
 
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch/
 
 
 
 
gpgcheck=0
 
 
 
 
priority =1
 
 
 
 
[ceph-source]
 
 
 
 
name=Ceph source packages
 
 
 
 
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/SRPMS
 
 
 
 
gpgcheck=0
 
 
 
 
priority=1
 
 

 




 准备系统配置
 

设置deploy主机的/etc/hosts文件
 

 
 

 
 
192.168.0.39 ceph-admin
192.168.0.40 mon1
192.168.0.41 osd1
192.168.0.42 osd2
192.168.0.43 osd3
 

 

 
 

修改deploy主机上的~/.ssh/config文件
 

 
 
Host ceph-admin
   Hostname ceph-admin
   User cephuser
Host mon1
   Hostname mon1
   User cephuser
Host osd1
   Hostname osd1
   User cephuser
Host osd2
   Hostname osd2
   User cephuser
Host osd3
   Hostname osd3
   User cephuser
 

 

 
 

修改权限
 

 
 
chmod 644 ~/.ssh/config
 

 

添加用户
 

 
 
useradd -d /home/cephuser -m cephuser
passwd cephuser
 

 

确保添加的用户用sudo权限
 

 
 
echo "cephuser ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/cephuser
chmod 0440 /etc/sudoers.d/cephuser sed -i s'/Defaults requiretty/#Defaults requiretty'/g /etc/sudoers
 

 

 
 

设置deploy主机可以无密码访问其他node
 

 
 
su - cephuser
ssh-keygen ssh-copy-id ceph-admin ssh-copy-id mon1 ssh-copy-id osd1 ssh-copy-id osd2 ssh-copy-id osd3
 




 
 

 
 

 
 

安装NTP服务
 

 
 
yum install -y ntp ntpdate ntp-doc
ntpdate 0.us.pool.ntp.org
hwclock --systohc
systemctl enable ntpd.service
systemctl start ntpd.service
 

 

禁用selinux
 

 
 
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
 

 

 
 

关闭防火墙
 

 
 
ssh root@ceph-admin
systemctl stop firewalld
systemctl disable firewalld
 

 

准备磁盘
 

note:测试时使用的磁盘不要太小，否则后面添加磁盘时会报错，建议磁盘大小为20G及以上。
 

检查磁盘
 

 
 
sudo fdisk -l /dev/vdb
 

 

格式化磁盘
 

 
 
sudo parted -s /dev/vdb mklabel gpt mkpart primary xfs 0% 100%
sudo mkfs.xfs /dev/vdb -f
 

 

 
 

查看磁盘格式
 

 
 
sudo blkid -o value -s TYPE /dev/vdb
 

 

部署阶段



安装ceph-deploy
 

 
 
sudo yum update -y && sudo yum install ceph-deploy -y
 

 

创建cluster目录
 

 
 
su - cephuser
mkdir cluster
cd cluster/
 

 

创建集群
 

 
 
ceph-deploy new mon1
 

 

修改ceph.conf文件
 

 
 
vim ceph.conf
# Your network address
public network = 192.168.0.0/24
osd pool default size = 3
 

 

安装ceph
 

 
 
ceph-deploy install ceph-admin mon1 osd1 osd2 osd3
 

 

初始化monitor，并收集所有密钥
 

 
 
ceph-deploy mon create-initial
ceph-deploy gatherkeys mon1
 

 

添加OSD到集群
 

检查OSD节点上所有可用的磁盘
 

 
 
ceph-deploy disk list osd1 osd2 osd3
 

 

使用zap选项删除所有osd节点上的分区
 

 
 
ceph-deploy disk zap osd1:/dev/vdb osd2:/dev/vdb osd3:/dev/vdb
 

 

准备OSD
 

 
 
ceph-deploy osd prepare osd1:/dev/vdb osd2:/dev/vdb osd3:/dev/vdb
 

 

激活OSD



 
 
ceph-deploy osd activate osd1:/dev/vdb1 osd2:/dev/vdb1 osd3:/dev/vdb1
 

 

查看OSD
 

 
 
ceph-deploy disk list osd1 osd2 osd3
 

 

显示两个分区
 

    

  1. 
/dev/sdb1 - Ceph Data
    2.  
 
  2. 
/dev/sdb2 - Ceph Journal
    3.  



 
 

用 ceph-deploy 把配置文件和 admin 密钥拷贝到管理节点和 Ceph 节点，这样你每次执行 Ceph 命令行时就无需指定 monitor 地址和 ceph.client.admin.keyring 了
 

 
 
ceph-deploy admin ceph-admin mon1 osd1 osd2 osd3
 

 

修改密钥权限
 

 
 
sudo chmod 644 /etc/ceph/ceph.client.admin.keyring
 

 

完成！
 

检查ceph
 

检查ceph状态
 

 
 
sudo ceph health
sudo ceph -s
 

 

 
标签：
相关文章