一、安装部署：
想直接一步创建集群的小伙伴直接按以下步骤安装(再往后是记录自己出过的错)：
1、生成docker镜像：
docker build -t 144.202.127.156/library/mongodb:3.4.10 .
docker push 144.202.127.156/library/mongodb:3.4.10 注：记得把地址换成自己的镜像仓库
FROM alpine:edge
MAINTAINER chengcuichao RUN apk update && \
echo http://dl-4.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories && \
apk add --no-cache mongodb numactl numactl-tools
Dockerfile
2、在kubernete上创建：
kubectl create -f mongodb.yml
在此注意几下几点：
1）、将为外置存储的Secret和storageclass换成自己的。 注：我用的是ceph，其他的根据自己的更改，也可以不用持久化存储，就当测试用。
2）、镜像换成自己的镜像地址。
3）、启动参数--replSet换成自己的，不换也可以。
apiVersion: v1
kind: Namespace
metadata:
name: basic-app
---
apiVersion: v1
kind: Secret
metadata:
namespace: basic-app
name: ceph-secret
type: "kubernetes.io/rbd"
data:
key: QVFEYmVRTmJZQ1B4TFJBQUg0QS9Tb01NZjF6NHB3L0p1Y3ZUQnc9PQ==
---
apiVersion: v1
kind: Secret
metadata:
namespace: basic-app
name: mongo-key
type: Opaque
data:
key: UERVU0hWVU9KT1ZQVUVYT0JXWU8=
---
apiVersion: v1
kind: ConfigMap
metadata:
name: mongodb-config
namespace: basic-app
data:
mongodb: |
systemLog:
destination: file
path: "/var/log/mongodb/mongodb.log"
logAppend: true
processManagement:
fork: false
net:
port: 27017
bindIp: 0.0.0.0
security:
keyFile: "/etc/conf.d/secret-key/key"
authorization: enabled
storage:
dbPath: /var/lib/mongodb
setParameter:
enableLocalhostAuthBypass: true
authenticationMechanisms: SCRAM-SHA-1
mongos: |
MONGOS_EXEC="/usr/bin/mongos"
MONGOS_RUN="/var/run/mongodb"
MONGOS_USER="mongodb"
MONGOS_IP="127.0.0.1"
MONGOS_PORT=""
MONGOS_CONFIGDB=""
MONGOS_OPTIONS=""
---
apiVersion: v1
kind: Service
metadata:
name: mongodb
namespace: basic-app
labels:
name: mongo
spec:
clusterIP: None
ports:
- port: 27017
targetPort: 27017
selector:
app: mongo-cluster
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: mongodb
namespace: basic-app
spec:
serviceName: mongodb
replicas: 3
template:
metadata:
labels:
app: mongo-cluster
spec:
terminationGracePeriodSeconds: 10
containers:
- name: mongod
image: 144.202.127.156/library/mongodb:3.6.5
command: ["sh", "-c", "chmod 600 -R /etc/conf.d/secret-key;numactl --interleave=all mongod -f /etc/conf.d/mongodb --auth --replSet icsoc"]
resources:
limits:
cpu: 2
memory: 2G
requests:
cpu: 1
memory: 1G
volumeMounts:
- name: mongodb-data
mountPath: /var/lib/mongodb
- name: mongo-config
mountPath: /etc/conf.d
- name: timezone-config
mountPath: /etc/localtime
- name: secret-key
mountPath: /etc/conf.d/
ports:
- containerPort: 27017
livenessProbe:
tcpSocket:
port: 27017
initialDelaySeconds: 15
periodSeconds: 20
volumes:
- name: mongo-config
configMap:
name: mongodb-config
- name: timezone-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
- name: secret-key
secret:
secretName: mongo-key
volumeClaimTemplates:
- metadata:
name: mongodb-data
annotations:
volume.beta.kubernetes.io/storage-class: "ceph-db"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 50Gi
mongodb.yml
3、初始化集群：
1、连进容器内：
kubectl exec -it mongodb-0 /bin/sh 2、执行初始化副本集：
mongo
rs.initiate({_id: "icsoc", version: 1, members: [
{ _id: 0, host : "mongodb-0.mongodb.basic-app.svc.cluster.local:27017" },
{ _id: 1, host : "mongodb-1.mongodb.basic-app.svc.cluster.local:27017" },
{ _id: 2, host : "mongodb-2.mongodb.basic-app.svc.cluster.local:27017" }
]}); 2、创建管理用户： 具体可参考：https://docs.mongodb.com/manual/tutorial/enable-authentication/
use admin
db.createUser(
{
user: "myUserAdmin",
pwd: "P@ssw0rd",
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
}
)
db.auth("myUserAdmin","P@ssw0rd") 3、之后就可以连进去创建用户，赋予角色权限使用。
二、以下记录创建详细步骤：
1、在kubenetes安装mongodb集群大体思路为：
1）、先在一个基础的镜像里安装mongodb，启动正常后无报错。
2）、再理清楚mogodb副本集集群是怎么启动的。
3）、之后编写在kubernetes创建资源的文件。
4）、先创建看看那进行不下去，再一个个解决。
2、在alpine:edge基础上编写dockerfile：
编写好的Dockerfile如下：
FROM alpine:edge
MAINTAINER chengcuichao RUN apk update && \
echo http://dl-4.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories && \
apk add --no-cache mongodb numactl COPY run.sh /root/
RUN chmod +x /root/run.sh CMD /root/run.sh
Dockerfile
先在docker上启动，mongo连进去后报错：
Server has startup warnings:
2018-07-15T12:25:52.064+0800 W CONTROL [main] --diaglog is deprecated and will be removed in a future release
2018-07-15T12:25:52.183+0800 I STORAGE [initandlisten]
2018-07-15T12:25:52.183+0800 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-07-15T12:25:52.183+0800 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem 第一个报错
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 第二个
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten]
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten]
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten] ** WARNING: You are running on a NUMA machine.
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten] ** We suggest launching mongod like this to avoid performance problems:
2018-07-15T12:26:02.364+0800 I CONTROL [initandlisten] ** numactl --interleave=all mongod [other options] 第三个
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten]
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 第四个
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten]
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2018-07-15T12:26:02.365+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 第五个
1）、第一个报错是文件系统的问题，还没解决 。
2）、第二个报错是要开启认证，在启动参数上加入--auth就可以。
3）、第三个报错需要在mongod命令前面加上numactl --interleave=all，在docker容器里执行numactl --interleave=all mongod -f /etc/conf.d/mongodb 在直接用docker起的容器里执行会报错：
set_mempolicy: Operation not permitted
setting interleave mask: Operation not permitted
但在statefulset的yml文件加上command: ["sh", "-c", "numactl --interleave=all mongod -f /etc/conf.d/mongodb --bind_ip 0.0.0.0"]，kubectl create -f mongodb.yml创建后不会报错。
4）、第四个和第五个报错需要执行：echo never > /sys/kernel/mm/transparent_hugepage/enabled，echo never > /sys/kernel/mm/transparent_hugepage/defrag，
但是在容器里执行会报错，就算用initContainers来为mongodb的容器创建运行环境，但是还会报：
/bin/sh: can't create /sys/kernel/mm/transparent_hugepage/enabled: Read-only file system
/bin/sh: can't create /sys/kernel/mm/transparent_hugepage/defrag: Read-only file system
为容器增加守护脚本，以daemonset方式运行： 参考：http://pauldone.blogspot.com/2017/06/mongodb-kubernetes-production-settings.html https://github.com/kubernetes/contrib/tree/master/startup-script
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
namespace: basic-app
name: hostvm-configurer
labels:
app: startup-script
spec:
template:
metadata:
labels:
app: startup-script
spec:
hostPID: true
containers:
- name: hostvm-configurer-container
# image: gcr.io/google-containers/startup-script:v1
image: 144.202.127.156/google_containers/startup-script:v1
securityContext:
privileged: true
env:
- name: STARTUP_SCRIPT
value: |
#! /bin/bash
set -o errexit
set -o pipefail
set -o nounset # Disable hugepages
echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag
config-daemonset
官方文档：https://docs.mongodb.com/manual/replication/
专门在k8s上安装Mongodb：http://k8smongodb.net/