mongodb 两地三中心

容灾级别

MongoDB两地三中心集群实现

同城存在shard，异地存在 shard；

同机房集群整体出现问题，切换到 同城 节点（同城节点优先级配置高），同城所有节点出现问题，切换到 异地节点；

既可以防止某个节点宕机，也可以防止某个城市当即；

在集群中 启用认证安全机制也是非常重要的；

两地三中心规划及实施

a. 准备虚拟机及数据库实例

1. 规划
# 172.16.40.143
primary: 172.16.40.143:10001
s1     : 172.16.40.143:10002

# 172.16.40.144
s3		 : 172.16.40.144:10003
s4		 : 172.16.40.144:10004

# 172.16.40.145
s5		 : 172.16.40.145:10005


2.准备实例
# 配置文件 - 172.16.40.143
su - mongod
mkdir -p /mongodb/{10001..10002}/{data,log}

cat > /mongodb/10001/mongod.conf << EOF
systemLog:
  destination: file
  logAppend: true
  path: /mongodb/10001/log/mongodb.log

storage:
  journal:
    enabled: true
  dbPath: /mongodb/10001/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.5
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true

processManagement:
  fork: true  
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 10001
  bindIp: 0.0.0.0

replication:
    oplogSizeMB: 2048 
    replSetName: my_repl
EOF

\cp /mongodb/10001/mongod.conf /mongodb/10002/
sed 's#10001#10002#g' /mongodb/10002/mongod.conf -i

# 启动
mongod -f /mongodb/10001/mongod.conf
mongod -f /mongodb/10002/mongod.conf




# 配置文件 172.16.40.144
su - mongod
mkdir -p /mongodb/{10003..10004}/{data,log}

cat > /mongodb/10003/mongod.conf << EOF
systemLog:
  destination: file
  logAppend: true
  path: /mongodb/10003/log/mongodb.log

storage:
  journal:
    enabled: true
  dbPath: /mongodb/10003/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.5
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true

processManagement:
  fork: true  
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 10003
  bindIp: 0.0.0.0

replication:
    oplogSizeMB: 2048 
    replSetName: my_repl
EOF

\cp /mongodb/10003/mongod.conf /mongodb/10004/
sed 's#10003#10004#g' /mongodb/10004/mongod.conf -i

# 启动
mongod -f /mongodb/10003/mongod.conf
mongod -f /mongodb/10004/mongod.conf



# 配置文件 - 172.16.40.145
su - mongod
mkdir -p /mongodb/10005/{data,log}

cat > /mongodb/10005/mongod.conf << EOF
systemLog:
  destination: file
  logAppend: true
  path: /mongodb/10005/log/mongodb.log

storage:
  journal:
    enabled: true
  dbPath: /mongodb/10005/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.5
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true

processManagement:
  fork: true  
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 10005
  bindIp: 0.0.0.0

replication:
    oplogSizeMB: 2048 
    replSetName: my_repl
EOF

# 启动
mongod -f /mongodb/10005/mongod.conf


# 配置复制集
# 这里需要注意，一般连接谁去配置 复制集，mongodb集群就会配置谁为主节点，这里链接 10001
mongo --port 10001
use admin
config = {_id: "my_repl", members: [
                                    {_id: 0, host: '172.16.40.143:10001'},
                                    {_id: 1, host: '172.16.40.143:10002'},
                                    {_id: 2, host: '172.16.40.144:10003'},
                                    {_id: 3, host: '172.16.40.144:10004'},
                                    {_id: 4, host: '172.16.40.145:10005'}
]}

rs.initiate(config)

my_repl:PRIMARY>		// mongo 一般会自动将链接节点配置为主节点；

rs.status()		// 可以查看集群信息

b. 两地三中心定制化配置

cfg=rs.config()
cfg.members[1].priority = 20
cfg.members[2].priority = 10
cfg.members[3].priority = 8
rs.reconfig(cfg)
r
# 注意：这里 members[1] 并不是根据 rs.config() 返回的节点 _id 来取值，而是根据 rs.config() 返回的列表中实例的顺序来取值；
priority 值越大，mongodb 被推举为主节点的几率越大；

c.复制集安全加固

# db01 == 172.16.40.143
# 这个算法可以自己去选择，不一定要用 base64
openssl rand -base64 756 > /mongodb/10001/keyfile 
cp -a /mongodb/10001/keyfile /mongodb/10002
chmod 600 /mongodb/10001/keyfile /mongodb/10002/keyfile

scp /mongodb/10001/keyfile 172.16.40.144:/mongodb/10003
scp /mongodb/10001/keyfile 172.16.40.144:/mongodb/10004
scp /mongodb/10001/keyfile 172.16.40.145:/mongodb/10005

chmod 600 /mongodb/10003/keyfile /mongodb/10004/keyfile
chmod 600 /mongodb/10005/keyfile

# 每个节点开启验证；

cat >> /mongodb/10001/mongod.conf << EOF
security:
  keyFile: /mongodb/10001/keyfile
EOF

cat >> /mongodb/10002/mongod.conf << EOF
security:
  keyFile: /mongodb/10002/keyfile
EOF

cat >> /mongodb/10003/mongod.conf << EOF
security:
  keyFile: /mongodb/10003/keyfile
EOF

cat >> /mongodb/10004/mongod.conf << EOF
security:
  keyFile: /mongodb/10004/keyfile
EOF

cat >> /mongodb/10005/mongod.conf << EOF
security:
  keyFile: /mongodb/10005/keyfile
EOF


# 然后重启所有节点，
# 先停止从节点，最后停止主节点
mongo --port 10005
mongo --port 10004
mongo --port 10003
mongo --port 10002
mongo --port 10001

use admin
db.shutdownServer()


# 启动所有节点
mongod -f /mongodb/10001/mongod.conf
mongod -f /mongodb/10002/mongod.conf
mongod -f /mongodb/10003/mongod.conf
mongod -f /mongodb/10004/mongod.conf
mongod -f /mongodb/10005/mongod.conf


# 登录主节点创建用户

use admin
db.createUser(
{
    user: 'root',
    pwd: 'root123',
    roles: [{ role: 'root', db: 'admin'}]
})

# 手工交互输入密码
db.createUser(
{
    user:'root1',
    pwd: passwordPrompt(),
    roles: [{ role: 'root', db: 'admin'}]
})

# 手工交互验证
my_repl:PRIMARY> use admin
switched to db admin
my_repl:PRIMARY> db.auth("root1", passwordPrompt())
Enter password:
1

# 在其他节点上就可以使用 root、root1用户来验证了
my_repl:SECONDARY> use admin
switched to db admin
my_repl:SECONDARY> db.auth("root1", passwordPrompt())
Enter password:
1
my_repl:SECONDARY> rs.status()