文章目录

一、环境规划
二、安装部署
- 2.1 安装包部署
- 2.2 config serve 配置服务器部署
- 2.3 shard 分片集群部署
- 2.4 mongos 路由服务器部署
三、分片集群功能测试

MongoDB分片集群主要由以下组件组成：

shard : 主要用来实际存储分片数据，每个分片都可以部署为一个副本集
mongos : 主要用来做查询路由器，将客户端请求与与各分片之间数据进行协调路由
config servers : 主要用来存储整个集群的元数据以及配置信息

MongoDB分片集群搭建部署一、环境规划二、安装部署三、分片集群功能测试

一、环境规划

1、服务器部署规划

服务器	IP	部署角色
sdw1	172.16.104.12	shard1(primary)、shard2(secondry)、shard3(secondry)、mongos、config server
sdw2	172.16.104.13	shard1(secondry)、shard2(primary)、shard3(secondry)、mongos、config server
sdw3	172.16.104.14	shard1(secondry)、shard2(secondry)、shard3(primary)、mongos、config server

2、端口规划

角色	端口
shard1	27001
shard2	27002
shard3	27003
mongos	20000
config server	21000

3、目录规划

角色	目录
config server	/data/mongodb40/config/{data,logs,conf}
shard	/data/mongodb40/shard{N}/{data,logs,conf}
mongos	/data/mongodb40/mongos/{logs,conf}

其中，data为数据目录、logs为日志目录、conf目录存储该节点的config文件以及keyfile文件。

二、安装部署

2.1 安装包部署

1、安装包部署

-- MongoDB官网
https://www.mongodb.com/try/download/community

-- 最新稳定版4.4下载
# wget -c https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.0.22.tgz

-- 解压
# tar xf mongodb-linux-x86_64-rhel70-4.0.22.tgz -C /usr/local/
# mv mongodb-linux-x86_64-rhel70-4.0.22/ mongodb40

2、环境配置

# vim /root/.bash_profile
PATH=$PATH:$HOME/bin:/usr/local/mongodb40/bin

# source /root/.bash_profile

3、创建规划目录

# mkdir -p /data/mongodb40/{shard1,shard2,shard3,config}/{data,logs,conf}
# mkdir -p /data/mongodb40/mongos/{conf,logs}

2.2 config serve 配置服务器部署

1、配置文件

[[email protected] conf]# cat /data/mongodb40/config/conf/config.conf
systemLog:
   verbosity: 0
   quiet: false
   traceAllExceptions: false
   path: "/data/mongodb40/config/logs/config.log"               # 日志目录
   logAppend: true                                              # 日志追加写入
   logRotate: reopen
   destination: file
processManagement:
   fork: true
   pidFilePath: "/data/mongodb40/config/conf/config.pid"        # pid文件
net:
   port: 21000                                                  # config server 监听端口【关键】
   bindIp: 0.0.0.0                                              # 开放IP【关键】
   maxIncomingConnections: 2000
   wireObjectCheck: true
security:
   keyFile: "/data/mongodb40/config/conf/KeyFile.file"          # Keyfile文件
   clusterAuthMode: keyFile
   authorization: enabled                                       # 是否开启用户密码认证
storage:
   dbPath: /data/mongodb40/config/data                          # 数据目录
   journal:
      enabled: true
      commitIntervalMs: 100
   directoryPerDB: true
   syncPeriodSecs: 60
   engine: wiredTiger
   wiredTiger:
      engineConfig:
         cacheSizeGB: 256
         #journalCompressor: snappy
         directoryForIndexes: false
      collectionConfig:
         blockCompressor: snappy
      indexConfig:
         prefixCompression: false
operationProfiling:
   slowOpThresholdMs: 100
   mode: slowOp
replication:
   #oplogSizeMB: 1024
   replSetName: configs
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: false
sharding:
   clusterRole: configsvr                                       # 定义为config server【关键】

2、生成keyfile

# openssl rand -base64 90 -out  /data/mongodb40/config/conf/KeyFile.file
# chmod 600  /data/mongodb40/config/conf/KeyFile.file

3、将第一个节点的keyfile、config文件传输至其他config server节点

# scp -r ./config.conf ./KeyFile.file sdw2:/data/mongodb40/config/conf/
# scp -r ./config.conf ./KeyFile.file sdw3:/data/mongodb40/config/conf/

4、启动config server副本集的各个节点

# mongod -f /data/mongodb40/config/conf/config.conf  &

5、初始化副本集

随便登录一个config server节点，进行config server副本集的初始化。

> cnf = {"_id":"configs","members":[{"_id":1,"host":"172.16.104.12:21000","priority":1 },{"_id":2, "host":"172.16.104.13:21000","priority":1},{"_id":3, "host":"172.16.104.14:21000","priority":1}]}
> rs.initiate(cnf)

6、创建用户

configs:PRIMARY> db.createUser({"user":"root","pwd":"123","roles":[{role:"root",db:"admin"}]})

2.3 shard 分片集群部署

1、配置文件

[[email protected] conf]# cat /data/mongodb40/shard1/config.conf
systemLog:
   verbosity: 0
   quiet: false
   traceAllExceptions: false
   path: "/data/mongodb40/shard1/logs/config.log"
   logAppend: true
   logRotate: reopen
   destination: file
processManagement:
   fork: true
   pidFilePath: "/data/mongodb40/shard1/conf/config.pid"
net:
   port: 27001                                              # shard1副本集节点端口
   bindIp: 0.0.0.0                                          # 开放IP
   maxIncomingConnections: 2000
   wireObjectCheck: true
security:
   keyFile: "/data/mongodb40/shard1/conf/KeyFile.file"
   clusterAuthMode: keyFile
   authorization: enabled
   #authorization: disabled
storage:
   dbPath: /data/mongodb40/shard1/data
   #indexBuildRetry: true
   journal:
      enabled: true
      commitIntervalMs: 100
   directoryPerDB: true
   syncPeriodSecs: 60
   engine: wiredTiger
   wiredTiger:
      engineConfig:
         cacheSizeGB: 256
         #journalCompressor: snappy
         directoryForIndexes: false
      collectionConfig:
         blockCompressor: snappy
      indexConfig:
         prefixCompression: false
operationProfiling:
   slowOpThresholdMs: 100
   mode: slowOp
replication:
   #oplogSizeMB: 1024
   replSetName: shard1
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: false
sharding:
   clusterRole: shardsvr                                    # 标识节点为shard分片节点

2、keyfile、config文件复制到相应的分片节点下

-- 直接传输之前创建的keyfile
# scp -r ./config.conf ./KeyFile.file sdw2:/data/mongodb40/config/conf/
# scp -r ./config.conf ./KeyFile.file sdw3:/data/mongodb40/config/conf/

3、启动shard1副本集各节点

# mongod -f /data/mongodb40/shard1/conf/config.conf &

5、初始化副本集

随便登录一个shard1节点，进行副本集的初始化。

> cnf = {"_id":"shard1","members":[{"_id":1,"host":"172.16.104.12:27001","priority":10 },{"_id":2, "host":"172.16.104.13:27001","hidden":true,"priority":0},{"_id":3, "host":"172.16.104.14:27001","priority":1}]}
> rs.initiate(cnf)

6、重复以上操作对shard2、shard3进行初始化，各shard副本集初始化配置

-- shard1
cnf = {"_id":"shard1","members":[{"_id":1,"host":"172.16.104.12:27001","priority":10 },{"_id":2, "host":"172.16.104.13:27001","priority":1},{"_id":3, "host":"172.16.104.14:27001","priority":1}]}

-- shard2
cnf = {"_id":"shard2","members":[{"_id":1,"host":"172.16.104.12:27002","priority":1 },{"_id":2, "host":"172.16.104.13:27002","priority":10},{"_id":3, "host":"172.16.104.14:27002","priority":1}]}

-- shard3
cnf = {"_id":"shard3","members":[{"_id":1,"host":"172.16.104.12:27003","priority":1},{"_id":2, "host":"172.16.104.13:27003","priority":1},{"_id":3, "host":"172.16.104.14:27003","priority":10}]}

8、创建用户

use admin
db.createUser({"user":"root","pwd":"123","roles":[{role:"root",db:"admin"}]})

2.4 mongos 路由服务器部署

1、配置文件

[[email protected] conf]# cat /data/mongodb40/config/conf/config.conf
systemLog:
   verbosity: 0
   quiet: false
   traceAllExceptions: false
   path: "/data/mongodb40/mongos/logs/config.log"
   logAppend: true
   logRotate: reopen
   destination: file
processManagement:
   fork: true
   pidFilePath: "/data/mongodb40/mongos/conf/config.pid"
net:
   port: 20000
   bindIp: 0.0.0.0
   maxIncomingConnections: 2000
   wireObjectCheck: true
security:
   keyFile: "/data/mongodb40/mongos/conf/KeyFile.file"
   clusterAuthMode: keyFile
sharding:
   configDB: configs/172.16.104.12:21000,172.16.104.13:21000,172.16.104.14:21000            # configDB : <config_repl_name>/config节点ip:port

2、keyfile

-- 直接传输之前创建的keyfile
# scp -r ./config.conf ./KeyFile.file sdw2:/data/mongodb40/config/conf/
# scp -r ./config.conf ./KeyFile.file sdw3:/data/mongodb40/config/conf/

3、启动mongos下各节点，注意是使用mongos启动，而不是mongod

# mongos -f /data/mongodb40/mongos/conf/config.conf &

4、为分片集群添加分片节点信息

随便登录一个mongos节点，添加对应的shard节点。

-- 登录mongos节点并进行用户认证
# mongo --port 20000
mongos> use admin
switched to db admin
mongos> db.auth("root","123")
1

-- 为分片集群添加分片信息
mongos> sh.addShard("shard1/172.16.104.12:27001,172.16.104.13:27001,172.16.104.14:27001")               # 添加shard1
mongos> sh.addShard("shard2/172.16.104.12:27002,172.16.104.13:27002,172.16.104.14:27002")               # 添加shard2
mongos> sh.addShard("shard3/172.16.104.12:27003,172.16.104.13:27003,172.16.104.14:27003")               # 添加shard3

-- 查看分片集群信息
mongos> sh.status()
--- Sharding Status ---
  sharding version: {
  	"_id" : 1,
  	"minCompatibleVersion" : 5,
  	"currentVersion" : 6,
  	"clusterId" : ObjectId("6069b489d65227986c7fbe7a")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/172.16.104.12:27001,172.16.104.13:27001,172.16.104.14:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/172.16.104.12:27002,172.16.104.13:27002,172.16.104.14:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/172.16.104.12:27003,172.16.104.13:27003,172.16.104.14:27003",  "state" : 1 }
  active mongoses:
        "4.0.22" : 3
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }

三、分片集群功能测试

1、对db1.test1进行分片

mongos> sh.enableSharding("db1");
{
	"ok" : 1,
	"operationTime" : Timestamp(1617541590, 5),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1617541590, 5),
		"signature" : {
			"hash" : BinData(0,"8DW8X4Bx5zkMHiCJ2LQzoTkqi4g="),
			"keyId" : NumberLong("6947282400699219974")
		}
	}
}
mongos> sh.shardCollection("db1.test1", {_id: 'hashed'});
{
	"collectionsharded" : "db1.test1",
	"collectionUUID" : UUID("33adf1e2-128e-4ee3-8624-2adaf5311468"),
	"ok" : 1,
	"operationTime" : Timestamp(1617541590, 76),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1617541590, 76),
		"signature" : {
			"hash" : BinData(0,"8DW8X4Bx5zkMHiCJ2LQzoTkqi4g="),
			"keyId" : NumberLong("6947282400699219974")
		}
	}
}

2、查看配置集合分片后的状态

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
  	"_id" : 1,
  	"minCompatibleVersion" : 5,
  	"currentVersion" : 6,
  	"clusterId" : ObjectId("6069b489d65227986c7fbe7a")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/172.16.104.12:27001,172.16.104.13:27001,172.16.104.14:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/172.16.104.12:27002,172.16.104.13:27002,172.16.104.14:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/172.16.104.12:27003,172.16.104.13:27003,172.16.104.14:27003",  "state" : 1 }
  active mongoses:
        "4.0.22" : 3
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }

3、模拟通过mongos进行数据写入

mongos> use db1
switched to db db1
mongos> for (var i = 0; i < 10000; i++) {db.test1.insert({i: i});}
WriteResult({ "nInserted" : 1 })
mongos> db.test1.find().count()
18378

4、查看数据写入后shard的数据分布状态，过一段时间后，可以发现每个shard的数据分布基本比较平均

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
  	"_id" : 1,
  	"minCompatibleVersion" : 5,
  	"currentVersion" : 6,
  	"clusterId" : ObjectId("6069b489d65227986c7fbe7a")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/172.16.104.12:27001,172.16.104.13:27001,172.16.104.14:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/172.16.104.12:27002,172.16.104.13:27002,172.16.104.14:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/172.16.104.12:27003,172.16.104.13:27003,172.16.104.14:27003",  "state" : 1 }
  active mongoses:
        "4.0.22" : 3
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                682 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1	342
                                shard2	341
                                shard3	341
                        too many chunks to print, use verbose if you want to force print
        {  "_id" : "db1",  "primary" : "shard2",  "partitioned" : true,  "version" : {  "uuid" : UUID("bd30b8c8-a9f5-4bdd-917e-2734e4a64ae1"),  "lastMod" : 1 } }
                db1.test1
                        shard key: { "_id" : "hashed" }
                        unique: false
                        balancing: true
                        chunks:
                                shard1	2
                                shard2	2
                                shard3	2
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-6148914691236517204") } on : shard1 Timestamp(1, 0)
                        { "_id" : NumberLong("-6148914691236517204") } -->> { "_id" : NumberLong("-3074457345618258602") } on : shard1 Timestamp(1, 1)
                        { "_id" : NumberLong("-3074457345618258602") } -->> { "_id" : NumberLong(0) } on : shard2 Timestamp(1, 2)
                        { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("3074457345618258602") } on : shard2 Timestamp(1, 3)
                        { "_id" : NumberLong("3074457345618258602") } -->> { "_id" : NumberLong("6148914691236517204") } on : shard3 Timestamp(1, 4)
                        { "_id" : NumberLong("6148914691236517204") } -->> { "_id" : { "$maxKey" : 1 } } on : shard3 Timestamp(1, 5)

mongos>

MongoDB分片集群搭建部署一、环境规划二、安装部署三、分片集群功能测试

文章目录

一、环境规划

二、安装部署

2.1 安装包部署

2.2 config serve 配置服务器部署

2.3 shard 分片集群部署

2.4 mongos 路由服务器部署

三、分片集群功能测试

继续阅读

windows 解压安装Mogdb

mongoDB 数据库中导入导出数据的方法

django 使用mongodb数据库 celery并发自增字段

mongodb导出导入数据库

MYSQL数据库数据导入mongoDB

windows导出导入mongodb数据库

解决MongoDB官网下载速度过慢Mongo DB官网：解决：

探花交友_第2章-完善个人信息与MongoDB入门

启动mongo warning

MongoDB学习笔记一ID自增长

303、副本集如何扩容

Mongodb4.0操作命令

Error: couldn‘t connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error

Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Erro

couldn‘t connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error conne

Ubuntu14.04 LTS下安装mongodb