How To Install Mongodb Sharded Cluster with Keyfile Access Control on Red Hat or CentOS

As you know security is one of the key points during database administration. In MongoDB sharded clusters security between components of the cluster is provided by Internal authentication. In this tutorial I will deploy a sharded cluster and enable internal authentication by using a keyfile.

Enforcing internal authentication also enforces user access control. To connect to the replica set, clients like the mongo shell need to use a user account.

Step 1: Create a User For Administration

Important Note: Create mongodb user in all servers and perform all operations with this user.

Step 2: Install MongoDB Community Edition

Using .rpm Packages (Recommended)

Configure the package management system (yum).

Create a /etc/yum.repos.d/mongodb-org-4.0.repo file so that you can install MongoDB directly using yum:

Note: If you want to install difference version you must change 4.0 in the above content.

Install MongoDB packages

To install the latest stable version of MongoDB, issue the following command:

Step 3: Disable SELINUX

If you are using SELinux, you must configure SELinux to allow MongoDB to start on Red Hat Linux-based systems (Red Hat Enterprise Linux or CentOS Linux).

Disable SELinux by setting the SELINUX setting to disabled in /etc/selinux/config.

Step 4: Stop Firewalld and Disable iptables

Disable and Stop Firewalld

Stop the firewall so the servers can communicate with each other.

Disable iptables

Step 5:  Recommended ulimit Settings

Configure some ulimit parameters as below for mongodb.

Edit limits.conf:

Edit 20-nproc.conf:

Edit 20-nproc.conf and update the value as below. 

• -f (file size): unlimited
• -t (cpu time): unlimited
• -v (virtual memory): unlimited
• -l (locked-in-memory size): unlimited
• -n (open files): 64000
• -m (memory size): unlimited
• -u (processes/threads): 64000

Note: You must logout and login or restart the server to activate changes.

Make sure that your ulimit settings are configured correctly. You can check current ulimit settings with the command below:

Step 6: Disable Transparent Huge Pages

Before starting mongod processes, it is recommended to disable Transparent Huge Pages. Mongodb often perform poorly with THP enabled. You can use the following document:

https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/

Step 7: Create Directories

Create a folder structure to store database files. The path structure will be as follows. You may want to change this path structure. Its your choice. 

Note: Create directories with mongodb user.

hostname1

hostname2

hostname3

Step 8: Add IP Addresses and Hostnames to /etc/hosts

We need to add hostname informations to /etc/hosts in all nodes to access between servers by the hostnames.

Step 9: Create a Keyfile

You can generate a keyfile using any method you choose. For example, the following operation uses openssl to generate a complex pseudo-random 1024 character string to use for a keyfile.
It then uses chmod to change file permissions to provide read permissions for the file owner only:

Copy the keyfile to each server hosting the sharded cluster members. Ensure that the user running the mongod or mongos instances is the owner of the file and can access the keyfile.

Copy the file to other nodes:

Step 10: Configure Config Servers

We will use a configuration file in this deployment so we will set :

security.keyFile to the keyfile’s path,
sharding.clusterRole to configsvr

Sample configuration file for config server:

There is a sample configuration file /etc/mongod.conf or you can use the below one according to our path structure. 

Create config-cfg.conf file on each config server

Content of the files:

hostname1:

hostname2:

hostname3:

You can include additional options as required for your configuration.

For instance, our deployment members are run on different hosts so we specify the net.bindIp setting here.

Start mongod

Start the mongod on each config server.

Now we have started the mongodb instance but we don’t have any authenticated users so we can do nothing!!

In this state MongoDB has an exception called “localhost exception”. By this we can create only one user and this user can be created only connecting from the localhost.

When all the replica members start running, it is time to initiate replica sets.

Grant Roles:

user1 is your username.

If you want to grant root privilege execute the below command to be able to perform all operations:

Note: If you want to use db.updateUser command, ensure all roles is available in the command. Because when you execute db.updateUser, it replaces old permissions. Thats why I use db.grantRolesToUser().

Connect to the one of the members of the config server :

Shutdown all the config replica set members:

Add below lines to each config replica set member’s configuration file:

Write myconfigreplicasetname instead of yourreplicasetname in our example. Because we specify our replica set name as “myconfigreplicasetname” in our config file.

Start mongod instance again:

Initiate config server’s replica sets:

Step 11: Configure Shards

I will create 3 shard shA, shB and shC in my example. There will be 2 copy of each shard. Therefore, we need 9 conf file.

I will create shard shA on hostname1 and shB on hostname2 and shC on hostname3, but this does not mean shA only resides on hostname1. This means that hostname1 will be the primary node of shA and hostname2 will be the primary node of shB and hostname3 will be the primary node of shC. Other nodes of each shard will be secondary nodes.

If you want, you can create all the shards on hostname1. If you perform all operations from hostname1, primary of all shards will be hostname1.

Shard Distribution in our Example

hostname1	hostname2	hostname3
shA_Primary	shA_Secondary	shA_Secondary
shB_Secondary	shB_Primary	shB_Secondary
shC_Secondary	shC_Secondary	shC_Primary

Create Shard shA on hostname1

Create a config file for shard shA:

Content of the file:

Note: Copy this file to hostname2 and hostname3 and change hostname.

Start mongod

Start the mongod on each shard server specifying the –config option and the path to the configuration file.

Now we have started the mongodb instance but we don’t have any authenticated users so we can do nothing!!

In this state MongoDB has an exception called “localhost exception”. By this we can create only one user and this user can be created only connecting from the localhost.

Grant Roles:

user1 is your username.

If you want to grant root privilege execute the below command to be able to perform all operations:

Connect to the one of the members of the shard server :

Shutdown all the shard replica set members:

Add below lines to each shard replica set member’s configuration file:

Write shA instead of <yourreplicasetname> in our example.

Start mongod instance again:

Initiate shA:

Create Shard shB on hostname2

Create a config file for shard shB:

Content of the file:

Note: Copy this file to hostname1 and hostname3 and change hostname.

Start mongod

Start the mongod on each shard server specifying the –config option and the path to the configuration file.

Now we have started the mongodb instance but we don’t have any authenticated users so we can do nothing!!

In this state MongoDB has an exception called “localhost exception”. By this we can create only one user and this user can be created only connecting from the localhost.

Grant Roles:

user1 is your username.

If you want to grant root privilege execute the below command to be able to perform all operations:

Connect to the one of the members of the shard server :

Shutdown all the shard replica set members:

Add below lines to each shard replica set member’s configuration file:

Write shB instead of <yourreplicasetname> in our example.

Start mongod instance again:

Initiate shB:

Create Shard shC on hostname3

Create a config file for shard shC:

Content of the file:

Note: Copy this file to hostname1 and hostname2 and change hostname.

Start mongod

Start the mongod on each shard server specifying the –config option and the path to the configuration file.

Now we have started the mongodb instance but we don’t have any authenticated users so we can do nothing!!

In this state MongoDB has an exception called “localhost exception”. By this we can create only one user and this user can be created only connecting from the localhost.

Grant Roles:

user1 is your username.

If you want to grant root privilege execute the below command to be able to perform all operations:

Connect to the one of the members of the shard server :

Shutdown all the shard replica set members:

Add below lines to each shard replica set member’s configuration file:

Write shC instead of <yourreplicasetname> in our example.

Start mongod instance again:

Initiate shC:

Step 12: Start Mongoses

Now it’s time to run mongoses. There is no replica for mongoses. But you can multiplies mongoses. I will create 3 mongoses in our example.

Create a mongos file on all mongos servers:

mongos config file is like this:

Note: Copy this file to other mongos servers and change hostnames.

To run the mongos process:

Step 13: Check All Mongos Components Started

Step 14: Connect to mongos and add shards

Note: shA,shB and shC our replica set names. If your replica set names is different, change the script above according to your replica set names.

Check sharded cluster status:

Connect mongos and run below command.

The sharded replica set configuration is done. Now you can restore a dump and start working on dbs 🙂

You may want to read below articles:

“Back Up a Sharded Cluster with File System Snapshots“,

“Automatizing backup process on sharded clusters“,

“Restore a Sharded Cluster with Database Dumps in MongoDB“