Before reading this article, you may want to read the following article to learn the concepts related to Elasticsearch.
We determine the number of replica and shard during indice creation. For example, when we create an indice with 16 shard and 2 replicas in a system with 3 data nodes, elasticsearch distributes the data to data nodes as follows.
As you can see in the screenshot below, a copy of each shard was created because we set the replica as 2. In order to monitor Nodes, Indices and Shards in Elasticsearch, I suggest you read the article “How To Monitor Elasticsearch Nodes, Indices and Shards Using Kibana“.
Create an Elasticsearch indice
First, we’re connecting to Kibana. To connect to Kibana, you can read the article “How To Install Kibana On Centos“.
Then click on “Dev Tools” to create an index with 2 replicas and 16 shard using the command below. The number_of_replicas value must be 1 for 2 replica. To complete the process, click to “click to send request”.
|
1 2 3 4 5 6 7 8 |
PUT my_indice_name { "settings": { "number_of_replicas": 1 , "number_of_shards": 16 } } |
If the script has been successfully completed and the indice is created, the values “true” should return as a result, as you can see in the screenshot below.
When creating Indices, we can ensure that the shards and replicas are created only on certain data nodes. To do this, we open the elasticsearch.yml file on each data node and define a rack name. For example, the r1 name is defined as the rack name in the example below.
|
1 |
node.attr.rack: r1 |
|
1 2 3 4 5 6 7 8 9 |
PUT my_indice_name { "settings": { "number_of_replicas": 1 , "number_of_shards": 16 , "index.routing.allocation.include.rack": "r1,r2" } } |
Specify IP:
|
1 2 3 4 5 6 7 8 9 |
PUT my_indice_name { "settings": { "number_of_replicas": 1 , "number_of_shards": 16 , "index.routing.allocation.include._ip": "10.172.2.32" } } |
Specify VLAN:
|
1 2 3 4 5 6 7 8 9 |
PUT my_indice_name { "settings": { "number_of_replicas": 1 , "number_of_shards": 16 , "index.routing.allocation.include._ip": "10.172.2.*" } } |
Mapping
Usually, we transfer our data from other database systems to elasticsearch. If we don’t perform mapping in elasticsearch, columns are transferred to elasticsearch as text. Therefore, it is more effective to create each column with specified type in index(this is mapping) before transfer our data.
You can see an indice create script with all elasticsearch core dataypes below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
PUT my_indexx { "mappings": { "type":{ "properties": { "name": { "type": "text" }, --- text data type "age": { "type": "integer" }, --- integer long smallint int, "created": { "type": "date" --- date datetim date type }, "salary": { "type": "float" }, -- virgüllü int "image": { "type": "binary" } -- image verisi } } },"settings": {"number_of_replicas": 1, "number_of_shards": 5 } } |
