By   May 24, 2017
0

Curator for Managing Elastic Search Indices

Curator is one of the officially supported plugins of Elastic Search. It supports a number of features including the following:

  • Add or remove indices (or both!) from an alias
  • Change shard routing allocation
  • Close indices
  • Create index
  • Delete indices
  • Delete snapshots
  • Open closed indices
  • forceMerge indices
  • reindex indices, including from remote clusters
  • Change the number of replicas per shard for indices
  • rollover indices
  • Take a snapshot (backup) of indices
  • Restore snapshots

Is this plugin officially supported?
Plugin is officially supported ElasticSearch client:
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html

Is this Open Source:
Yes, the code is available here:

https://github.com/elastic/curator

Purging Index Data

If space is a big concern for Elastic Search databases where you keep your logs then we can always schedule purging older data. Let’s download the plugin first:

https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html

Required Configurations
We need to update two configuration files here:

  1. action.yml
  2. config.yml

action.yml
Here we are deleting all indices which follows logstash- where index data is older than 6 days.

actions:
  1:
    action: delete_indices
    description: "Delete Older Indices"
    options:
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
     - filtertype: pattern
       kind: prefix
       value: logstash-
     - filtertype: age
       source: name
       direction: older
       timestring: '%Y.%m.%d'
       unit: days
       unit_count: 6
       field:
       stats_result:
       epoch:
       exclude: False

config.yml
This is used to provide connection information for host. It is also used to specify log configuration:

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts: HOST_NAME
  port: 9200
  use_ssl: False
  ssl_no_validate: False
  timeout: 30
  master_only: False

logging:
  loglevel: INFO
  logformat: default
  blacklist: []

Running Curator
We can run curator as follows:

curator.exe --config config.yml action.yml