How to perform basic Elasticsearch operations

Perform basic Elasticsearch 7.x operations.

This blog post describes only basic operations and it is already too long. There is a lot more to learn, so I strongly suggest to study Elasticsearch Reference.

Create index

Create bookmarks_index index.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index?pretty=true \
       --data-ascii \
'{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  },  
  "mappings": {
    "properties" : {
      "name": {"type": "keyword"},
      "address": {"type": "keyword"},
      "date": {"type": "date"},
      "tag": {"type": "keyword", "store": true}
    }
  }
}'
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "bookmarks_index"
}

Alter index settings

Alter index settings.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index/_settings?pretty=true \
       --data-ascii \
'{
  "index": {
    "number_of_replicas": "0"
  }
}'
{
  "acknowledged" : true
}

Display index settings.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_settings?pretty=true
{
  "bookmarks_index" : {
    "settings" : {
      "index" : {
        "creation_date" : "1578215671819",
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
        "version" : {
          "created" : "7050199"
        },
        "provided_name" : "bookmarks_index"
      }
    }
  }
}

Alter index mappings

You cannot modify existing field as it could invalidate data that is already indexed. To perform such operation you need to create new index that contains updated mapping and reindex data into it.

Alter index mappings to add an alias.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index/_mappings?pretty=true \
       --data-ascii \
'{
  "properties": {
    "url": {
      "type": "alias",
      "path": "address"
    }
  }
}'
{
  "acknowledged" : true
}

Display index mappings.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_mappings?pretty=true
{
  "bookmarks_index" : {
    "mappings" : {
      "properties" : {
        "address" : {
          "type" : "keyword"
        },
        "date" : {
          "type" : "date"
        },
        "name" : {
          "type" : "keyword"
        },
        "tag" : {
          "type" : "keyword",
          "store" : true
        },
        "url" : {
          "type" : "alias",
          "path" : "address"
        }
      }
    }
  }
}

Alter index aliases

Add aliases to an index.

$ curl --silent \
       --request PUT \
       http://10.0.2.15:9200/bookmarks_index/_alias/bookmarks?pretty=true 
{
  "acknowledged" : true
}
$ curl --silent \
       --request PUT \
       http://10.0.2.15:9200/bookmarks_index/_alias/mybookmarks?pretty=true 
{
  "acknowledged" : true
}

Delete an index alias.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks_index/_alias/mybookmarks?pretty=true 
{
  "acknowledged" : true
}

Display index aliases.

$ curl --silent \
       http://10.0.2.15:9200/bookmarks_index/_alias?pretty=true
{
  "bookmarks_index" : {
    "aliases" : {
      "bookmarks" : { }
    }
  }
}

Check if index exists

Determine if non-existent index exists.

$ curl --silent --head http://10.0.2.15:9200/bookmark/
HTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 377

Determine if index exists using its alias.

$ curl --silent --head http://10.0.2.15:9200/bookmarks/
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 433

Determine if index exists using its real name.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 433

List indices

List indices.

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          0            0       283b           283b
$ curl --silent "http://10.0.2.15:9200/_cat/indices?v&h=index,status"
index           status
bookmarks_index open

Create documents

Add some documents and wait for a refresh each time.

$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Personal blog", 
   "address": "https://blog.sleeplessbeastie.eu", 
   "tag": ["personal", "blog"],
   "date":"2020-01-05"
}'
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "V5gUdW8BkI-uNnF5JfU7",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}
$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Debian", 
   "address": "https://www.debian.org", 
   "tag": ["linux", "debian"],
   "date":"2020-01-05"
}'
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WJgYdW8BkI-uNnF5hvVG",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}
$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Debian packages", 
   "address": "https://www.debian.org/distrib/packages", 
   "tag": ["linux", "debian", "packages"],
   "date":"2020-01-05"
}'
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 1
}

Search documents

Match all documents.

$ curl --silent \
       --header 'Content-Type: application/json' \
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "match_all": {} 
   }
}'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "V5gUdW8BkI-uNnF5JfU7",
        "_score" : 1.0,
        "_source" : {
          "name" : "Personal blog",
          "address" : "https://blog.sleeplessbeastie.eu",
          "tag" : [
            "personal",
            "blog"
          ],
          "date" : "2020-01-05"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WJgYdW8BkI-uNnF5hvVG",
        "_score" : 1.0,
        "_source" : {
          "name" : "Debian",
          "address" : "https://www.debian.org",
          "tag" : [
            "linux",
            "debian"
          ],
          "date" : "2020-01-05"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 1.0,
        "_source" : {
          "name" : "Debian packages",
          "address" : "https://www.debian.org/distrib/packages",
          "tag" : [
            "linux",
            "debian",
            "packages"
          ],
          "date" : "2020-01-05"
        }
      }
    ]
  }
}

Exact match query.

$ curl --silent \
       -header 'Content-Type: application/json'
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "term": { "url":"https://www.debian.org/distrib/packages"}
  }, 
  "_source": ["name", "address"]
}'
{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.9808292,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 0.9808292,
        "_source" : {
          "address" : "https://www.debian.org/distrib/packages",
          "name" : "Debian packages"
        }
      }
    ]
  }
}

Wildcard query.

$ curl --silent \
       --header 'Content-Type: application/json'
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "wildcard": { "url":"*debian*"} 
  }, 
  "_source": ["name", "address"]
}'
{
  "took" : 27,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WJgYdW8BkI-uNnF5hvVG",
        "_score" : 1.0,
        "_source" : {
          "address" : "https://www.debian.org",
          "name" : "Debian"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 1.0,
        "_source" : {
          "address" : "https://www.debian.org/distrib/packages",
          "name" : "Debian packages"
        }
      }
    ]
  }
}

More useful query that contains multiple query and filter context.

$ curl --silent \
       --header 'Content-Type: application/json' \
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "bool": {
      "must": [
        {"wildcard": { "url":"*debian*"}}, 
        {"prefix": { "url":"https://*"}}
      ],
      "filter": [
        { "term": { "tag": "packages"}},
        { "range": { "date": { "gte": "2020-01-05" }}}
      ]  
    }
  } 
}'
{
  "took" : 52,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 2.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 2.0,
        "_source" : {
          "name" : "Debian packages",
          "address" : "https://www.debian.org/distrib/packages",
          "tag" : [
            "linux",
            "debian",
            "packages"
          ],
          "date" : "2020-01-05"
        }
      }
    ]
  }
}

Update documents

Perform partial update.

$ curl  --silent \
        --header 'Content-Type: application/json' \
        --request POST \
        http://10.0.2.15:9200/bookmarks/_update/WZgZdW8BkI-uNnF5AfWy?pretty=true \
        --data-ascii \
'{ 
   "doc": {
      "name": "Debian - Packages"
    }
}'
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 3,
  "_primary_term" : 1
}

Perform scripted update.

$ curl  --silent \
        --header 'Content-Type: application/json' \
        --request POST \
        http://10.0.2.15:9200/bookmarks/_update/WZgZdW8BkI-uNnF5AfWy?pretty=true \
        --data-ascii \
'{
    "script" : {
        "source": "if (ctx._source.tag.contains(params.tag)) { ctx._source.tag.remove(ctx._source.tag.indexOf(params.tag)) }",
        "lang": "painless",
        "params" : {
            "tag" : "packages"
        }
    }
}'
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 3,
  "result" : "updated",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 4,
  "_primary_term" : 1
}

Get document.

Get single document.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_doc/WZgZdW8BkI-uNnF5AfWy?pretty=true
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 3,
  "_seq_no" : 4,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "Debian - Packages",
    "address" : "https://www.debian.org/distrib/packages",
    "tag" : [
      "linux",
      "debian"
    ],
    "date" : "2020-01-05"
  }
}

Get multiple documents.

$ curl --silent \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks/_mget?pretty=true \
       --data-ascii \
'{ 
   "ids": [ "WZgZdW8BkI-uNnF5AfWy","WJgYdW8BkI-uNnF5hvVG" ]
}'
{
  "docs" : [
    {
      "_index" : "bookmarks_index",
      "_type" : "_doc",
      "_id" : "WZgZdW8BkI-uNnF5AfWy",
      "_version" : 3,
      "_seq_no" : 4,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "Debian - Packages",
        "address" : "https://www.debian.org/distrib/packages",
        "tag" : [
          "linux",
          "debian"
        ],
        "date" : "2020-01-05"
      }
    },
    {
      "_index" : "bookmarks_index",
      "_type" : "_doc",
      "_id" : "WJgYdW8BkI-uNnF5hvVG",
      "_version" : 1,
      "_seq_no" : 1,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "Debian",
        "address" : "https://www.debian.org",
        "tag" : [
          "linux",
          "debian"
        ],
        "date" : "2020-01-05"
      }
    }
  ]
}

Delete document

Delete document.

$ curl --silent \
       --request DELETE \
       "http://10.0.2.15:9200/bookmarks_index/_doc/V5gUdW8BkI-uNnF5JfU7?pretty=true"
{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "V5gUdW8BkI-uNnF5JfU7",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 5,
  "_primary_term" : 1
}

Notice, document will become unavailable, but it will not be deleted immediately.

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          3            1     22.1kb         22.1kb

You really don't need to, but you can use force merge to only expunge segments containing document deletions.

$ curl --silent \
       --request POST \
      "http://10.0.2.15:9200/bookmarks/_forcemerge?only_expunge_deletes=true&pretty=true"
{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}
$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          2            0      4.7kb           4.7k

Count documents

Count documents.

$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true
{
  "count" : 2,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Check if document exists

Determine if non-existing/deleted document exists.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/_doc/V5gUdW8BkI-uNnF5JfU7
HTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 86

Determine if document exists.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/_doc/WJgYdW8BkI-uNnF5hvVG
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 247

Disable index

Ensure that each shard has only a single segment on disk.

$ curl --silent \
       --request POST \
      "http://10.0.2.15:9200/bookmarks/_forcemerge?max_num_segments=1&pretty=true"
{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}

Close index to disable read/write operations.

$ curl --silent \
       --request POST \
       http://10.0.2.15:9200/bookmarks/_close?pretty=true
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : {
    "bookmarks_index" : {
      "closed" : true
    }
  }
}

You cannot perform read/write operations.

$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_closed_exception",
        "reason" : "closed",
        "index_uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
        "index" : "bookmarks_index"
      }
    ],
    "type" : "index_closed_exception",
    "reason" : "closed",
    "index_uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
    "index" : "bookmarks_index"
  },
  "status" : 400
}

Open index to re-enable read/write operations.

$ curl --silent \
       --request POST \
       http://10.0.2.15:9200/bookmarks/_open?pretty=true
{
  "acknowledged" : true,
  "shards_acknowledged" : true
}
$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true
{
  "count" : 2,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Delete index

Do not use an alias to delete index.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks?pretty=true
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "The provided expression [bookmarks] matches an alias, specify the corresponding concrete indices instead."
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "The provided expression [bookmarks] matches an alias, specify the corresponding concrete indices instead."
  },
  "status" : 400
}

Delete index using its real name.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks?pretty=true
{
  "acknowledged" : true
}