http://api.example.comElasticsearch provides REST APIs that are used by the UI components and can be called directly to configure and access Elasticsearch features.
This documentation is derived from the 8.18 branch of the elasticsearch-specification repository. It is provided under license Attribution-NonCommercial-NoDerivatives 4.0 International.
This documentation contains work-in-progress information for future Elastic Stack releases.
Last update on May 6, 2025.
This API is provided under license Apache 2.0.
The API accepts 3 different authentication methods:
Elasticsearch APIs support key-based authentication. You must create an API key and use the encoded value in the request header. For example:
curl -X GET "${ES_URL}/_cat/indices?v=true" \
-H "Authorization: ApiKey ${API_KEY}"
To get API keys, use the /_security/api_key APIs.
Basic auth tokens are constructed with the Basic keyword, followed by a space, followed by a base64-encoded string of your username:password
(separated by a : colon).
Example: send a Authorization: Basic aGVsbG86aGVsbG8=
HTTP header with your requests to authenticate with the API.
Elasticsearch APIs support the use of bearer tokens in the Authorization HTTP header to authenticate with the API.
For examples, refer to Token-based authentication services
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
the name of the autoscaling policy
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_autoscaling/policy/{name}' \
--header "Authorization: $API_KEY"{
"roles": <roles>,
"deciders": <deciders>
}NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
the name of the autoscaling policy
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Decider settings.
curl \
--request PUT 'http://api.example.com/_autoscaling/policy/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"roles\": [],\n \"deciders\": {\n \"fixed\": {\n }\n }\n}"'{
"roles": [],
"deciders": {
"fixed": {
}
}
}{
"roles" : [ "data_hot" ],
"deciders": {
"fixed": {
}
}
}{
"acknowledged": true
}NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
the name of the autoscaling policy
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/_autoscaling/policy/{name}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
This API gets the current autoscaling capacity based on the configured autoscaling policy. It will return information to size the cluster appropriately to the current workload.
The required_capacity is calculated as the maximum of the required_capacity result of all individual deciders that are enabled for the policy.
The operator should verify that the current_nodes match the operator’s knowledge of the cluster to avoid making autoscaling decisions based on stale or incomplete information.
The response contains decider-specific information you can use to diagnose how and why autoscaling determined a certain capacity was required. This information is provided for diagnosis only. Do not use this information to make autoscaling decisions.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_autoscaling/capacity' \
--header "Authorization: $API_KEY"{
policies: {}
}A list of analytics collections to limit the returned information
curl \
--request GET 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"{
"my_analytics_collection": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection"
}
},
"my_analytics_collection2": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection2"
}
}
}The name of the analytics collection to be created or updated.
curl \
--request PUT 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"The associated data stream is also deleted.
The name of the analytics collection to be deleted
curl \
--request DELETE 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"curl \
--request GET 'http://api.example.com/_application/analytics' \
--header "Authorization: $API_KEY"{
"my_analytics_collection": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection"
}
},
"my_analytics_collection2": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection2"
}
}
}The name of the behavioral analytics collection.
The analytics event type.
Values are page_view, search, or search_click.
Whether the response type has to include more details
curl \
--request POST 'http://api.example.com/_application/analytics/{collection_name}/event/{event_type}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"session\": {\n \"id\": \"1797ca95-91c9-4e2e-b1bd-9c38e6f386a9\"\n },\n \"user\": {\n \"id\": \"5f26f01a-bbee-4202-9298-81261067abbd\"\n },\n \"search\":{\n \"query\": \"search term\",\n \"results\": {\n \"items\": [\n {\n \"document\": {\n \"id\": \"123\",\n \"index\": \"products\"\n }\n }\n ],\n \"total_results\": 10\n },\n \"sort\": {\n \"name\": \"relevance\"\n },\n \"search_application\": \"website\"\n },\n \"document\":{\n \"id\": \"123\",\n \"index\": \"products\"\n }\n}"'{
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search":{
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document":{
"id": "123",
"index": "products"
}
}The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API.
All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.
Get the cluster's index aliases, including filter and routing information. This API does not return data stream aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
To indicated that the request should never timeout, you can set it to -1.
curl \
--request GET 'http://api.example.com/_cat/aliases' \
--header "Authorization: $API_KEY"[
{
"alias": "alias1",
"index": "test1",
"filter": "-",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias1",
"index": "test1",
"filter": "*",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias3",
"index": "test1",
"filter": "-",
"routing.index": "1",
"routing.search": "1",
"is_write_index": "true"
},
{
"alias": "alias4",
"index": "test1",
"filter": "-",
"routing.index": "2",
"routing.search": "1,2",
"is_write_index": "true"
}
]Get the cluster's index aliases, including filter and routing information. This API does not return data stream aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
A comma-separated list of aliases to retrieve. Supports wildcards (*). To retrieve all aliases, omit this parameter or use * or _all.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
To indicated that the request should never timeout, you can set it to -1.
curl \
--request GET 'http://api.example.com/_cat/aliases/{name}' \
--header "Authorization: $API_KEY"[
{
"alias": "alias1",
"index": "test1",
"filter": "-",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias1",
"index": "test1",
"filter": "*",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias3",
"index": "test1",
"filter": "-",
"routing.index": "1",
"routing.search": "1",
"is_write_index": "true"
},
{
"alias": "alias4",
"index": "test1",
"filter": "-",
"routing.index": "2",
"routing.search": "1,2",
"is_write_index": "true"
}
]Get a snapshot of the number of shards allocated to each data node and their disk space.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/allocation' \
--header "Authorization: $API_KEY"[
{
"shards": "1",
"shards.undesired": "0",
"write_load.forecast": "0.0",
"disk.indices.forecast": "260b",
"disk.indices": "260b",
"disk.used": "47.3gb",
"disk.avail": "43.4gb",
"disk.total": "100.7gb",
"disk.percent": "46",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "CSUXak2",
"node.role": "himrst"
}
]Get a snapshot of the number of shards allocated to each data node and their disk space.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
A comma-separated list of node identifiers or names used to limit the returned information.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/allocation/{node_id}' \
--header "Authorization: $API_KEY"[
{
"shards": "1",
"shards.undesired": "0",
"write_load.forecast": "0.0",
"disk.indices.forecast": "260b",
"disk.indices": "260b",
"disk.used": "47.3gb",
"disk.avail": "43.4gb",
"disk.total": "100.7gb",
"disk.percent": "46",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "CSUXak2",
"node.role": "himrst"
}
]Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/component_templates' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/component_templates/{name}' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]Get quick access to a document count for a data stream, an index, or an entire cluster. The document count only includes live documents, not deleted documents which have not yet been removed by the merge process.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the count API.
curl \
--request GET 'http://api.example.com/_cat/count' \
--header "Authorization: $API_KEY"[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "120"
}
][
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "121"
}
]Get quick access to a document count for a data stream, an index, or an entire cluster. The document count only includes live documents, not deleted documents which have not yet been removed by the merge process.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the count API.
A comma-separated list of data streams, indices, and aliases used to limit the request.
It supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
curl \
--request GET 'http://api.example.com/_cat/count/{index}' \
--header "Authorization: $API_KEY"[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "120"
}
][
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "121"
}
]Get the amount of heap memory currently used by the field data cache on every data node in the cluster.
IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes stats API.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of fields used to limit returned information.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/fielddata' \
--header "Authorization: $API_KEY"[
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "body",
"size": "544b"
}
][
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "1127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "body",
"size": "544b"
},
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "soul",
"size": "480b"
}
]Get the amount of heap memory currently used by the field data cache on every data node in the cluster.
IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes stats API.
Comma-separated list of fields used to limit returned information. To retrieve all fields, omit this parameter.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of fields used to limit returned information.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/fielddata/{fields}' \
--header "Authorization: $API_KEY"[
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "body",
"size": "544b"
}
][
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "1127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "body",
"size": "544b"
},
{
"id": "Nqk-6inXQq-OxUfOUI8jNQ",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "Nqk-6in",
"field": "soul",
"size": "480b"
}
]curl \
--request GET 'http://api.example.com/_cat' \
--header "Authorization: $API_KEY"Get high-level information about indices in a cluster, including backing indices for data streams.
Use this request to get the following information for each index in a cluster:
These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.
CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
The type of index that wildcard patterns can match.
The health status used to limit returned indices. By default, the response includes indices of any health status.
Values are green, GREEN, yellow, YELLOW, red, or RED.
If true, the response includes information from segments that are not loaded into memory.
If true, the response only includes information from primary shards.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
Period to wait for a connection to the master node.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/indices/{index}' \
--header "Authorization: $API_KEY"[
{
"health": "yellow",
"status": "open",
"index": "my-index-000001",
"uuid": "u8FNjxh8Rfy_awN11oDKYQ",
"pri": "1",
"rep": "1",
"docs.count": "1200",
"docs.deleted": "0",
"store.size": "88.1kb",
"pri.store.size": "88.1kb",
"dataset.size": "88.1kb"
},
{
"health": "green",
"status": "open",
"index": "my-index-000002",
"uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
"pri": "1",
"rep": "0",
"docs.count": "0",
"docs.deleted": "0",
"store.size": "260b",
"pri.store.size": "260b",
"dataset.size": "260b"
}
]Get information about the master node, including the ID, bound IP address, and name.
IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/master' \
--header "Authorization: $API_KEY"[
{
"id": "YzWoH_2BT-6UjVGDyPdqYg",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "YzWoH_2"
}
]Get configuration and usage information about data frame analytics jobs.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get data frame analytics jobs statistics API.
The ID of the data frame analytics to fetch
Whether to ignore if a wildcard expression matches no configs. (This includes _all string or when no configs have been specified)
The unit in which to display byte values
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Comma-separated list of column names or column aliases used to sort the response.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY"[
{
"id": "classifier_job_1",
"type": "classification",
"create_time": "2020-02-12T11:49:09.594Z",
"state": "stopped"
},
{
"id": "classifier_job_2",
"type": "classification",
"create_time": "2020-02-12T11:49:14.479Z",
"state": "stopped"
},
{
"id": "classifier_job_3",
"type": "classification",
"create_time": "2020-02-12T11:49:16.928Z",
"state": "stopped"
},
{
"id": "classifier_job_4",
"type": "classification",
"create_time": "2020-02-12T11:49:19.127Z",
"state": "stopped"
},
{
"id": "classifier_job_5",
"type": "classification",
"create_time": "2020-02-12T11:49:21.349Z",
"state": "stopped"
}
]Get configuration and usage information about datafeeds.
This API returns a maximum of 10,000 datafeeds.
If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage
cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
Comma-separated list of column names to display.
Comma-separated list of column names or column aliases used to sort the response.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/datafeeds' \
--header "Authorization: $API_KEY"[
{
"id": "datafeed-high_sum_total_sales",
"state": "stopped",
"buckets.count": "743",
"search.count": "7"
},
{
"id": "datafeed-low_request_rate",
"state": "stopped",
"buckets.count": "1457",
"search.count": "3"
},
{
"id": "datafeed-response_code_rates",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
},
{
"id": "datafeed-url_scanning",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
}
]Get configuration and usage information about datafeeds.
This API returns a maximum of 10,000 datafeeds.
If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage
cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.
A numerical character string that uniquely identifies the datafeed.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
Comma-separated list of column names to display.
Comma-separated list of column names or column aliases used to sort the response.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY"[
{
"id": "datafeed-high_sum_total_sales",
"state": "stopped",
"buckets.count": "743",
"search.count": "7"
},
{
"id": "datafeed-low_request_rate",
"state": "stopped",
"buckets.count": "1457",
"search.count": "3"
},
{
"id": "datafeed-response_code_rates",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
},
{
"id": "datafeed-url_scanning",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
}
]Get configuration and usage information for anomaly detection jobs.
This API returns a maximum of 10,000 jobs.
If the Elasticsearch security features are enabled, you must have monitor_ml,
monitor, manage_ml, or manage cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get anomaly detection job statistics API.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty jobs array when there are no matches and the subset of results when there
are partial matches. If false, the API returns a 404 status code when there are no matches or only partial
matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Comma-separated list of column names or column aliases used to sort the response.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/anomaly_detectors' \
--header "Authorization: $API_KEY"[
{
"id": "high_sum_total_sales",
"s": "closed",
"dpr": "14022",
"mb": "1.5mb"
},
{
"id": "low_request_rate",
"s": "closed",
"dpr": "1216",
"mb": "40.5kb"
},
{
"id": "response_code_rates",
"s": "closed",
"dpr": "28146",
"mb": "132.7kb"
},
{
"id": "url_scanning",
"s": "closed",
"dpr": "28146",
"mb": "501.6kb"
}
]Get configuration and usage information about inference trained models.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get trained models statistics API.
Specifies what to do when the request: contains wildcard expressions and there are no models that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, the API returns an empty array when there are no matches and the subset of results when there are partial matches.
If false, the API returns a 404 status code when there are no matches or only partial matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
A comma-separated list of column names to display.
A comma-separated list of column names or aliases used to sort the response.
Skips the specified number of transforms.
The maximum number of transforms to display.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/trained_models' \
--header "Authorization: $API_KEY"[
{
"id": "ddddd-1580216177138",
"heap_size": "0b",
"operations": "196",
"create_time": "2025-03-25T00:01:38.662Z",
"type": "pytorch",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
},
{
"id": "lang_ident_model_1",
"heap_size": "1mb",
"operations": "39629",
"create_time": "2019-12-05T12:28:34.594Z",
"type": "lang_ident",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
}
]Get information about custom node attributes. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/nodeattrs' \
--header "Authorization: $API_KEY"[
{
"node": "node-0",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"attr": "testattr",
"value": "test"
}
][
{
"name": "node-0",
"pid": "19566",
"attr": "testattr",
"value": "test"
}
]Get information about the nodes in a cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
If true, return the full node ID. If false, return the shortened node ID.
If true, the response includes information from segments that are not loaded into memory.
A comma-separated list of columns names to display. It supports simple wildcards.
A comma-separated list of column names or aliases that determines the sort order.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The period to wait for a connection to the master node.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/nodes' \
--header "Authorization: $API_KEY"[
{
"ip": "127.0.0.1",
"heap.percent": "65",
"ram.percent": "99",
"cpu": "42",
"load_1m": "3.07",
"load_5m": null,
"load_15m": null,
"node.role": "cdfhilmrstw",
"master": "*",
"name": "mJw06l1"
}
][
{
"id": "veJR",
"ip": "127.0.0.1",
"port": "59938",
"v": "9.0.0",
"m": "*"
}
]Get information about cluster-level changes that have not yet taken effect. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the pending cluster tasks API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/pending_tasks' \
--header "Authorization: $API_KEY"[
{ "insertOrder": "1685", "timeInQueue": "855ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1686", "timeInQueue": "843ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1693", "timeInQueue": "753ms", "priority": "HIGH", "source": "refresh-mapping [foo][[t]]"},
{ "insertOrder": "1688", "timeInQueue": "816ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1689", "timeInQueue": "802ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1690", "timeInQueue": "787ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1691", "timeInQueue": "773ms", "priority": "HIGH", "source": "update-mapping [foo][t]"}
]Get a list of plugins running on each node of a cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Include bootstrap plugins in the response
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/plugins' \
--header "Authorization: $API_KEY"[
{ "name": "U7321H6", "component": "analysis-icu", "version": "8.17.0", "description": "The ICU Analysis plugin integrates the Lucene ICU module into Elasticsearch, adding ICU-related analysis components."},
{"name": "U7321H6", "component": "analysis-kuromoji", "verison": "8.17.0", description: "The Japanese (kuromoji) Analysis plugin integrates Lucene kuromoji analysis module into elasticsearch."},
{"name" "U7321H6", "component": "analysis-nori", "version": "8.17.0", "description": "The Korean (nori) Analysis plugin integrates Lucene nori analysis module into elasticsearch."},
{"name": "U7321H6", "component": "analysis-phonetic", "verison": "8.17.0", "description": "The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch."},
{"name": "U7321H6", "component": "analysis-smartcn", "verison": "8.17.0", "description": "Smart Chinese Analysis plugin integrates Lucene Smart Chinese analysis module into elasticsearch."},
{"name": "U7321H6", "component": "analysis-stempel", "verison": "8.17.0", "description": "The Stempel (Polish) Analysis plugin integrates Lucene stempel (polish) analysis module into elasticsearch."},
{"name": "U7321H6", "component": "analysis-ukrainian", "verison": "8.17.0", "description": "The Ukrainian Analysis plugin integrates the Lucene UkrainianMorfologikAnalyzer into elasticsearch."},
{"name": "U7321H6", "component": "discovery-azure-classic", "verison": "8.17.0", "description": "The Azure Classic Discovery plugin allows to use Azure Classic API for the unicast discovery mechanism"},
{"name": "U7321H6", "component": "discovery-ec2", "verison": "8.17.0", "description": "The EC2 discovery plugin allows to use AWS API for the unicast discovery mechanism."},
{"name": "U7321H6", "component": "discovery-gce", "verison": "8.17.0", "description": "The Google Compute Engine (GCE) Discovery plugin allows to use GCE API for the unicast discovery mechanism."},
{"name": "U7321H6", "component": "mapper-annotated-text", "verison": "8.17.0", "description": "The Mapper Annotated_text plugin adds support for text fields with markup used to inject annotation tokens into the index."},
{"name": "U7321H6", "component": "mapper-murmur3", "verison": "8.17.0", "description": "The Mapper Murmur3 plugin allows to compute hashes of a field's values at index-time and to store them in the index."},
{"name": "U7321H6", "component": "mapper-size", "verison": "8.17.0", "description": "The Mapper Size plugin allows document to record their uncompressed size at index time."},
{"name": "U7321H6", "component": "store-smb", "verison": "8.17.0", "description": "The Store SMB plugin adds support for SMB stores."}
]Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.
If true, the response only includes ongoing shard recoveries.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
If true, the response includes detailed information about shard recoveries.
Comma-separated list or wildcard expression of index names to limit the returned information
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/recovery' \
--header "Authorization: $API_KEY"[
{
"index": "my-index-000001 ",
"shard": "0",
"time": "13ms",
"type": "store",
"stage": "done",
"source_host": "n/a",
"source_node": "n/a",
"target_host": "127.0.0.1",
"target_node": "node-0",
"repository": "n/a",
"snapshot": "n/a",
"files": "0",
"files_recovered": "0",
"files_percent": "100.0%",
"files_total": "13",
"bytes": "0b",
"bytes_recovered": "0b",
"bytes_percent": "100.0%",
"bytes_total": "9928b",
"translog_ops": "0",
"translog_ops_recovered": "0",
"translog_ops_percent": "100.0%"
}
][
{
"i": "my-index-000001",
"s": "0",
"t": "1252ms",
"ty": "peer",
"st": "done",
"shost": "192.168.1.1",
"thost": "192.168.1.1",
"f": "0",
"fp": "100.0%",
"b": "0b",
"bp": "100.0%",
}
][
{
"i": "my-index-000001",
"s": "0",
"t": "1978ms",
"ty": "snapshot",
"st": "done",
"rep": "my-repo",
"snap": "snap-1",
"f": "79",
"fp": "8.0%",
"b": "12086",
"bp": "9.0%"
}
]Get a list of snapshot repositories for a cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get snapshot repository API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/repositories' \
--header "Authorization: $API_KEY"[
{
"id": "repo1",
"type": "fs"
},
{
"id": "repo2",
"type": "s3"
}
]Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index segments API.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/segments' \
--header "Authorization: $API_KEY"[
{
"index": "test",
"shard": "0",
"prirep": "p",
"ip": "127.0.0.1",
"segment": "_0",
"generation": "0",
"docs.count": "1",
"docs.deleted": "0",
"size": "3kb",
"size.memory": "0",
"committed": "false",
"searchable": "true",
"version": "9.12.0",
"compound": "true"
},
{
"index": "test1",
"shard": "0",
"prirep": "p",
"ip": "127.0.0.1",
"segment": "_0",
"generation": "0",
"docs.count": "1",
"docs.deleted": "0",
"size": "3kb",
"size.memory": "0",
"committed": "false",
"searchable": "true",
"version": "9.12.0",
"compound": "true"
}
]Get information about the shards in a cluster. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Period to wait for a connection to the master node.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/shards' \
--header "Authorization: $API_KEY"[
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "RELOCATING",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA -> -> 192.168.56.30 bGG90GE"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "INITIALIZING",
"docs": "0",
"store": "14.3mb",
"dataset": "249b",
"ip": "192.168.56.30",
"node": "bGG90GE"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.10 H5dfFeA"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.30 bGG90GE"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.20 I8hydUG"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "UNASSIGNED",
"unassigned.reason": "ALLOCATION_FAILED"
}
]Get information about the shards in a cluster. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
A comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Period to wait for a connection to the master node.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/shards/{index}' \
--header "Authorization: $API_KEY"[
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "RELOCATING",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA -> -> 192.168.56.30 bGG90GE"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"docs": "3014",
"store": "31.1mb",
"dataset": "249b",
"ip": "192.168.56.10",
"node": "H5dfFeA"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "INITIALIZING",
"docs": "0",
"store": "14.3mb",
"dataset": "249b",
"ip": "192.168.56.30",
"node": "bGG90GE"
}
][
{
"index": "my-index-000001",
"shard": "0",
"prirep": "p",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.10 H5dfFeA"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.30 bGG90GE"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "STARTED",
"unassigned.reason": "3014 31.1mb 192.168.56.20 I8hydUG"
},
{
"index": "my-index-000001",
"shard": "0",
"prirep": "r",
"state": "UNASSIGNED",
"unassigned.reason": "ALLOCATION_FAILED"
}
]Get information about the snapshots stored in one or more repositories. A snapshot is a backup of an index or running Elasticsearch cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get snapshot API.
A comma-separated list of snapshot repositories used to limit the request.
Accepts wildcard expressions.
_all returns all repositories.
If any repository fails during the request, Elasticsearch returns an error.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Period to wait for a connection to the master node.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/snapshots/{repository}' \
--header "Authorization: $API_KEY"[
{
"id": "snap1",
"repository": "repo1",
"status": "FAILED",
"start_epoch": "1445616705",
"start_time": "18:11:45",
"end_epoch": "1445616978",
"end_time": "18:16:18",
"duration": "4.6m",
"indices": "1",
"successful_shards": "4",
"failed_shards": "1",
"total_shards": "5"
},
{
"id": "snap2",
"repository": "repo1",
"status": "SUCCESS",
"start_epoch": "1445634298",
"start_time": "23:04:58",
"end_epoch": "1445634672",
"end_time": "23:11:12",
"duration": "6.2m",
"indices": "2",
"successful_shards": "10",
"failed_shards": "0",
"total_shards": "10"
}
]Get information about tasks currently running in the cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the task management API.
The task action names, which are used to limit the response.
If true, the response includes detailed information about shard recoveries.
Unique node identifiers, which are used to limit the response.
The parent task identifier, which is used to limit the response.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
If true, the request blocks until the task has completed.
curl \
--request GET 'http://api.example.com/_cat/tasks' \
--header "Authorization: $API_KEY"[
{
"action": "cluster:monitor/tasks/lists[n]",
"task_id": "oTUltX4IQMOUUVeiohTt8A:124",
"parent_task_id": "oTUltX4IQMOUUVeiohTt8A:123",
"type": "direct",
"start_time": "1458585884904",
"timestamp": "01:48:24",
"running_time": "44.1micros",
"ip": "127.0.0.1:9300",
"node": "oTUltX4IQMOUUVeiohTt8A"
},
{
"action": "cluster:monitor/tasks/lists",
"task_id": "oTUltX4IQMOUUVeiohTt8A:123",
"parent_task_id": "-",
"type": "transport",
"start_time": "1458585884904",
"timestamp": "01:48:24",
"running_time": "186.2micros",
"ip": "127.0.0.1:9300",
"node": "oTUltX4IQMOUUVeiohTt8A"
}
]Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/templates' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-0",
"index_patterns": "[te*]",
"order": "500",
"version": null,
"composed_of": "[]"
},
{
"name": "my-template-1",
"index_patterns": "[tea*]",
"order": "501",
"version": null,
"composed_of": "[]"
},
{
"name": "my-template-2",
"index_patterns": "[teak*]",
"order": "502",
"version": "7",
"composed_of": "[]"
}
]Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.
The name of the template to return. Accepts wildcard expressions. If omitted, all templates are returned.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/templates/{name}' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-0",
"index_patterns": "[te*]",
"order": "500",
"version": null,
"composed_of": "[]"
},
{
"name": "my-template-1",
"index_patterns": "[tea*]",
"order": "501",
"version": null,
"composed_of": "[]"
},
{
"name": "my-template-2",
"index_patterns": "[teak*]",
"order": "502",
"version": "7",
"composed_of": "[]"
}
]Get thread pool statistics for each node in a cluster. Returned information includes all built-in thread pools and custom thread pools. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
A comma-separated list of thread pool names used to limit the request. Accepts wildcard expressions.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/thread_pool/{thread_pool_patterns}' \
--header "Authorization: $API_KEY"[
{
"node_name": "node-0",
"name": "analyze",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "fetch_shard_started",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "fetch_shard_store",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "flush",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "write",
"active": "0",
"queue": "0",
"rejected": "0"
}
][
{
"id": "0EWUhXeBQtaVGlexUeVwMg",
"name": "generic",
"active": "0",
"rejected": "0",
"completed": "70"
}
]Get explanations for shard allocations in the cluster. For unassigned shards, it provides an explanation for why the shard is unassigned. For assigned shards, it provides an explanation for why the shard is remaining on its current node and has not moved or rebalanced to another node. This API can be very useful when attempting to diagnose why a shard is unassigned or why a shard continues to remain on its current node when you might expect otherwise.
If true, returns information about disk usage and shard sizes.
If true, returns YES decisions in explanation.
Period to wait for a connection to the master node.
Specifies the node ID or the name of the node to only explain a shard that is currently located on the specified node.
If true, returns explanation for the primary shard for the given shard ID.
Specifies the ID of the shard that you would like an explanation for.
curl \
--request POST 'http://api.example.com/_cluster/allocation/explain' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\": \"my-index-000001\",\n \"shard\": 0,\n \"primary\": false,\n \"current_node\": \"my-node\"\n}"'{
"index": "my-index-000001",
"shard": 0,
"primary": false,
"current_node": "my-node"
}{
"index" : "my-index-000001",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "INDEX_CREATED",
"at" : "2017-01-04T18:08:16.600Z",
"last_allocation_status" : "no"
},
"can_allocate" : "no",
"allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
"node_allocation_decisions" : [
{
"node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
"node_name" : "node-0",
"transport_address" : "127.0.0.1:9401",
"roles" : ["data", "data_cold", "data_content", "data_frozen", "data_hot", "data_warm", "ingest", "master", "ml", "remote_cluster_client", "transform"],
"node_attributes" : {},
"node_decision" : "no",
"weight_ranking" : 1,
"deciders" : [
{
"decider" : "filter",
"decision" : "NO",
"explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]"
}
]
}
]
}{
"index" : "my-index-000001",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"at" : "2017-01-04T18:03:28.464Z",
"failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException",
"reason": "ALLOCATION_FAILED",
"failed_allocation_attempts": 5,
"last_allocation_status": "no",
},
"can_allocate": "no",
"allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
"node_allocation_decisions" : [
{
"node_id" : "3sULLVJrRneSg0EfBB-2Ew",
"node_name" : "node_t0",
"transport_address" : "127.0.0.1:9400",
"roles" : ["data_content", "data_hot"],
"node_decision" : "no",
"store" : {
"matching_size" : "4.2kb",
"matching_size_in_bytes" : 4325
},
"deciders" : [
{
"decider": "max_retry",
"decision" : "NO",
"explanation": "shard has exceeded the maximum number of retries [5] on failed allocation attempts - manually call [POST /_cluster/reroute?retry_failed] to retry, [unassigned_info[[reason=ALLOCATION_FAILED], at[2024-07-30T21:04:12.166Z], failed_attempts[5], failed_nodes[[mEKjwwzLT1yJVb8UxT6anw]], delayed=false, details[failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException], allocation_status[deciders_no]]]"
}
]
}
]
}Update the cluster voting config exclusions by node IDs or node names. By default, if there are more than three master-eligible nodes in the cluster and you remove fewer than half of the master-eligible nodes in the cluster at once, the voting configuration automatically shrinks. If you want to shrink the voting configuration to contain fewer than three nodes or to remove half or more of the master-eligible nodes in the cluster at once, use this API to remove departing nodes from the voting configuration manually. The API adds an entry for each specified node to the cluster’s voting configuration exclusions list. It then waits until the cluster has reconfigured its voting configuration to exclude the specified nodes.
Clusters should have no voting configuration exclusions in normal operation.
Once the excluded nodes have stopped, clear the voting configuration exclusions with DELETE /_cluster/voting_config_exclusions.
This API waits for the nodes to be fully removed from the cluster before it returns.
If your cluster has voting configuration exclusions for nodes that you no longer intend to remove, use DELETE /_cluster/voting_config_exclusions?wait_for_removal=false to clear the voting configuration exclusions without waiting for the nodes to leave the cluster.
A response to POST /_cluster/voting_config_exclusions with an HTTP status code of 200 OK guarantees that the node has been removed from the voting configuration and will not be reinstated until the voting configuration exclusions are cleared by calling DELETE /_cluster/voting_config_exclusions.
If the call to POST /_cluster/voting_config_exclusions fails or returns a response with an HTTP status code other than 200 OK then the node may not have been removed from the voting configuration.
In that case, you may safely retry the call.
NOTE: Voting exclusions are required only when you remove at least half of the master-eligible nodes from a cluster in a short time period. They are not required when removing master-ineligible nodes or when removing fewer than half of the master-eligible nodes.
A comma-separated list of the names of the nodes to exclude from the voting configuration. If specified, you may not also specify node_ids.
A comma-separated list of the persistent ids of the nodes to exclude from the voting configuration. If specified, you may not also specify node_names.
Period to wait for a connection to the master node.
When adding a voting configuration exclusion, the API waits for the specified nodes to be excluded from the voting configuration before returning. If the timeout expires before the appropriate condition is satisfied, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_cluster/voting_config_exclusions' \
--header "Authorization: $API_KEY"Remove master-eligible nodes from the voting configuration exclusion list.
Period to wait for a connection to the master node.
Specifies whether to wait for all excluded nodes to be removed from the cluster before clearing the voting configuration exclusions list. Defaults to true, meaning that all excluded nodes must be removed from the cluster before this API takes any action. If set to false then the voting configuration exclusions list is cleared even if some excluded nodes are still in the cluster.
curl \
--request DELETE 'http://api.example.com/_cluster/voting_config_exclusions' \
--header "Authorization: $API_KEY"By default, it returns only settings that have been explicitly defined.
If true, returns settings in flat format.
If true, returns default cluster settings from the local node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_cluster/settings' \
--header "Authorization: $API_KEY"Configure and update dynamic settings on a running cluster.
You can also configure dynamic settings locally on an unstarted or shut down node in elasticsearch.yml.
Updates made with this API can be persistent, which apply across cluster restarts, or transient, which reset after a cluster restart. You can also reset transient or persistent settings by assigning them a null value.
If you configure the same setting using multiple methods, Elasticsearch applies the settings in following order of precedence: 1) Transient setting; 2) Persistent setting; 3) elasticsearch.yml setting; 4) Default setting value.
For example, you can apply a transient setting to override a persistent setting or elasticsearch.yml setting.
However, a change to an elasticsearch.yml setting will not override a defined transient or persistent setting.
TIP: In Elastic Cloud, use the user settings feature to configure all cluster settings. This method automatically rejects unsafe settings that could break your cluster.
If you run Elasticsearch on your own hardware, use this API to configure dynamic cluster settings.
Only use elasticsearch.yml for static cluster settings and node settings.
The API doesn’t require a restart and ensures a setting’s value is the same on all nodes.
WARNING: Transient cluster settings are no longer recommended. Use persistent cluster settings instead. If a cluster becomes unstable, transient settings can clear unexpectedly, resulting in a potentially undesired cluster configuration.
Return settings in flat format (default: false)
Explicit operation timeout for connection to master node
Explicit operation timeout
curl \
--request PUT 'http://api.example.com/_cluster/settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"persistent\" : {\n \"indices.recovery.max_bytes_per_sec\" : \"50mb\"\n }\n}"'{
"persistent" : {
"indices.recovery.max_bytes_per_sec" : "50mb"
}
}{
"persistent": {
"action.auto_create_index": "my-index-000001,index10,-index1*,+ind*"
}
}Limits the information returned to the specific target. Supports a comma-separated list, such as http,ingest.
curl \
--request GET 'http://api.example.com/_info/{target}' \
--header "Authorization: $API_KEY"Get information about cluster-level changes (such as create index, update mapping, allocate or fail shard) that have not yet taken effect.
NOTE: This API returns a list of any pending updates to the cluster state. These are distinct from the tasks reported by the task management API which include periodic tasks and tasks initiated by the user, such as node stats, search queries, or create index requests. However, if a user-initiated task such as a create index command causes a cluster state update, the activity of this task might be reported by both task api and pending cluster tasks API.
If true, the request retrieves information from the local node only.
If false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_cluster/pending_tasks' \
--header "Authorization: $API_KEY"Get information about configured remote clusters. The API returns connection and endpoint information keyed by the configured remote cluster alias.
This API returns information that reflects current state on the local cluster.
The connected field does not necessarily reflect whether a remote cluster is down or unavailable, only whether there is currently an open connection to it.
Elasticsearch does not spontaneously try to reconnect to a disconnected remote cluster.
To trigger a reconnection, attempt a cross-cluster search, ES|QL cross-cluster search, or try the resolve cluster endpoint.
curl \
--request GET 'http://api.example.com/_remote/info' \
--header "Authorization: $API_KEY"Manually change the allocation of individual shards in the cluster. For example, a shard can be moved from one node to another explicitly, an allocation can be canceled, and an unassigned shard can be explicitly allocated to a specific node.
It is important to note that after processing any reroute commands Elasticsearch will perform rebalancing as normal (respecting the values of settings such as cluster.routing.rebalance.enable) in order to remain in a balanced state.
For example, if the requested allocation includes moving a shard from node1 to node2 then this may cause a shard to be moved from node2 back to node1 to even things out.
The cluster can be set to disable allocations using the cluster.routing.allocation.enable setting.
If allocations are disabled then the only allocations that will be performed are explicit ones given using the reroute command, and consequent allocations due to rebalancing.
The cluster will attempt to allocate a shard a maximum of index.allocation.max_retries times in a row (defaults to 5), before giving up and leaving the shard unallocated.
This scenario can be caused by structural problems such as having an analyzer which refers to a stopwords file which doesn’t exist on all nodes.
Once the problem has been corrected, allocation can be manually retried by calling the reroute API with the ?retry_failed URI query parameter, which will attempt a single retry round for these shards.
If true, then the request simulates the operation. It will calculate the result of applying the commands to the current cluster state and return the resulting cluster state after the commands (and rebalancing) have been applied; it will not actually perform the requested changes.
If true, then the response contains an explanation of why the commands can or cannot run.
Limits the information returned to the specified metrics.
If true, then retries allocation of shards that are blocked due to too many subsequent allocation failures.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_cluster/reroute' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"commands\": [\n {\n \"move\": {\n \"index\": \"test\", \"shard\": 0,\n \"from_node\": \"node1\", \"to_node\": \"node2\"\n }\n },\n {\n \"allocate_replica\": {\n \"index\": \"test\", \"shard\": 1,\n \"node\": \"node3\"\n }\n }\n ]\n}"'{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Return settings in flat format (default: false)
Return local information, do not retrieve the state from master node (default: false)
Specify timeout for connection to master
Wait for the metadata version to be equal or greater than the specified metadata version
The maximum time to wait for wait_for_metadata_version before timing out
curl \
--request GET 'http://api.example.com/_cluster/state' \
--header "Authorization: $API_KEY"Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Limit the information returned to the specified metrics
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Return settings in flat format (default: false)
Return local information, do not retrieve the state from master node (default: false)
Specify timeout for connection to master
Wait for the metadata version to be equal or greater than the specified metadata version
The maximum time to wait for wait_for_metadata_version before timing out
curl \
--request GET 'http://api.example.com/_cluster/state/{metric}' \
--header "Authorization: $API_KEY"Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Return settings in flat format (default: false)
Return local information, do not retrieve the state from master node (default: false)
Specify timeout for connection to master
Wait for the metadata version to be equal or greater than the specified metadata version
The maximum time to wait for wait_for_metadata_version before timing out
curl \
--request GET 'http://api.example.com/_cluster/state/{metric}/{index}' \
--header "Authorization: $API_KEY"curl \
--request HEAD 'http://api.example.com/' \
--header "Authorization: $API_KEY"Clear the archived repositories metering information in the cluster.
Comma-separated list of node IDs or names used to limit returned information.
Specifies the maximum archive_version to be cleared from the archive.
curl \
--request DELETE 'http://api.example.com/_nodes/{node_id}/_repositories_metering/{max_archive_version}' \
--header "Authorization: $API_KEY"Get repositories metering information for a cluster. This API exposes monotonically non-decreasing counters and it is expected that clients would durably store the information needed to compute aggregations over a period of time. Additionally, the information exposed by this API is volatile, meaning that it will not be present after node restarts.
curl \
--request GET 'http://api.example.com/_nodes/{node_id}/_repositories_metering' \
--header "Authorization: $API_KEY"Get a breakdown of the hot threads on each selected node in the cluster. The output is plain text with a breakdown of the top hot threads for each node.
If true, known idle threads (e.g. waiting in a socket select, or to get a task from an empty queue) are filtered out.
The interval to do the second sampling of threads.
Number of samples of thread stacktrace.
Specifies the number of hot threads to provide information for.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The type to sample.
Values are cpu, wait, block, gpu, or mem.
The sort order for 'cpu' type (default: total)
Values are cpu, wait, block, gpu, or mem.
curl \
--request GET 'http://api.example.com/_nodes/hot_threads' \
--header "Authorization: $API_KEY"Get a breakdown of the hot threads on each selected node in the cluster. The output is plain text with a breakdown of the top hot threads for each node.
List of node IDs or names used to limit returned information.
If true, known idle threads (e.g. waiting in a socket select, or to get a task from an empty queue) are filtered out.
The interval to do the second sampling of threads.
Number of samples of thread stacktrace.
Specifies the number of hot threads to provide information for.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The type to sample.
Values are cpu, wait, block, gpu, or mem.
The sort order for 'cpu' type (default: total)
Values are cpu, wait, block, gpu, or mem.
curl \
--request GET 'http://api.example.com/_nodes/{node_id}/hot_threads' \
--header "Authorization: $API_KEY"By default, the API returns all attributes and core settings for cluster nodes.
Limits the information returned to the specific metrics. Supports a comma-separated list, such as http,ingest.
If true, returns settings in flat format.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/{metric}' \
--header "Authorization: $API_KEY"{
"_nodes": {},
"cluster_name": "elasticsearch",
"nodes": {
"USpTGYaBSIKbgSUJR2Z9lg": {
"name": "node-0",
"transport_address": "192.168.17:9300",
"host": "node-0.elastic.co",
"ip": "192.168.17",
"version": "{version}",
"transport_version": 100000298,
"index_version": 100000074,
"component_versions": {
"ml_config_version": 100000162,
"transform_config_version": 100000096
},
"build_flavor": "default",
"build_type": "{build_type}",
"build_hash": "587409e",
"roles": [
"master",
"data",
"ingest"
],
"attributes": {},
"plugins": [
{
"name": "analysis-icu",
"version": "{version}",
"description": "The ICU Analysis plugin integrates Lucene ICU
module into elasticsearch, adding ICU relates analysis components.",
"classname":
"org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
"has_native_controller": false
}
],
"modules": [
{
"name": "lang-painless",
"version": "{version}",
"description": "An easy, safe and fast scripting language for
Elasticsearch",
"classname": "org.elasticsearch.painless.PainlessPlugin",
"has_native_controller": false
}
]
}
}
}By default, the API returns all attributes and core settings for cluster nodes.
If true, returns settings in flat format.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/{node_id}/{metric}' \
--header "Authorization: $API_KEY"{
"_nodes": {},
"cluster_name": "elasticsearch",
"nodes": {
"USpTGYaBSIKbgSUJR2Z9lg": {
"name": "node-0",
"transport_address": "192.168.17:9300",
"host": "node-0.elastic.co",
"ip": "192.168.17",
"version": "{version}",
"transport_version": 100000298,
"index_version": 100000074,
"component_versions": {
"ml_config_version": 100000162,
"transform_config_version": 100000096
},
"build_flavor": "default",
"build_type": "{build_type}",
"build_hash": "587409e",
"roles": [
"master",
"data",
"ingest"
],
"attributes": {},
"plugins": [
{
"name": "analysis-icu",
"version": "{version}",
"description": "The ICU Analysis plugin integrates Lucene ICU
module into elasticsearch, adding ICU relates analysis components.",
"classname":
"org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
"has_native_controller": false
}
],
"modules": [
{
"name": "lang-painless",
"version": "{version}",
"description": "An easy, safe and fast scripting language for
Elasticsearch",
"classname": "org.elasticsearch.painless.PainlessPlugin",
"has_native_controller": false
}
]
}
}
}Secure settings are stored in an on-disk keystore. Certain of these settings are reloadable. That is, you can change them on disk and reload them without restarting any nodes in the cluster. When you have updated reloadable secure settings in your keystore, you can use this API to reload those settings on each node.
When the Elasticsearch keystore is password protected and not simply obfuscated, you must provide the password for the keystore when you reload the secure settings. Reloading the settings for the whole cluster assumes that the keystores for all nodes are protected with the same password; this method is allowed only when inter-node communications are encrypted. Alternatively, you can reload the secure settings on each node by locally accessing the API and passing the node-specific Elasticsearch keystore password.
The names of particular nodes in the cluster to target.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_nodes/{node_id}/reload_secure_settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"secure_settings_password\": \"keystore-password\"\n}"'{
"secure_settings_password": "keystore-password"
}{
"_nodes": {
"total": 1,
"successful": 1,
"failed": 0
},
"cluster_name": "my_cluster",
"nodes": {
"pQHNt5rXTTWNvUgOrdynKg": {
"name": "node-0"
}
}
}Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/usage' \
--header "Authorization: $API_KEY"Limits the information returned to the specific metrics.
A comma-separated list of the following options: _all, rest_actions.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/usage/{metric}' \
--header "Authorization: $API_KEY"A comma-separated list of node IDs or names to limit the returned information; use _local to return information from the node you're connecting to, leave empty to get information from all nodes
Limits the information returned to the specific metrics.
A comma-separated list of the following options: _all, rest_actions.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/{node_id}/usage/{metric}' \
--header "Authorization: $API_KEY"The connector and sync jobs APIs provide a convenient way to create and manage Elastic connectors and sync jobs in an internal index. Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure:
This API provides an alternative to relying solely on Kibana UI for connector and sync job management. The API comes with a set of validations and assertions to ensure that the state representation in the internal index remains valid.
Update the last_seen field in the connector and set it to the current timestamp.
The unique identifier of the connector to be checked in
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_check_in' \
--header "Authorization: $API_KEY"{
"result": "updated"
}The unique identifier of the connector to be created or updated. ID is auto-generated if not provided.
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"search-google-drive\",\n \"name\": \"My Connector\",\n \"service_type\": \"google_drive\"\n}"'{
"index_name": "search-google-drive",
"name": "My Connector",
"service_type": "google_drive"
}{
"index_name": "search-google-drive",
"name": "My Connector",
"description": "My Connector to sync data to Elastic index from Google Drive",
"service_type": "google_drive",
"language": "english"
}{
"result": "created",
"id": "my-connector"
}Removes a connector and associated sync jobs. This is a destructive action that is not recoverable. NOTE: This action doesn’t delete any API keys, ingest pipelines, or data indices associated with the connector. These need to be removed manually.
The unique identifier of the connector to be deleted
A flag indicating if associated sync jobs should be also removed. Defaults to false.
curl \
--request DELETE 'http://api.example.com/_connector/{connector_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}curl \
--request PUT 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"search-google-drive\",\n \"name\": \"My Connector\",\n \"service_type\": \"google_drive\"\n}"'{
"index_name": "search-google-drive",
"name": "My Connector",
"service_type": "google_drive"
}{
"index_name": "search-google-drive",
"name": "My Connector",
"description": "My Connector to sync data to Elastic index from Google Drive",
"service_type": "google_drive",
"language": "english"
}{
"result": "created",
"id": "my-connector"
}Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure. Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud. Self-managed connectors (Connector clients) are self-managed on your infrastructure.
curl \
--request POST 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"description":"string","index_name":"string","is_native":true,"language":"string","name":"string","service_type":"string"}'Cancel a connector sync job, which sets the status to cancelling and updates cancellation_requested_at to the current time.
The connector service is then responsible for setting the status of connector sync jobs to cancelled.
The unique identifier of the connector sync job
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_cancel' \
--header "Authorization: $API_KEY"Check in a connector sync job and set the last_seen field to the current time before updating it in the internal index.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier of the connector sync job to be checked in.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_check_in' \
--header "Authorization: $API_KEY"This action updates the job status to in_progress and sets the last_seen and started_at timestamps to the current time.
Additionally, it can set the sync_cursor property for the sync job.
This API is not intended for direct connector management by users. It supports the implementation of services that utilize the connector protocol to communicate with Elasticsearch.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier of the connector sync job.
The cursor object from the last incremental sync job.
This should reference the sync_cursor field in the connector state for which the job runs.
The host name of the current system that will run the job.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_claim' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"sync_cursor":{},"worker_hostname":"string"}'Remove a connector sync job and its associated data. This is a destructive action that is not recoverable.
The unique identifier of the connector sync job to be deleted
curl \
--request DELETE 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Set the error field for a connector sync job and set its status to error.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier for the connector sync job.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_error' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"error\": \"some-error\"\n}"'{
"error": "some-error"
}Get information about all stored connector sync jobs listed by their creation date in ascending order.
Starting offset (default: 0)
Specifies a max number of results to get
A sync job status to fetch connector sync jobs for
Values are canceling, canceled, completed, error, in_progress, pending, or suspended.
A connector id to fetch connector sync jobs for
A comma-separated list of job types to fetch the sync jobs for
curl \
--request GET 'http://api.example.com/_connector/_sync_job' \
--header "Authorization: $API_KEY"Create a connector sync job document in the internal index and initialize its counters and timestamps with default values.
Values are full, incremental, or access_control.
Values are on_demand or scheduled.
curl \
--request POST 'http://api.example.com/_connector/_sync_job' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"id\": \"connector-id\",\n \"job_type\": \"full\",\n \"trigger_method\": \"on_demand\"\n}"'{
"id": "connector-id",
"job_type": "full",
"trigger_method": "on_demand"
}Stats include: deleted_document_count, indexed_document_count, indexed_document_volume, and total_document_count.
You can also update last_seen.
This API is mainly used by the connector service for updating sync job information.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier of the connector sync job.
The number of documents the sync job deleted.
The number of documents the sync job indexed.
The total size of the data (in MiB) the sync job indexed.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
The total number of documents in the target index after the sync job finished.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_stats' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"deleted_document_count":42.0,"indexed_document_count":42.0,"indexed_document_volume":42.0,"last_seen":"string","metadata":{"additionalProperty1":{},"additionalProperty2":{}},"total_document_count":42.0}'Activates the valid draft filtering for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
--header "Authorization: $API_KEY"Update the api_key_id and api_key_secret_id fields of a connector.
You can specify the ID of the API key used for authorization and the ID of the connector secret where the API key is stored.
The connector secret ID is required only for Elastic managed (native) connectors.
Self-managed connectors (connector clients) do not use this field.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_api_key_id' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"api_key_id\": \"my-api-key-id\",\n \"api_key_secret_id\": \"my-connector-secret-id\"\n}"'{
"api_key_id": "my-api-key-id",
"api_key_secret_id": "my-connector-secret-id"
}{
"result": "updated"
}Update the configuration field in the connector document.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_configuration' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"values\": {\n \"tenant_id\": \"my-tenant-id\",\n \"tenant_name\": \"my-sharepoint-site\",\n \"client_id\": \"foo\",\n \"secret_value\": \"bar\",\n \"site_collections\": \"*\"\n }\n}"'{
"values": {
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
}
}{
"values": {
"secret_value": "foo-bar"
}
}{
"result": "updated"
}Set the error field for the connector. If the error provided in the request body is non-null, the connector’s status is updated to error. Otherwise, if the error is reset to null, the connector status is updated to connected.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_error' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"error\": \"Houston, we have a problem!\"\n}"'{
"error": "Houston, we have a problem!"
}{
"result": "updated"
}Update the draft filtering configuration of a connector and marks the draft validation state as edited. The filtering draft is activated once validated by the running Elastic connector service. The filtering property is used to configure sync rules (both basic and advanced) for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"rules\": [\n {\n \"field\": \"file_extension\",\n \"id\": \"exclude-txt-files\",\n \"order\": 0,\n \"policy\": \"exclude\",\n \"rule\": \"equals\",\n \"value\": \"txt\"\n },\n {\n \"field\": \"_\",\n \"id\": \"DEFAULT\",\n \"order\": 1,\n \"policy\": \"include\",\n \"rule\": \"regex\",\n \"value\": \".*\"\n }\n ]\n}"'{
"rules": [
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
]
}{
"advanced_snippet": {
"value": [{
"tables": [
"users",
"orders"
],
"query": "SELECT users.id AS id, orders.order_id AS order_id FROM users JOIN orders ON users.id = orders.user_id"
}]
}
}{
"result": "updated"
}Update the draft filtering validation info for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_validation' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"validation":{"errors":[{"ids":["string"],"messages":["string"]}],"state":"edited"}}'Update the index_name field of a connector, specifying the index where the data ingested by the connector is stored.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_index_name' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"data-from-my-google-drive\"\n}"'{
"index_name": "data-from-my-google-drive"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_name' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"name\": \"Custom connector\",\n \"description\": \"This is my customized connector\"\n}"'{
"name": "Custom connector",
"description": "This is my customized connector"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_native' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"is_native":true}'When you create a new connector, the configuration of an ingest pipeline is populated with default settings.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_pipeline' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\": {\n \"extract_binary_content\": true,\n \"name\": \"my-connector-pipeline\",\n \"reduce_whitespace\": true,\n \"run_ml_inference\": true\n }\n}"'{
"pipeline": {
"extract_binary_content": true,
"name": "my-connector-pipeline",
"reduce_whitespace": true,
"run_ml_inference": true
}
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_scheduling' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scheduling\": {\n \"access_control\": {\n \"enabled\": true,\n \"interval\": \"0 10 0 * * ?\"\n },\n \"full\": {\n \"enabled\": true,\n \"interval\": \"0 20 0 * * ?\"\n },\n \"incremental\": {\n \"enabled\": false,\n \"interval\": \"0 30 0 * * ?\"\n }\n }\n}"'{
"scheduling": {
"access_control": {
"enabled": true,
"interval": "0 10 0 * * ?"
},
"full": {
"enabled": true,
"interval": "0 20 0 * * ?"
},
"incremental": {
"enabled": false,
"interval": "0 30 0 * * ?"
}
}
}{
"scheduling": {
"full": {
"enabled": true,
"interval": "0 10 0 * * ?"
}
}
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_service_type' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service_type\": \"sharepoint_online\"\n}"'{
"service_type": "sharepoint_online"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_status' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"status\": \"needs_configuration\"\n}"'{
"status": "needs_configuration"
}{
"result": "updated"
}Get cross-cluster replication auto-follow patterns.
The auto-follow pattern collection that you want to retrieve. If you do not specify a name, the API returns information for all collections.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request GET 'http://api.example.com/_ccr/auto_follow/{name}' \
--header "Authorization: $API_KEY"{
"patterns": [
{
"name": "my_auto_follow_pattern",
"pattern": {
"active": true,
"remote_cluster" : "remote_cluster",
"leader_index_patterns" :
[
"leader_index*"
],
"leader_index_exclusion_patterns":
[
"leader_index_001"
],
"follow_index_pattern" : "{{leader_index}}-follower"
}
}
]
}Create a collection of cross-cluster replication auto-follow patterns for a remote cluster. Newly created indices on the remote cluster that match any of the patterns are automatically configured as follower indices. Indices on the remote cluster that were created before the auto-follow pattern was created will not be auto-followed even if they match the pattern.
This API can also be used to update auto-follow patterns. NOTE: Follower indices that were configured automatically before updating an auto-follow pattern will remain unchanged even if they do not match against the new patterns.
The name of the collection of auto-follow patterns.
Period to wait for a connection to the master node.
The remote cluster containing the leader indices to match against.
The maximum number of outstanding reads requests from the remote cluster.
Settings to override from the leader index. Note that certain settings can not be overrode (e.g., index.number_of_shards).
The maximum number of outstanding reads requests from the remote cluster.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
The maximum number of operations to pull per read from the remote cluster.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
The maximum number of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the number of queued operations goes below the limit.
The maximum number of operations per bulk write request executed on the follower.
curl \
--request PUT 'http://api.example.com/_ccr/auto_follow/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"remote_cluster\" : \"remote_cluster\",\n \"leader_index_patterns\" :\n [\n \"leader_index*\"\n ],\n \"follow_index_pattern\" : \"{{leader_index}}-follower\",\n \"settings\": {\n \"index.number_of_replicas\": 0\n },\n \"max_read_request_operation_count\" : 1024,\n \"max_outstanding_read_requests\" : 16,\n \"max_read_request_size\" : \"1024k\",\n \"max_write_request_operation_count\" : 32768,\n \"max_write_request_size\" : \"16k\",\n \"max_outstanding_write_requests\" : 8,\n \"max_write_buffer_count\" : 512,\n \"max_write_buffer_size\" : \"512k\",\n \"max_retry_delay\" : \"10s\",\n \"read_poll_timeout\" : \"30s\"\n}"'{
"remote_cluster" : "remote_cluster",
"leader_index_patterns" :
[
"leader_index*"
],
"follow_index_pattern" : "{{leader_index}}-follower",
"settings": {
"index.number_of_replicas": 0
},
"max_read_request_operation_count" : 1024,
"max_outstanding_read_requests" : 16,
"max_read_request_size" : "1024k",
"max_write_request_operation_count" : 32768,
"max_write_request_size" : "16k",
"max_outstanding_write_requests" : 8,
"max_write_buffer_count" : 512,
"max_write_buffer_size" : "512k",
"max_retry_delay" : "10s",
"read_poll_timeout" : "30s"
}{
"acknowledged": true
}Delete a collection of cross-cluster replication auto-follow patterns.
The auto-follow pattern collection to delete.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request DELETE 'http://api.example.com/_ccr/auto_follow/{name}' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Remove the cross-cluster replication follower retention leases from the leader.
A following index takes out retention leases on its leader index. These leases are used to increase the likelihood that the shards of the leader index retain the history of operations that the shards of the following index need to run replication. When a follower index is converted to a regular index by the unfollow API (either by directly calling the API or by index lifecycle management tasks), these leases are removed. However, removal of the leases can fail, for example when the remote cluster containing the leader index is unavailable. While the leases will eventually expire on their own, their extended existence can cause the leader index to hold more history than necessary and prevent index lifecycle management from performing some operations on the leader index. This API exists to enable manually removing the leases when the unfollow API is unable to do so.
NOTE: This API does not stop replication by a following index. If you use this API with a follower index that is still actively following, the following index will add back retention leases on the leader. The only purpose of this API is to handle the case of failure to remove the following retention leases after the unfollow API is invoked.
the name of the leader index for which specified follower retention leases should be removed
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/{index}/_ccr/forget_follower' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"follower_cluster\" : \"\u003cfollower_cluster\u003e\",\n \"follower_index\" : \"\u003cfollower_index\u003e\",\n \"follower_index_uuid\" : \"\u003cfollower_index_uuid\u003e\",\n \"leader_remote_cluster\" : \"\u003cleader_remote_cluster\u003e\"\n}"'{
"follower_cluster" : "<follower_cluster>",
"follower_index" : "<follower_index>",
"follower_index_uuid" : "<follower_index_uuid>",
"leader_remote_cluster" : "<leader_remote_cluster>"
}{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0,
"failures" : [ ]
}
}Get cross-cluster replication auto-follow patterns.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request GET 'http://api.example.com/_ccr/auto_follow' \
--header "Authorization: $API_KEY"{
"patterns": [
{
"name": "my_auto_follow_pattern",
"pattern": {
"active": true,
"remote_cluster" : "remote_cluster",
"leader_index_patterns" :
[
"leader_index*"
],
"leader_index_exclusion_patterns":
[
"leader_index_001"
],
"follow_index_pattern" : "{{leader_index}}-follower"
}
}
]
}Pause a cross-cluster replication auto-follow pattern. When the API returns, the auto-follow pattern is inactive. New indices that are created on the remote cluster and match the auto-follow patterns are ignored.
You can resume auto-following with the resume auto-follow pattern API. When it resumes, the auto-follow pattern is active again and automatically configures follower indices for newly created indices on the remote cluster that match its patterns. Remote indices that were created while the pattern was paused will also be followed, unless they have been deleted or closed in the interim.
The name of the auto-follow pattern to pause.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_ccr/auto_follow/{name}/pause' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Pause a cross-cluster replication follower index. The follower index will not fetch any additional operations from the leader index. You can resume following with the resume follower API. You can pause and resume a follower index to change the configuration of the following task.
The name of the follower index.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/{index}/_ccr/pause_follow' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Resume a cross-cluster replication auto-follow pattern that was paused. The auto-follow pattern will resume configuring following indices for newly created indices that match its patterns on the remote cluster. Remote indices created while the pattern was paused will also be followed unless they have been deleted or closed in the interim.
The name of the auto-follow pattern to resume.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_ccr/auto_follow/{name}/resume' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Resume a cross-cluster replication follower index that was paused. The follower index could have been paused with the pause follower API. Alternatively it could be paused due to replication that cannot be retried due to failures during following tasks. When this API returns, the follower index will resume fetching operations from the leader index.
The name of the follow index to resume following.
Period to wait for a connection to the master node.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
curl \
--request POST 'http://api.example.com/{index}/_ccr/resume_follow' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"max_read_request_operation_count\" : 1024,\n \"max_outstanding_read_requests\" : 16,\n \"max_read_request_size\" : \"1024k\",\n \"max_write_request_operation_count\" : 32768,\n \"max_write_request_size\" : \"16k\",\n \"max_outstanding_write_requests\" : 8,\n \"max_write_buffer_count\" : 512,\n \"max_write_buffer_size\" : \"512k\",\n \"max_retry_delay\" : \"10s\",\n \"read_poll_timeout\" : \"30s\"\n}"'{
"max_read_request_operation_count" : 1024,
"max_outstanding_read_requests" : 16,
"max_read_request_size" : "1024k",
"max_write_request_operation_count" : 32768,
"max_write_request_size" : "16k",
"max_outstanding_write_requests" : 8,
"max_write_buffer_count" : 512,
"max_write_buffer_size" : "512k",
"max_retry_delay" : "10s",
"read_poll_timeout" : "30s"
}{
"acknowledged" : true
}Convert a cross-cluster replication follower index to a regular index. The API stops the following task associated with a follower index and removes index metadata and settings associated with cross-cluster replication. The follower index must be paused and closed before you call the unfollow API.
Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.
The name of the follower index.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/{index}/_ccr/unfollow' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Get information about one or more data streams.
Comma-separated list of data stream names used to limit the request.
Wildcard (*) expressions are supported. If omitted, all data streams are returned.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
If true, returns all relevant default configurations for the index template.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Whether the maximum timestamp for each data stream should be calculated and returned.
curl \
--request GET 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"{
"data_streams": [
{
"name": "my-data-stream",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-2099.03.07-000001",
"index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
},
{
"index_name": ".ds-my-data-stream-2099.03.08-000002",
"index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 2,
"_meta": {
"my-meta-field": "foo"
},
"status": "GREEN",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
},
{
"name": "my-data-stream-two",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-two-2099.03.08-000001",
"index_uuid": "3liBu2SYS5axasRt6fUIpA",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 1,
"_meta": {
"my-meta-field": "foo"
},
"status": "YELLOW",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
}
]
}You must have a matching index template with data stream enabled.
Name of the data stream, which must meet the following criteria:
Lowercase only;
Cannot include \, /, *, ?, ", <, >, |, ,, #, :, or a space character;
Cannot start with -, _, +, or .ds-;
Cannot be . or ..;
Cannot be longer than 255 bytes. Multi-byte characters count towards this limit faster.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request PUT 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"Deletes one or more data streams and their backing indices.
Comma-separated list of data streams to delete. Wildcard (*) expressions are supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Type of data stream that wildcard patterns can match. Supports comma-separated values,such as open,hidden.
curl \
--request DELETE 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"Get statistics for one or more data streams.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
curl \
--request GET 'http://api.example.com/_data_stream/_stats' \
--header "Authorization: $API_KEY"{
"_shards": {
"total": 10,
"successful": 5,
"failed": 0
},
"data_stream_count": 2,
"backing_indices": 5,
"total_store_size": "7kb",
"total_store_size_bytes": 7268,
"data_streams": [
{
"data_stream": "my-data-stream",
"backing_indices": 3,
"store_size": "3.7kb",
"store_size_bytes": 3772,
"maximum_timestamp": 1607512028000
},
{
"data_stream": "my-data-stream-two",
"backing_indices": 2,
"store_size": "3.4kb",
"store_size_bytes": 3496,
"maximum_timestamp": 1607425567000
}
]
}Get statistics for one or more data streams.
Comma-separated list of data streams used to limit the request.
Wildcard expressions (*) are supported.
To target all data streams in a cluster, omit this parameter or use *.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
curl \
--request GET 'http://api.example.com/_data_stream/{name}/_stats' \
--header "Authorization: $API_KEY"{
"_shards": {
"total": 10,
"successful": 5,
"failed": 0
},
"data_stream_count": 2,
"backing_indices": 5,
"total_store_size": "7kb",
"total_store_size_bytes": 7268,
"data_streams": [
{
"data_stream": "my-data-stream",
"backing_indices": 3,
"store_size": "3.7kb",
"store_size_bytes": 3772,
"maximum_timestamp": 1607512028000
},
{
"data_stream": "my-data-stream-two",
"backing_indices": 2,
"store_size": "3.4kb",
"store_size_bytes": 3496,
"maximum_timestamp": 1607425567000
}
]
}Update the data stream lifecycle of the specified data streams.
Comma-separated list of data streams used to limit the request.
Supports wildcards (*).
To target all data streams use * or _all.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Valid values are: all, hidden, open, closed, none.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle
that's disabled (enabled: false) will have no effect on the data stream.
curl \
--request PUT 'http://api.example.com/_data_stream/{name}/_lifecycle' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"data_retention\": \"7d\"\n}"'{
"data_retention": "7d"
}{
"downsampling": [
{
"after": "1d",
"fixed_interval": "10m"
},
{
"after": "7d",
"fixed_interval": "1d"
}
]
}{
"acknowledged": true
}Aggregate a time series (TSDS) index and store pre-computed statistical summaries (min, max, sum, value_count and avg) for each metric field grouped by a configured time interval.
For example, a TSDS index that contains metrics sampled every 10 seconds can be downsampled to an hourly index.
All documents within an hour interval are summarized and stored as a single document in the downsample index.
NOTE: Only indices in a time series data stream are supported.
Neither field nor document level security can be defined on the source index.
The source index must be read only (index.blocks.write: true).
Name of the time series index to downsample.
Name of the index to create.
A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and
y (year)
curl \
--request POST 'http://api.example.com/{index}/_downsample/{target_index}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fixed_interval\": \"1d\"\n}"'{
"fixed_interval": "1d"
}Get information about an index or data stream's current data stream lifecycle status, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any errors encountered during lifecycle execution.
The name of the index to explain
indicates if the API should return the default values the system uses for the index's lifecycle
Specify timeout for connection to master
curl \
--request GET 'http://api.example.com/{index}/_lifecycle/explain' \
--header "Authorization: $API_KEY"{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"rollover_date_millis" : 1679475564293,
"time_since_rollover" : "121ms",
"lifecycle" : { },
"generation_time" : "121ms"
}
}{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"lifecycle" : {
"enabled": true
},
"error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"
}
}Get statistics about the data streams that are managed by a data stream lifecycle.
curl \
--request GET 'http://api.example.com/_lifecycle/stats' \
--header "Authorization: $API_KEY"{
"last_run_duration_in_millis": 2,
"last_run_duration": "2ms",
"time_between_starts_in_millis": 9998,
"time_between_starts": "9.99s",
"data_streams_count": 2,
"data_streams": [
{
"name": "my-data-stream",
"backing_indices_in_total": 2,
"backing_indices_in_error": 0
},
{
"name": "my-other-stream",
"backing_indices_in_total": 2,
"backing_indices_in_error": 1
}
]
}Converts an index alias to a data stream.
You must have a matching index template that is data stream enabled.
The alias must meet the following criteria:
The alias must have a write index;
All indices for the alias must have a @timestamp field mapping of a date or date_nanos field type;
The alias must not have any filters;
The alias must not use custom routing.
If successful, the request removes the alias and creates a data stream with the same name.
The indices for the alias become hidden backing indices for the stream.
The write index for the alias becomes the write index for the stream.
Name of the index alias to convert to a data stream.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_data_stream/_migrate/{name}' \
--header "Authorization: $API_KEY"Performs one or more data stream modification actions in a single atomic operation.
curl \
--request POST 'http://api.example.com/_data_stream/_modify' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"actions":[{"add_backing_index":{"data_stream":"string","index":"string"},"remove_backing_index":{"data_stream":"string","index":"string"}}]}'Promote a data stream from a replicated data stream managed by cross-cluster replication (CCR) to a regular data stream.
With CCR auto following, a data stream from a remote cluster can be replicated to the local cluster. These data streams can't be rolled over in the local cluster. These replicated data streams roll over only if the upstream data stream rolls over. In the event that the remote cluster is no longer available, the data stream in the local cluster can be promoted to a regular data stream, which allows these data streams to be rolled over in the local cluster.
NOTE: When promoting a data stream, ensure the local cluster has a data stream enabled index template that matches the data stream. If this is missing, the data stream will not be able to roll over until a matching index template is created. This will affect the lifecycle management of the data stream and interfere with the data stream size and retention.
The name of the data stream
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_data_stream/_promote/{name}' \
--header "Authorization: $API_KEY"Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
The name of the data stream, index, or index alias to perform bulk actions on.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request PUT 'http://api.example.com/{index}/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
The name of the data stream, index, or index alias to perform bulk actions on.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request POST 'http://api.example.com/{index}/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs
Using _create guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/ API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
If true, the request's actions must target a data stream (existing or to be created).
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
The explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
curl \
--request PUT 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs
Using _create guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/ API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
If true, the request's actions must target a data stream (existing or to be created).
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
The explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
curl \
--request POST 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}