http://api.example.comThis documentation is derived from the main branch of the elasticsearch-specification repository. It is provided under license Attribution-NonCommercial-NoDerivatives 4.0 International.
Last update on May 6, 2025.
This API is provided under license Apache 2.0.
Elasticsearch APIs use key-based authentication. You must create an API key and use the encoded value in the request header. For example:
curl -X GET "${ES_URL}/_cat/indices?v=true" \
-H "Authorization: ApiKey ${API_KEY}"
For more information about where to find API keys for the Elasticsearch endpoint (${ES_URL}) for a project, go to Get started with Elasticsearch Serverless.
A list of analytics collections to limit the returned information
curl \
--request GET 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"{
"my_analytics_collection": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection"
}
},
"my_analytics_collection2": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection2"
}
}
}The name of the analytics collection to be created or updated.
curl \
--request PUT 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"The associated data stream is also deleted.
The name of the analytics collection to be deleted
curl \
--request DELETE 'http://api.example.com/_application/analytics/{name}' \
--header "Authorization: $API_KEY"curl \
--request GET 'http://api.example.com/_application/analytics' \
--header "Authorization: $API_KEY"{
"my_analytics_collection": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection"
}
},
"my_analytics_collection2": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection2"
}
}
}The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API.
All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.
Get the cluster's index aliases, including filter and routing information. This API does not return data stream aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
To indicated that the request should never timeout, you can set it to -1.
curl \
--request GET 'http://api.example.com/_cat/aliases' \
--header "Authorization: $API_KEY"[
{
"alias": "alias1",
"index": "test1",
"filter": "-",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias1",
"index": "test1",
"filter": "*",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias3",
"index": "test1",
"filter": "-",
"routing.index": "1",
"routing.search": "1",
"is_write_index": "true"
},
{
"alias": "alias4",
"index": "test1",
"filter": "-",
"routing.index": "2",
"routing.search": "1,2",
"is_write_index": "true"
}
]Get the cluster's index aliases, including filter and routing information. This API does not return data stream aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.
A comma-separated list of aliases to retrieve. Supports wildcards (*). To retrieve all aliases, omit this parameter or use * or _all.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
To indicated that the request should never timeout, you can set it to -1.
curl \
--request GET 'http://api.example.com/_cat/aliases/{name}' \
--header "Authorization: $API_KEY"[
{
"alias": "alias1",
"index": "test1",
"filter": "-",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias1",
"index": "test1",
"filter": "*",
"routing.index": "-",
"routing.search": "-",
"is_write_index": "true"
},
{
"alias": "alias3",
"index": "test1",
"filter": "-",
"routing.index": "1",
"routing.search": "1",
"is_write_index": "true"
},
{
"alias": "alias4",
"index": "test1",
"filter": "-",
"routing.index": "2",
"routing.search": "1,2",
"is_write_index": "true"
}
]Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/component_templates' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/component_templates/{name}' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]Get quick access to a document count for a data stream, an index, or an entire cluster. The document count only includes live documents, not deleted documents which have not yet been removed by the merge process.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the count API.
curl \
--request GET 'http://api.example.com/_cat/count' \
--header "Authorization: $API_KEY"[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "120"
}
][
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "121"
}
]Get quick access to a document count for a data stream, an index, or an entire cluster. The document count only includes live documents, not deleted documents which have not yet been removed by the merge process.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the count API.
A comma-separated list of data streams, indices, and aliases used to limit the request.
It supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
curl \
--request GET 'http://api.example.com/_cat/count/{index}' \
--header "Authorization: $API_KEY"[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "120"
}
][
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "121"
}
]curl \
--request GET 'http://api.example.com/_cat' \
--header "Authorization: $API_KEY"Get high-level information about indices in a cluster, including backing indices for data streams.
Use this request to get the following information for each index in a cluster:
These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.
CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
The type of index that wildcard patterns can match.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.The health status used to limit returned indices. By default, the response includes indices of any health status.
Supported values include:
green (or GREEN): All shards are assigned.yellow (or YELLOW): All primary shards are assigned, but one or more replica shards are unassigned. If a node in the cluster fails, some data could be unavailable until that node is repaired.red (or RED): One or more primary shards are unassigned, so some data is unavailable. This can occur briefly during cluster startup as primary shards are assigned.Values are green, GREEN, yellow, YELLOW, red, or RED.
If true, the response includes information from segments that are not loaded into memory.
If true, the response only includes information from primary shards.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
Period to wait for a connection to the master node.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/indices' \
--header "Authorization: $API_KEY"[
{
"health": "yellow",
"status": "open",
"index": "my-index-000001",
"uuid": "u8FNjxh8Rfy_awN11oDKYQ",
"pri": "1",
"rep": "1",
"docs.count": "1200",
"docs.deleted": "0",
"store.size": "88.1kb",
"pri.store.size": "88.1kb",
"dataset.size": "88.1kb"
},
{
"health": "green",
"status": "open",
"index": "my-index-000002",
"uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
"pri": "1",
"rep": "0",
"docs.count": "0",
"docs.deleted": "0",
"store.size": "260b",
"pri.store.size": "260b",
"dataset.size": "260b"
}
]Get high-level information about indices in a cluster, including backing indices for data streams.
Use this request to get the following information for each index in a cluster:
These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.
CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
The type of index that wildcard patterns can match.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.The health status used to limit returned indices. By default, the response includes indices of any health status.
Supported values include:
green (or GREEN): All shards are assigned.yellow (or YELLOW): All primary shards are assigned, but one or more replica shards are unassigned. If a node in the cluster fails, some data could be unavailable until that node is repaired.red (or RED): One or more primary shards are unassigned, so some data is unavailable. This can occur briefly during cluster startup as primary shards are assigned.Values are green, GREEN, yellow, YELLOW, red, or RED.
If true, the response includes information from segments that are not loaded into memory.
If true, the response only includes information from primary shards.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
Period to wait for a connection to the master node.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/indices/{index}' \
--header "Authorization: $API_KEY"[
{
"health": "yellow",
"status": "open",
"index": "my-index-000001",
"uuid": "u8FNjxh8Rfy_awN11oDKYQ",
"pri": "1",
"rep": "1",
"docs.count": "1200",
"docs.deleted": "0",
"store.size": "88.1kb",
"pri.store.size": "88.1kb",
"dataset.size": "88.1kb"
},
{
"health": "green",
"status": "open",
"index": "my-index-000002",
"uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
"pri": "1",
"rep": "0",
"docs.count": "0",
"docs.deleted": "0",
"store.size": "260b",
"pri.store.size": "260b",
"dataset.size": "260b"
}
]Get configuration and usage information about data frame analytics jobs.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get data frame analytics jobs statistics API.
Whether to ignore if a wildcard expression matches no configs. (This includes _all string or when no configs have been specified)
The unit in which to display byte values
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Supported values include:
assignment_explanation (or ae): Contains messages relating to the selection of a node.create_time (or ct, createTime): The time when the data frame analytics job was created.description (or d): A description of a job.dest_index (or di, destIndex): Name of the destination index.failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.id: Identifier for the data frame analytics job.model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for
the data frame analytics job.node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is
assigned to.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned
to.node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is
assigned to.node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.progress (or p): The progress report of the data frame analytics job by phase.source_index (or si, sourceIndex): Name of the source index.state (or s): Current state of the data frame analytics job.type (or t): The type of analysis that the data frame analytics job performs.version (or v): The Elasticsearch version number in which the data frame analytics job was
created.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
assignment_explanation (or ae): Contains messages relating to the selection of a node.create_time (or ct, createTime): The time when the data frame analytics job was created.description (or d): A description of a job.dest_index (or di, destIndex): Name of the destination index.failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.id: Identifier for the data frame analytics job.model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for
the data frame analytics job.node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is
assigned to.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned
to.node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is
assigned to.node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.progress (or p): The progress report of the data frame analytics job by phase.source_index (or si, sourceIndex): Name of the source index.state (or s): Current state of the data frame analytics job.type (or t): The type of analysis that the data frame analytics job performs.version (or v): The Elasticsearch version number in which the data frame analytics job was
created.Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/data_frame/analytics' \
--header "Authorization: $API_KEY"[
{
"id": "classifier_job_1",
"type": "classification",
"create_time": "2020-02-12T11:49:09.594Z",
"state": "stopped"
},
{
"id": "classifier_job_2",
"type": "classification",
"create_time": "2020-02-12T11:49:14.479Z",
"state": "stopped"
},
{
"id": "classifier_job_3",
"type": "classification",
"create_time": "2020-02-12T11:49:16.928Z",
"state": "stopped"
},
{
"id": "classifier_job_4",
"type": "classification",
"create_time": "2020-02-12T11:49:19.127Z",
"state": "stopped"
},
{
"id": "classifier_job_5",
"type": "classification",
"create_time": "2020-02-12T11:49:21.349Z",
"state": "stopped"
}
]Get configuration and usage information about data frame analytics jobs.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get data frame analytics jobs statistics API.
The ID of the data frame analytics to fetch
Whether to ignore if a wildcard expression matches no configs. (This includes _all string or when no configs have been specified)
The unit in which to display byte values
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Supported values include:
assignment_explanation (or ae): Contains messages relating to the selection of a node.create_time (or ct, createTime): The time when the data frame analytics job was created.description (or d): A description of a job.dest_index (or di, destIndex): Name of the destination index.failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.id: Identifier for the data frame analytics job.model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for
the data frame analytics job.node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is
assigned to.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned
to.node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is
assigned to.node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.progress (or p): The progress report of the data frame analytics job by phase.source_index (or si, sourceIndex): Name of the source index.state (or s): Current state of the data frame analytics job.type (or t): The type of analysis that the data frame analytics job performs.version (or v): The Elasticsearch version number in which the data frame analytics job was
created.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
assignment_explanation (or ae): Contains messages relating to the selection of a node.create_time (or ct, createTime): The time when the data frame analytics job was created.description (or d): A description of a job.dest_index (or di, destIndex): Name of the destination index.failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.id: Identifier for the data frame analytics job.model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for
the data frame analytics job.node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is
assigned to.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned
to.node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is
assigned to.node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.progress (or p): The progress report of the data frame analytics job by phase.source_index (or si, sourceIndex): Name of the source index.state (or s): Current state of the data frame analytics job.type (or t): The type of analysis that the data frame analytics job performs.version (or v): The Elasticsearch version number in which the data frame analytics job was
created.Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY"[
{
"id": "classifier_job_1",
"type": "classification",
"create_time": "2020-02-12T11:49:09.594Z",
"state": "stopped"
},
{
"id": "classifier_job_2",
"type": "classification",
"create_time": "2020-02-12T11:49:14.479Z",
"state": "stopped"
},
{
"id": "classifier_job_3",
"type": "classification",
"create_time": "2020-02-12T11:49:16.928Z",
"state": "stopped"
},
{
"id": "classifier_job_4",
"type": "classification",
"create_time": "2020-02-12T11:49:19.127Z",
"state": "stopped"
},
{
"id": "classifier_job_5",
"type": "classification",
"create_time": "2020-02-12T11:49:21.349Z",
"state": "stopped"
}
]Get configuration and usage information about datafeeds.
This API returns a maximum of 10,000 datafeeds.
If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage
cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
Comma-separated list of column names to display.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/datafeeds' \
--header "Authorization: $API_KEY"[
{
"id": "datafeed-high_sum_total_sales",
"state": "stopped",
"buckets.count": "743",
"search.count": "7"
},
{
"id": "datafeed-low_request_rate",
"state": "stopped",
"buckets.count": "1457",
"search.count": "3"
},
{
"id": "datafeed-response_code_rates",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
},
{
"id": "datafeed-url_scanning",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
}
]Get configuration and usage information about datafeeds.
This API returns a maximum of 10,000 datafeeds.
If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage
cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.
A numerical character string that uniquely identifies the datafeed.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
Comma-separated list of column names to display.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY"[
{
"id": "datafeed-high_sum_total_sales",
"state": "stopped",
"buckets.count": "743",
"search.count": "7"
},
{
"id": "datafeed-low_request_rate",
"state": "stopped",
"buckets.count": "1457",
"search.count": "3"
},
{
"id": "datafeed-response_code_rates",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
},
{
"id": "datafeed-url_scanning",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
}
]Get configuration and usage information for anomaly detection jobs.
This API returns a maximum of 10,000 jobs.
If the Elasticsearch security features are enabled, you must have monitor_ml,
monitor, manage_ml, or manage cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get anomaly detection job statistics API.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty jobs array when there are no matches and the subset of results when there
are partial matches. If false, the API returns a 404 status code when there are no matches or only partial
matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Supported values include:
assignment_explanation (or ae): For open anomaly detection jobs only, contains messages relating to the
selection of a node to run the job.buckets.count (or bc, bucketsCount): The number of bucket results produced by the job.buckets.time.exp_avg (or btea, bucketsTimeExpAvg): Exponential moving average of all bucket processing times, in milliseconds.buckets.time.exp_avg_hour (or bteah, bucketsTimeExpAvgHour): Exponentially-weighted moving average of bucket processing times calculated
in a 1 hour time window, in milliseconds.buckets.time.max (or btmax, bucketsTimeMax): Maximum among all bucket processing times, in milliseconds.buckets.time.min (or btmin, bucketsTimeMin): Minimum among all bucket processing times, in milliseconds.buckets.time.total (or btt, bucketsTimeTotal): Sum of all bucket processing times, in milliseconds.data.buckets (or db, dataBuckets): The number of buckets processed.data.earliest_record (or der, dataEarliestRecord): The timestamp of the earliest chronologically input document.data.empty_buckets (or deb, dataEmptyBuckets): The number of buckets which did not contain any data.data.input_bytes (or dib, dataInputBytes): The number of bytes of input data posted to the anomaly detection job.data.input_fields (or dif, dataInputFields): The total number of fields in input documents posted to the anomaly
detection job. This count includes fields that are not used in the analysis.
However, be aware that if you are using a datafeed, it extracts only the
required fields from the documents it retrieves before posting them to the job.data.input_records (or dir, dataInputRecords): The number of input documents posted to the anomaly detection job.data.invalid_dates (or did, dataInvalidDates): The number of input documents with either a missing date field or a date
that could not be parsed.data.last (or dl, dataLast): The timestamp at which data was last analyzed, according to server time.data.last_empty_bucket (or dleb, dataLastEmptyBucket): The timestamp of the last bucket that did not contain any data.data.last_sparse_bucket (or dlsb, dataLastSparseBucket): The timestamp of the last bucket that was considered sparse.data.latest_record (or dlr, dataLatestRecord): The timestamp of the latest chronologically input document.data.missing_fields (or dmf, dataMissingFields): The number of input documents that are missing a field that the anomaly
detection job is configured to analyze. Input documents with missing fields
are still processed because it is possible that not all fields are missing.data.out_of_order_timestamps (or doot, dataOutOfOrderTimestamps): The number of input documents that have a timestamp chronologically
preceding the start of the current anomaly detection bucket offset by the
latency window. This information is applicable only when you provide data
to the anomaly detection job by using the post data API. These out of order
documents are discarded, since jobs require time series data to be in
ascending chronological order.data.processed_fields (or dpf, dataProcessedFields): The total number of fields in all the documents that have been processed by
the anomaly detection job. Only fields that are specified in the detector
configuration object contribute to this count. The timestamp is not
included in this count.data.processed_records (or dpr, dataProcessedRecords): The number of input documents that have been processed by the anomaly
detection job. This value includes documents with missing fields, since
they are nonetheless analyzed. If you use datafeeds and have aggregations
in your search query, the processed record count is the number of
aggregation results processed, not the number of Elasticsearch documents.data.sparse_buckets (or dsb, dataSparseBuckets): The number of buckets that contained few data points compared to the
expected number of data points.forecasts.memory.avg (or fmavg, forecastsMemoryAvg): The average memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.max (or fmmax, forecastsMemoryMax): The maximum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.min (or fmmin, forecastsMemoryMin): The minimum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.total (or fmt, forecastsMemoryTotal): The total memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.records.avg (or fravg, forecastsRecordsAvg): The average number of model_forecast` documents written for forecasts
related to the anomaly detection job.forecasts.records.max (or frmax, forecastsRecordsMax): The maximum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.min (or frmin, forecastsRecordsMin): The minimum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.total (or frt, forecastsRecordsTotal): The total number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.time.avg (or ftavg, forecastsTimeAvg): The average runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.max (or ftmax, forecastsTimeMax): The maximum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.min (or ftmin, forecastsTimeMin): The minimum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.total (or ftt, forecastsTimeTotal): The total runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.total (or ft, forecastsTotal): The number of individual forecasts currently available for the job.id: Identifier for the anomaly detection job.model.bucket_allocation_failures (or mbaf, modelBucketAllocationFailures): The number of buckets for which new entities in incoming data were not
processed due to insufficient model memory.model.by_fields (or mbf, modelByFields): The number of by field values that were analyzed by the models. This value
is cumulative for all detectors in the job.model.bytes (or mb, modelBytes): The number of bytes of memory used by the models. This is the maximum value
since the last time the model was persisted. If the job is closed, this
value indicates the latest size.model.bytes_exceeded (or mbe, modelBytesExceeded): The number of bytes over the high limit for memory usage at the last
allocation failure.model.categorization_status (or mcs, modelCategorizationStatus): The status of categorization for the job: ok or warn. If ok,
categorization is performing acceptably well (or not being used at all). If
warn, categorization is detecting a distribution of categories that
suggests the input data is inappropriate for categorization. Problems could
be that there is only one category, more than 90% of categories are rare,
the number of categories is greater than 50% of the number of categorized
documents, there are no frequently matched categories, or more than 50% of
categories are dead.model.categorized_doc_count (or mcdc, modelCategorizedDocCount): The number of documents that have had a field categorized.model.dead_category_count (or mdcc, modelDeadCategoryCount): The number of categories created by categorization that will never be
assigned again because another category’s definition makes it a superset of
the dead category. Dead categories are a side effect of the way
categorization has no prior training.model.failed_category_count (or mdcc, modelFailedCategoryCount): The number of times that categorization wanted to create a new category but
couldn’t because the job had hit its model memory limit. This count does
not track which specific categories failed to be created. Therefore, you
cannot use this value to determine the number of unique categories that
were missed.model.frequent_category_count (or mfcc, modelFrequentCategoryCount): The number of categories that match more than 1% of categorized documents.model.log_time (or mlt, modelLogTime): The timestamp when the model stats were gathered, according to server time.model.memory_limit (or mml, modelMemoryLimit): The timestamp when the model stats were gathered, according to server time.model.memory_status (or mms, modelMemoryStatus): The status of the mathematical models: ok, soft_limit, or hard_limit.
If ok, the models stayed below the configured value. If soft_limit, the
models used more than 60% of the configured memory limit and older unused
models will be pruned to free up space. Additionally, in categorization jobs
no further category examples will be stored. If hard_limit, the models
used more space than the configured memory limit. As a result, not all
incoming data was processed.model.over_fields (or mof, modelOverFields): The number of over field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.partition_fields (or mpf, modelPartitionFields): The number of partition field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.rare_category_count (or mrcc, modelRareCategoryCount): The number of categories that match just one categorized document.model.timestamp (or mt, modelTimestamp): The timestamp of the last record when the model stats were gathered.model.total_category_count (or mtcc, modelTotalCategoryCount): The number of categories created by categorization.node.address (or na, nodeAddress): The network address of the node that runs the job. This information is
available only for open jobs.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that runs the job. This information is
available only for open jobs.node.id (or ni, nodeId): The unique identifier of the node that runs the job. This information is
available only for open jobs.node.name (or nn, nodeName): The name of the node that runs the job. This information is available only
for open jobs.opened_time (or ot): For open jobs only, the elapsed time for which the job has been open.state (or s): The status of the anomaly detection job: closed, closing, failed,
opened, or opening. If closed, the job finished successfully with its
model state persisted. The job must be opened before it can accept further
data. If closing, the job close action is in progress and has not yet
completed. A closing job cannot accept further data. If failed, the job
did not finish successfully due to an error. This situation can occur due
to invalid input data, a fatal error occurring during the analysis, or an
external interaction such as the process being killed by the Linux out of
memory (OOM) killer. If the job had irrevocably failed, it must be force
closed and then deleted. If the datafeed can be corrected, the job can be
closed and then re-opened. If opened, the job is available to receive and
process data. If opening, the job open action is in progress and has not
yet completed.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
assignment_explanation (or ae): For open anomaly detection jobs only, contains messages relating to the
selection of a node to run the job.buckets.count (or bc, bucketsCount): The number of bucket results produced by the job.buckets.time.exp_avg (or btea, bucketsTimeExpAvg): Exponential moving average of all bucket processing times, in milliseconds.buckets.time.exp_avg_hour (or bteah, bucketsTimeExpAvgHour): Exponentially-weighted moving average of bucket processing times calculated
in a 1 hour time window, in milliseconds.buckets.time.max (or btmax, bucketsTimeMax): Maximum among all bucket processing times, in milliseconds.buckets.time.min (or btmin, bucketsTimeMin): Minimum among all bucket processing times, in milliseconds.buckets.time.total (or btt, bucketsTimeTotal): Sum of all bucket processing times, in milliseconds.data.buckets (or db, dataBuckets): The number of buckets processed.data.earliest_record (or der, dataEarliestRecord): The timestamp of the earliest chronologically input document.data.empty_buckets (or deb, dataEmptyBuckets): The number of buckets which did not contain any data.data.input_bytes (or dib, dataInputBytes): The number of bytes of input data posted to the anomaly detection job.data.input_fields (or dif, dataInputFields): The total number of fields in input documents posted to the anomaly
detection job. This count includes fields that are not used in the analysis.
However, be aware that if you are using a datafeed, it extracts only the
required fields from the documents it retrieves before posting them to the job.data.input_records (or dir, dataInputRecords): The number of input documents posted to the anomaly detection job.data.invalid_dates (or did, dataInvalidDates): The number of input documents with either a missing date field or a date
that could not be parsed.data.last (or dl, dataLast): The timestamp at which data was last analyzed, according to server time.data.last_empty_bucket (or dleb, dataLastEmptyBucket): The timestamp of the last bucket that did not contain any data.data.last_sparse_bucket (or dlsb, dataLastSparseBucket): The timestamp of the last bucket that was considered sparse.data.latest_record (or dlr, dataLatestRecord): The timestamp of the latest chronologically input document.data.missing_fields (or dmf, dataMissingFields): The number of input documents that are missing a field that the anomaly
detection job is configured to analyze. Input documents with missing fields
are still processed because it is possible that not all fields are missing.data.out_of_order_timestamps (or doot, dataOutOfOrderTimestamps): The number of input documents that have a timestamp chronologically
preceding the start of the current anomaly detection bucket offset by the
latency window. This information is applicable only when you provide data
to the anomaly detection job by using the post data API. These out of order
documents are discarded, since jobs require time series data to be in
ascending chronological order.data.processed_fields (or dpf, dataProcessedFields): The total number of fields in all the documents that have been processed by
the anomaly detection job. Only fields that are specified in the detector
configuration object contribute to this count. The timestamp is not
included in this count.data.processed_records (or dpr, dataProcessedRecords): The number of input documents that have been processed by the anomaly
detection job. This value includes documents with missing fields, since
they are nonetheless analyzed. If you use datafeeds and have aggregations
in your search query, the processed record count is the number of
aggregation results processed, not the number of Elasticsearch documents.data.sparse_buckets (or dsb, dataSparseBuckets): The number of buckets that contained few data points compared to the
expected number of data points.forecasts.memory.avg (or fmavg, forecastsMemoryAvg): The average memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.max (or fmmax, forecastsMemoryMax): The maximum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.min (or fmmin, forecastsMemoryMin): The minimum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.total (or fmt, forecastsMemoryTotal): The total memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.records.avg (or fravg, forecastsRecordsAvg): The average number of model_forecast` documents written for forecasts
related to the anomaly detection job.forecasts.records.max (or frmax, forecastsRecordsMax): The maximum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.min (or frmin, forecastsRecordsMin): The minimum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.total (or frt, forecastsRecordsTotal): The total number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.time.avg (or ftavg, forecastsTimeAvg): The average runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.max (or ftmax, forecastsTimeMax): The maximum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.min (or ftmin, forecastsTimeMin): The minimum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.total (or ftt, forecastsTimeTotal): The total runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.total (or ft, forecastsTotal): The number of individual forecasts currently available for the job.id: Identifier for the anomaly detection job.model.bucket_allocation_failures (or mbaf, modelBucketAllocationFailures): The number of buckets for which new entities in incoming data were not
processed due to insufficient model memory.model.by_fields (or mbf, modelByFields): The number of by field values that were analyzed by the models. This value
is cumulative for all detectors in the job.model.bytes (or mb, modelBytes): The number of bytes of memory used by the models. This is the maximum value
since the last time the model was persisted. If the job is closed, this
value indicates the latest size.model.bytes_exceeded (or mbe, modelBytesExceeded): The number of bytes over the high limit for memory usage at the last
allocation failure.model.categorization_status (or mcs, modelCategorizationStatus): The status of categorization for the job: ok or warn. If ok,
categorization is performing acceptably well (or not being used at all). If
warn, categorization is detecting a distribution of categories that
suggests the input data is inappropriate for categorization. Problems could
be that there is only one category, more than 90% of categories are rare,
the number of categories is greater than 50% of the number of categorized
documents, there are no frequently matched categories, or more than 50% of
categories are dead.model.categorized_doc_count (or mcdc, modelCategorizedDocCount): The number of documents that have had a field categorized.model.dead_category_count (or mdcc, modelDeadCategoryCount): The number of categories created by categorization that will never be
assigned again because another category’s definition makes it a superset of
the dead category. Dead categories are a side effect of the way
categorization has no prior training.model.failed_category_count (or mdcc, modelFailedCategoryCount): The number of times that categorization wanted to create a new category but
couldn’t because the job had hit its model memory limit. This count does
not track which specific categories failed to be created. Therefore, you
cannot use this value to determine the number of unique categories that
were missed.model.frequent_category_count (or mfcc, modelFrequentCategoryCount): The number of categories that match more than 1% of categorized documents.model.log_time (or mlt, modelLogTime): The timestamp when the model stats were gathered, according to server time.model.memory_limit (or mml, modelMemoryLimit): The timestamp when the model stats were gathered, according to server time.model.memory_status (or mms, modelMemoryStatus): The status of the mathematical models: ok, soft_limit, or hard_limit.
If ok, the models stayed below the configured value. If soft_limit, the
models used more than 60% of the configured memory limit and older unused
models will be pruned to free up space. Additionally, in categorization jobs
no further category examples will be stored. If hard_limit, the models
used more space than the configured memory limit. As a result, not all
incoming data was processed.model.over_fields (or mof, modelOverFields): The number of over field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.partition_fields (or mpf, modelPartitionFields): The number of partition field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.rare_category_count (or mrcc, modelRareCategoryCount): The number of categories that match just one categorized document.model.timestamp (or mt, modelTimestamp): The timestamp of the last record when the model stats were gathered.model.total_category_count (or mtcc, modelTotalCategoryCount): The number of categories created by categorization.node.address (or na, nodeAddress): The network address of the node that runs the job. This information is
available only for open jobs.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that runs the job. This information is
available only for open jobs.node.id (or ni, nodeId): The unique identifier of the node that runs the job. This information is
available only for open jobs.node.name (or nn, nodeName): The name of the node that runs the job. This information is available only
for open jobs.opened_time (or ot): For open jobs only, the elapsed time for which the job has been open.state (or s): The status of the anomaly detection job: closed, closing, failed,
opened, or opening. If closed, the job finished successfully with its
model state persisted. The job must be opened before it can accept further
data. If closing, the job close action is in progress and has not yet
completed. A closing job cannot accept further data. If failed, the job
did not finish successfully due to an error. This situation can occur due
to invalid input data, a fatal error occurring during the analysis, or an
external interaction such as the process being killed by the Linux out of
memory (OOM) killer. If the job had irrevocably failed, it must be force
closed and then deleted. If the datafeed can be corrected, the job can be
closed and then re-opened. If opened, the job is available to receive and
process data. If opening, the job open action is in progress and has not
yet completed.The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/anomaly_detectors' \
--header "Authorization: $API_KEY"[
{
"id": "high_sum_total_sales",
"s": "closed",
"dpr": "14022",
"mb": "1.5mb"
},
{
"id": "low_request_rate",
"s": "closed",
"dpr": "1216",
"mb": "40.5kb"
},
{
"id": "response_code_rates",
"s": "closed",
"dpr": "28146",
"mb": "132.7kb"
},
{
"id": "url_scanning",
"s": "closed",
"dpr": "28146",
"mb": "501.6kb"
}
]Get configuration and usage information for anomaly detection jobs.
This API returns a maximum of 10,000 jobs.
If the Elasticsearch security features are enabled, you must have monitor_ml,
monitor, manage_ml, or manage cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get anomaly detection job statistics API.
Identifier for the anomaly detection job.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty jobs array when there are no matches and the subset of results when there
are partial matches. If false, the API returns a 404 status code when there are no matches or only partial
matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
Comma-separated list of column names to display.
Supported values include:
assignment_explanation (or ae): For open anomaly detection jobs only, contains messages relating to the
selection of a node to run the job.buckets.count (or bc, bucketsCount): The number of bucket results produced by the job.buckets.time.exp_avg (or btea, bucketsTimeExpAvg): Exponential moving average of all bucket processing times, in milliseconds.buckets.time.exp_avg_hour (or bteah, bucketsTimeExpAvgHour): Exponentially-weighted moving average of bucket processing times calculated
in a 1 hour time window, in milliseconds.buckets.time.max (or btmax, bucketsTimeMax): Maximum among all bucket processing times, in milliseconds.buckets.time.min (or btmin, bucketsTimeMin): Minimum among all bucket processing times, in milliseconds.buckets.time.total (or btt, bucketsTimeTotal): Sum of all bucket processing times, in milliseconds.data.buckets (or db, dataBuckets): The number of buckets processed.data.earliest_record (or der, dataEarliestRecord): The timestamp of the earliest chronologically input document.data.empty_buckets (or deb, dataEmptyBuckets): The number of buckets which did not contain any data.data.input_bytes (or dib, dataInputBytes): The number of bytes of input data posted to the anomaly detection job.data.input_fields (or dif, dataInputFields): The total number of fields in input documents posted to the anomaly
detection job. This count includes fields that are not used in the analysis.
However, be aware that if you are using a datafeed, it extracts only the
required fields from the documents it retrieves before posting them to the job.data.input_records (or dir, dataInputRecords): The number of input documents posted to the anomaly detection job.data.invalid_dates (or did, dataInvalidDates): The number of input documents with either a missing date field or a date
that could not be parsed.data.last (or dl, dataLast): The timestamp at which data was last analyzed, according to server time.data.last_empty_bucket (or dleb, dataLastEmptyBucket): The timestamp of the last bucket that did not contain any data.data.last_sparse_bucket (or dlsb, dataLastSparseBucket): The timestamp of the last bucket that was considered sparse.data.latest_record (or dlr, dataLatestRecord): The timestamp of the latest chronologically input document.data.missing_fields (or dmf, dataMissingFields): The number of input documents that are missing a field that the anomaly
detection job is configured to analyze. Input documents with missing fields
are still processed because it is possible that not all fields are missing.data.out_of_order_timestamps (or doot, dataOutOfOrderTimestamps): The number of input documents that have a timestamp chronologically
preceding the start of the current anomaly detection bucket offset by the
latency window. This information is applicable only when you provide data
to the anomaly detection job by using the post data API. These out of order
documents are discarded, since jobs require time series data to be in
ascending chronological order.data.processed_fields (or dpf, dataProcessedFields): The total number of fields in all the documents that have been processed by
the anomaly detection job. Only fields that are specified in the detector
configuration object contribute to this count. The timestamp is not
included in this count.data.processed_records (or dpr, dataProcessedRecords): The number of input documents that have been processed by the anomaly
detection job. This value includes documents with missing fields, since
they are nonetheless analyzed. If you use datafeeds and have aggregations
in your search query, the processed record count is the number of
aggregation results processed, not the number of Elasticsearch documents.data.sparse_buckets (or dsb, dataSparseBuckets): The number of buckets that contained few data points compared to the
expected number of data points.forecasts.memory.avg (or fmavg, forecastsMemoryAvg): The average memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.max (or fmmax, forecastsMemoryMax): The maximum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.min (or fmmin, forecastsMemoryMin): The minimum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.total (or fmt, forecastsMemoryTotal): The total memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.records.avg (or fravg, forecastsRecordsAvg): The average number of model_forecast` documents written for forecasts
related to the anomaly detection job.forecasts.records.max (or frmax, forecastsRecordsMax): The maximum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.min (or frmin, forecastsRecordsMin): The minimum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.total (or frt, forecastsRecordsTotal): The total number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.time.avg (or ftavg, forecastsTimeAvg): The average runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.max (or ftmax, forecastsTimeMax): The maximum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.min (or ftmin, forecastsTimeMin): The minimum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.total (or ftt, forecastsTimeTotal): The total runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.total (or ft, forecastsTotal): The number of individual forecasts currently available for the job.id: Identifier for the anomaly detection job.model.bucket_allocation_failures (or mbaf, modelBucketAllocationFailures): The number of buckets for which new entities in incoming data were not
processed due to insufficient model memory.model.by_fields (or mbf, modelByFields): The number of by field values that were analyzed by the models. This value
is cumulative for all detectors in the job.model.bytes (or mb, modelBytes): The number of bytes of memory used by the models. This is the maximum value
since the last time the model was persisted. If the job is closed, this
value indicates the latest size.model.bytes_exceeded (or mbe, modelBytesExceeded): The number of bytes over the high limit for memory usage at the last
allocation failure.model.categorization_status (or mcs, modelCategorizationStatus): The status of categorization for the job: ok or warn. If ok,
categorization is performing acceptably well (or not being used at all). If
warn, categorization is detecting a distribution of categories that
suggests the input data is inappropriate for categorization. Problems could
be that there is only one category, more than 90% of categories are rare,
the number of categories is greater than 50% of the number of categorized
documents, there are no frequently matched categories, or more than 50% of
categories are dead.model.categorized_doc_count (or mcdc, modelCategorizedDocCount): The number of documents that have had a field categorized.model.dead_category_count (or mdcc, modelDeadCategoryCount): The number of categories created by categorization that will never be
assigned again because another category’s definition makes it a superset of
the dead category. Dead categories are a side effect of the way
categorization has no prior training.model.failed_category_count (or mdcc, modelFailedCategoryCount): The number of times that categorization wanted to create a new category but
couldn’t because the job had hit its model memory limit. This count does
not track which specific categories failed to be created. Therefore, you
cannot use this value to determine the number of unique categories that
were missed.model.frequent_category_count (or mfcc, modelFrequentCategoryCount): The number of categories that match more than 1% of categorized documents.model.log_time (or mlt, modelLogTime): The timestamp when the model stats were gathered, according to server time.model.memory_limit (or mml, modelMemoryLimit): The timestamp when the model stats were gathered, according to server time.model.memory_status (or mms, modelMemoryStatus): The status of the mathematical models: ok, soft_limit, or hard_limit.
If ok, the models stayed below the configured value. If soft_limit, the
models used more than 60% of the configured memory limit and older unused
models will be pruned to free up space. Additionally, in categorization jobs
no further category examples will be stored. If hard_limit, the models
used more space than the configured memory limit. As a result, not all
incoming data was processed.model.over_fields (or mof, modelOverFields): The number of over field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.partition_fields (or mpf, modelPartitionFields): The number of partition field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.rare_category_count (or mrcc, modelRareCategoryCount): The number of categories that match just one categorized document.model.timestamp (or mt, modelTimestamp): The timestamp of the last record when the model stats were gathered.model.total_category_count (or mtcc, modelTotalCategoryCount): The number of categories created by categorization.node.address (or na, nodeAddress): The network address of the node that runs the job. This information is
available only for open jobs.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that runs the job. This information is
available only for open jobs.node.id (or ni, nodeId): The unique identifier of the node that runs the job. This information is
available only for open jobs.node.name (or nn, nodeName): The name of the node that runs the job. This information is available only
for open jobs.opened_time (or ot): For open jobs only, the elapsed time for which the job has been open.state (or s): The status of the anomaly detection job: closed, closing, failed,
opened, or opening. If closed, the job finished successfully with its
model state persisted. The job must be opened before it can accept further
data. If closing, the job close action is in progress and has not yet
completed. A closing job cannot accept further data. If failed, the job
did not finish successfully due to an error. This situation can occur due
to invalid input data, a fatal error occurring during the analysis, or an
external interaction such as the process being killed by the Linux out of
memory (OOM) killer. If the job had irrevocably failed, it must be force
closed and then deleted. If the datafeed can be corrected, the job can be
closed and then re-opened. If opened, the job is available to receive and
process data. If opening, the job open action is in progress and has not
yet completed.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
assignment_explanation (or ae): For open anomaly detection jobs only, contains messages relating to the
selection of a node to run the job.buckets.count (or bc, bucketsCount): The number of bucket results produced by the job.buckets.time.exp_avg (or btea, bucketsTimeExpAvg): Exponential moving average of all bucket processing times, in milliseconds.buckets.time.exp_avg_hour (or bteah, bucketsTimeExpAvgHour): Exponentially-weighted moving average of bucket processing times calculated
in a 1 hour time window, in milliseconds.buckets.time.max (or btmax, bucketsTimeMax): Maximum among all bucket processing times, in milliseconds.buckets.time.min (or btmin, bucketsTimeMin): Minimum among all bucket processing times, in milliseconds.buckets.time.total (or btt, bucketsTimeTotal): Sum of all bucket processing times, in milliseconds.data.buckets (or db, dataBuckets): The number of buckets processed.data.earliest_record (or der, dataEarliestRecord): The timestamp of the earliest chronologically input document.data.empty_buckets (or deb, dataEmptyBuckets): The number of buckets which did not contain any data.data.input_bytes (or dib, dataInputBytes): The number of bytes of input data posted to the anomaly detection job.data.input_fields (or dif, dataInputFields): The total number of fields in input documents posted to the anomaly
detection job. This count includes fields that are not used in the analysis.
However, be aware that if you are using a datafeed, it extracts only the
required fields from the documents it retrieves before posting them to the job.data.input_records (or dir, dataInputRecords): The number of input documents posted to the anomaly detection job.data.invalid_dates (or did, dataInvalidDates): The number of input documents with either a missing date field or a date
that could not be parsed.data.last (or dl, dataLast): The timestamp at which data was last analyzed, according to server time.data.last_empty_bucket (or dleb, dataLastEmptyBucket): The timestamp of the last bucket that did not contain any data.data.last_sparse_bucket (or dlsb, dataLastSparseBucket): The timestamp of the last bucket that was considered sparse.data.latest_record (or dlr, dataLatestRecord): The timestamp of the latest chronologically input document.data.missing_fields (or dmf, dataMissingFields): The number of input documents that are missing a field that the anomaly
detection job is configured to analyze. Input documents with missing fields
are still processed because it is possible that not all fields are missing.data.out_of_order_timestamps (or doot, dataOutOfOrderTimestamps): The number of input documents that have a timestamp chronologically
preceding the start of the current anomaly detection bucket offset by the
latency window. This information is applicable only when you provide data
to the anomaly detection job by using the post data API. These out of order
documents are discarded, since jobs require time series data to be in
ascending chronological order.data.processed_fields (or dpf, dataProcessedFields): The total number of fields in all the documents that have been processed by
the anomaly detection job. Only fields that are specified in the detector
configuration object contribute to this count. The timestamp is not
included in this count.data.processed_records (or dpr, dataProcessedRecords): The number of input documents that have been processed by the anomaly
detection job. This value includes documents with missing fields, since
they are nonetheless analyzed. If you use datafeeds and have aggregations
in your search query, the processed record count is the number of
aggregation results processed, not the number of Elasticsearch documents.data.sparse_buckets (or dsb, dataSparseBuckets): The number of buckets that contained few data points compared to the
expected number of data points.forecasts.memory.avg (or fmavg, forecastsMemoryAvg): The average memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.max (or fmmax, forecastsMemoryMax): The maximum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.min (or fmmin, forecastsMemoryMin): The minimum memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.memory.total (or fmt, forecastsMemoryTotal): The total memory usage in bytes for forecasts related to the anomaly
detection job.forecasts.records.avg (or fravg, forecastsRecordsAvg): The average number of model_forecast` documents written for forecasts
related to the anomaly detection job.forecasts.records.max (or frmax, forecastsRecordsMax): The maximum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.min (or frmin, forecastsRecordsMin): The minimum number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.records.total (or frt, forecastsRecordsTotal): The total number of model_forecast documents written for forecasts
related to the anomaly detection job.forecasts.time.avg (or ftavg, forecastsTimeAvg): The average runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.max (or ftmax, forecastsTimeMax): The maximum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.min (or ftmin, forecastsTimeMin): The minimum runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.time.total (or ftt, forecastsTimeTotal): The total runtime in milliseconds for forecasts related to the anomaly
detection job.forecasts.total (or ft, forecastsTotal): The number of individual forecasts currently available for the job.id: Identifier for the anomaly detection job.model.bucket_allocation_failures (or mbaf, modelBucketAllocationFailures): The number of buckets for which new entities in incoming data were not
processed due to insufficient model memory.model.by_fields (or mbf, modelByFields): The number of by field values that were analyzed by the models. This value
is cumulative for all detectors in the job.model.bytes (or mb, modelBytes): The number of bytes of memory used by the models. This is the maximum value
since the last time the model was persisted. If the job is closed, this
value indicates the latest size.model.bytes_exceeded (or mbe, modelBytesExceeded): The number of bytes over the high limit for memory usage at the last
allocation failure.model.categorization_status (or mcs, modelCategorizationStatus): The status of categorization for the job: ok or warn. If ok,
categorization is performing acceptably well (or not being used at all). If
warn, categorization is detecting a distribution of categories that
suggests the input data is inappropriate for categorization. Problems could
be that there is only one category, more than 90% of categories are rare,
the number of categories is greater than 50% of the number of categorized
documents, there are no frequently matched categories, or more than 50% of
categories are dead.model.categorized_doc_count (or mcdc, modelCategorizedDocCount): The number of documents that have had a field categorized.model.dead_category_count (or mdcc, modelDeadCategoryCount): The number of categories created by categorization that will never be
assigned again because another category’s definition makes it a superset of
the dead category. Dead categories are a side effect of the way
categorization has no prior training.model.failed_category_count (or mdcc, modelFailedCategoryCount): The number of times that categorization wanted to create a new category but
couldn’t because the job had hit its model memory limit. This count does
not track which specific categories failed to be created. Therefore, you
cannot use this value to determine the number of unique categories that
were missed.model.frequent_category_count (or mfcc, modelFrequentCategoryCount): The number of categories that match more than 1% of categorized documents.model.log_time (or mlt, modelLogTime): The timestamp when the model stats were gathered, according to server time.model.memory_limit (or mml, modelMemoryLimit): The timestamp when the model stats were gathered, according to server time.model.memory_status (or mms, modelMemoryStatus): The status of the mathematical models: ok, soft_limit, or hard_limit.
If ok, the models stayed below the configured value. If soft_limit, the
models used more than 60% of the configured memory limit and older unused
models will be pruned to free up space. Additionally, in categorization jobs
no further category examples will be stored. If hard_limit, the models
used more space than the configured memory limit. As a result, not all
incoming data was processed.model.over_fields (or mof, modelOverFields): The number of over field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.partition_fields (or mpf, modelPartitionFields): The number of partition field values that were analyzed by the models. This
value is cumulative for all detectors in the job.model.rare_category_count (or mrcc, modelRareCategoryCount): The number of categories that match just one categorized document.model.timestamp (or mt, modelTimestamp): The timestamp of the last record when the model stats were gathered.model.total_category_count (or mtcc, modelTotalCategoryCount): The number of categories created by categorization.node.address (or na, nodeAddress): The network address of the node that runs the job. This information is
available only for open jobs.node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that runs the job. This information is
available only for open jobs.node.id (or ni, nodeId): The unique identifier of the node that runs the job. This information is
available only for open jobs.node.name (or nn, nodeName): The name of the node that runs the job. This information is available only
for open jobs.opened_time (or ot): For open jobs only, the elapsed time for which the job has been open.state (or s): The status of the anomaly detection job: closed, closing, failed,
opened, or opening. If closed, the job finished successfully with its
model state persisted. The job must be opened before it can accept further
data. If closing, the job close action is in progress and has not yet
completed. A closing job cannot accept further data. If failed, the job
did not finish successfully due to an error. This situation can occur due
to invalid input data, a fatal error occurring during the analysis, or an
external interaction such as the process being killed by the Linux out of
memory (OOM) killer. If the job had irrevocably failed, it must be force
closed and then deleted. If the datafeed can be corrected, the job can be
closed and then re-opened. If opened, the job is available to receive and
process data. If opening, the job open action is in progress and has not
yet completed.The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/anomaly_detectors/{job_id}' \
--header "Authorization: $API_KEY"[
{
"id": "high_sum_total_sales",
"s": "closed",
"dpr": "14022",
"mb": "1.5mb"
},
{
"id": "low_request_rate",
"s": "closed",
"dpr": "1216",
"mb": "40.5kb"
},
{
"id": "response_code_rates",
"s": "closed",
"dpr": "28146",
"mb": "132.7kb"
},
{
"id": "url_scanning",
"s": "closed",
"dpr": "28146",
"mb": "501.6kb"
}
]Get configuration and usage information about inference trained models.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get trained models statistics API.
Specifies what to do when the request: contains wildcard expressions and there are no models that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, the API returns an empty array when there are no matches and the subset of results when there are partial matches.
If false, the API returns a 404 status code when there are no matches or only partial matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
A comma-separated list of column names to display.
Supported values include:
create_time (or ct): The time when the trained model was created.created_by (or c, createdBy): Information on the creator of the trained model.data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only
displayed if it is still available.description (or d): The description of the trained model.heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.id: Identifier for the trained model.ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the
trained model.ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained
model.ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.license (or l): The license level of the trained model.operations (or o, modelOperations): The estimated number of operations to use the trained model. This number
helps measuring the computational complexity of the model.version (or v): The Elasticsearch version number in which the trained model was created.A comma-separated list of column names or aliases used to sort the response.
Supported values include:
create_time (or ct): The time when the trained model was created.created_by (or c, createdBy): Information on the creator of the trained model.data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only
displayed if it is still available.description (or d): The description of the trained model.heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.id: Identifier for the trained model.ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the
trained model.ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained
model.ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.license (or l): The license level of the trained model.operations (or o, modelOperations): The estimated number of operations to use the trained model. This number
helps measuring the computational complexity of the model.version (or v): The Elasticsearch version number in which the trained model was created.Skips the specified number of transforms.
The maximum number of transforms to display.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/trained_models' \
--header "Authorization: $API_KEY"[
{
"id": "ddddd-1580216177138",
"heap_size": "0b",
"operations": "196",
"create_time": "2025-03-25T00:01:38.662Z",
"type": "pytorch",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
},
{
"id": "lang_ident_model_1",
"heap_size": "1mb",
"operations": "39629",
"create_time": "2019-12-05T12:28:34.594Z",
"type": "lang_ident",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
}
]Get configuration and usage information about inference trained models.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get trained models statistics API.
A unique identifier for the trained model.
Specifies what to do when the request: contains wildcard expressions and there are no models that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, the API returns an empty array when there are no matches and the subset of results when there are partial matches.
If false, the API returns a 404 status code when there are no matches or only partial matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
A comma-separated list of column names to display.
Supported values include:
create_time (or ct): The time when the trained model was created.created_by (or c, createdBy): Information on the creator of the trained model.data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only
displayed if it is still available.description (or d): The description of the trained model.heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.id: Identifier for the trained model.ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the
trained model.ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained
model.ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.license (or l): The license level of the trained model.operations (or o, modelOperations): The estimated number of operations to use the trained model. This number
helps measuring the computational complexity of the model.version (or v): The Elasticsearch version number in which the trained model was created.A comma-separated list of column names or aliases used to sort the response.
Supported values include:
create_time (or ct): The time when the trained model was created.created_by (or c, createdBy): Information on the creator of the trained model.data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only
displayed if it is still available.description (or d): The description of the trained model.heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.id: Identifier for the trained model.ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the
trained model.ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained
model.ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.license (or l): The license level of the trained model.operations (or o, modelOperations): The estimated number of operations to use the trained model. This number
helps measuring the computational complexity of the model.version (or v): The Elasticsearch version number in which the trained model was created.Skips the specified number of transforms.
The maximum number of transforms to display.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/trained_models/{model_id}' \
--header "Authorization: $API_KEY"[
{
"id": "ddddd-1580216177138",
"heap_size": "0b",
"operations": "196",
"create_time": "2025-03-25T00:01:38.662Z",
"type": "pytorch",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
},
{
"id": "lang_ident_model_1",
"heap_size": "1mb",
"operations": "39629",
"create_time": "2019-12-05T12:28:34.594Z",
"type": "lang_ident",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
}
]Get configuration and usage information about transforms.
CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.
Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Skips the specified number of transforms.
Comma-separated list of column names to display.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
The maximum number of transforms to obtain.
curl \
--request GET 'http://api.example.com/_cat/transforms' \
--header "Authorization: $API_KEY"[
{
"id" : "ecommerce_transform",
"state" : "started",
"checkpoint" : "1",
"documents_processed" : "705",
"checkpoint_progress" : "100.00",
"changes_last_detection_time" : null
}
]Get configuration and usage information about transforms.
CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.
A transform identifier or a wildcard expression. If you do not specify one of these options, the API returns information for all transforms.
Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Skips the specified number of transforms.
Comma-separated list of column names to display.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
The maximum number of transforms to obtain.
curl \
--request GET 'http://api.example.com/_cat/transforms/{transform_id}' \
--header "Authorization: $API_KEY"[
{
"id" : "ecommerce_transform",
"state" : "started",
"checkpoint" : "1",
"documents_processed" : "705",
"checkpoint_progress" : "100.00",
"changes_last_detection_time" : null
}
]Limits the information returned to the specific target. Supports a comma-separated list, such as http,ingest.
Supported values include: _all, http, ingest, thread_pool, script
curl \
--request GET 'http://api.example.com/_info/{target}' \
--header "Authorization: $API_KEY"curl \
--request HEAD 'http://api.example.com/' \
--header "Authorization: $API_KEY"The connector and sync jobs APIs provide a convenient way to create and manage Elastic connectors and sync jobs in an internal index.
Connectors are Elasticsearch integrations for syncing content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure.
This API provides an alternative to relying solely on Kibana UI for connector and sync job management. The API comes with a set of validations and assertions to ensure that the state representation in the internal index remains valid.
This API requires the manage_connector privilege or, for read-only endpoints, the monitor_connector privilege.
Update the last_seen field in the connector and set it to the current timestamp.
The unique identifier of the connector to be checked in
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_check_in' \
--header "Authorization: $API_KEY"{
"result": "updated"
}The unique identifier of the connector
A flag to indicate if the desired connector should be fetched, even if it was soft-deleted.
curl \
--request GET 'http://api.example.com/_connector/{connector_id}' \
--header "Authorization: $API_KEY"The unique identifier of the connector to be created or updated. ID is auto-generated if not provided.
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"search-google-drive\",\n \"name\": \"My Connector\",\n \"service_type\": \"google_drive\"\n}"'{
"index_name": "search-google-drive",
"name": "My Connector",
"service_type": "google_drive"
}{
"index_name": "search-google-drive",
"name": "My Connector",
"description": "My Connector to sync data to Elastic index from Google Drive",
"service_type": "google_drive",
"language": "english"
}{
"result": "created",
"id": "my-connector"
}Removes a connector and associated sync jobs. This is a destructive action that is not recoverable. NOTE: This action doesn’t delete any API keys, ingest pipelines, or data indices associated with the connector. These need to be removed manually.
The unique identifier of the connector to be deleted
A flag indicating if associated sync jobs should be also removed. Defaults to false.
A flag indicating if the connector should be hard deleted.
curl \
--request DELETE 'http://api.example.com/_connector/{connector_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Starting offset (default: 0)
Specifies a max number of results to get
A comma-separated list of connector index names to fetch connector documents for
A comma-separated list of connector names to fetch connector documents for
A comma-separated list of connector service types to fetch connector documents for
A flag to indicate if the desired connector should be fetched, even if it was soft-deleted.
A wildcard query string that filters connectors with matching name, description or index name
curl \
--request GET 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY"curl \
--request PUT 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"search-google-drive\",\n \"name\": \"My Connector\",\n \"service_type\": \"google_drive\"\n}"'{
"index_name": "search-google-drive",
"name": "My Connector",
"service_type": "google_drive"
}{
"index_name": "search-google-drive",
"name": "My Connector",
"description": "My Connector to sync data to Elastic index from Google Drive",
"service_type": "google_drive",
"language": "english"
}{
"result": "created",
"id": "my-connector"
}Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure. Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud. Self-managed connectors (Connector clients) are self-managed on your infrastructure.
curl \
--request POST 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"description":"string","index_name":"string","is_native":true,"language":"string","name":"string","service_type":"string"}'Cancel a connector sync job, which sets the status to cancelling and updates cancellation_requested_at to the current time.
The connector service is then responsible for setting the status of connector sync jobs to cancelled.
The unique identifier of the connector sync job
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_cancel' \
--header "Authorization: $API_KEY"The unique identifier of the connector sync job
curl \
--request GET 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}' \
--header "Authorization: $API_KEY"Remove a connector sync job and its associated data. This is a destructive action that is not recoverable.
The unique identifier of the connector sync job to be deleted
curl \
--request DELETE 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Get information about all stored connector sync jobs listed by their creation date in ascending order.
Starting offset (default: 0)
Specifies a max number of results to get
A sync job status to fetch connector sync jobs for
Values are canceling, canceled, completed, error, in_progress, pending, or suspended.
A connector id to fetch connector sync jobs for
A comma-separated list of job types to fetch the sync jobs for
Supported values include: full, incremental, access_control
curl \
--request GET 'http://api.example.com/_connector/_sync_job' \
--header "Authorization: $API_KEY"Create a connector sync job document in the internal index and initialize its counters and timestamps with default values.
Values are full, incremental, or access_control.
Values are on_demand or scheduled.
curl \
--request POST 'http://api.example.com/_connector/_sync_job' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"id\": \"connector-id\",\n \"job_type\": \"full\",\n \"trigger_method\": \"on_demand\"\n}"'{
"id": "connector-id",
"job_type": "full",
"trigger_method": "on_demand"
}Activates the valid draft filtering for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
--header "Authorization: $API_KEY"Update the api_key_id and api_key_secret_id fields of a connector.
You can specify the ID of the API key used for authorization and the ID of the connector secret where the API key is stored.
The connector secret ID is required only for Elastic managed (native) connectors.
Self-managed connectors (connector clients) do not use this field.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_api_key_id' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"api_key_id\": \"my-api-key-id\",\n \"api_key_secret_id\": \"my-connector-secret-id\"\n}"'{
"api_key_id": "my-api-key-id",
"api_key_secret_id": "my-connector-secret-id"
}{
"result": "updated"
}Update the configuration field in the connector document.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_configuration' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"values\": {\n \"tenant_id\": \"my-tenant-id\",\n \"tenant_name\": \"my-sharepoint-site\",\n \"client_id\": \"foo\",\n \"secret_value\": \"bar\",\n \"site_collections\": \"*\"\n }\n}"'{
"values": {
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
}
}{
"values": {
"secret_value": "foo-bar"
}
}{
"result": "updated"
}Set the error field for the connector. If the error provided in the request body is non-null, the connector’s status is updated to error. Otherwise, if the error is reset to null, the connector status is updated to connected.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_error' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"error\": \"Houston, we have a problem!\"\n}"'{
"error": "Houston, we have a problem!"
}{
"result": "updated"
}Update the draft filtering configuration of a connector and marks the draft validation state as edited. The filtering draft is activated once validated by the running Elastic connector service. The filtering property is used to configure sync rules (both basic and advanced) for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"rules\": [\n {\n \"field\": \"file_extension\",\n \"id\": \"exclude-txt-files\",\n \"order\": 0,\n \"policy\": \"exclude\",\n \"rule\": \"equals\",\n \"value\": \"txt\"\n },\n {\n \"field\": \"_\",\n \"id\": \"DEFAULT\",\n \"order\": 1,\n \"policy\": \"include\",\n \"rule\": \"regex\",\n \"value\": \".*\"\n }\n ]\n}"'{
"rules": [
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
]
}{
"advanced_snippet": {
"value": [{
"tables": [
"users",
"orders"
],
"query": "SELECT users.id AS id, orders.order_id AS order_id FROM users JOIN orders ON users.id = orders.user_id"
}]
}
}{
"result": "updated"
}Update the draft filtering validation info for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_validation' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"validation":{"errors":[{"ids":["string"],"messages":["string"]}],"state":"edited"}}'Update the index_name field of a connector, specifying the index where the data ingested by the connector is stored.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_index_name' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_name\": \"data-from-my-google-drive\"\n}"'{
"index_name": "data-from-my-google-drive"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_name' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"name\": \"Custom connector\",\n \"description\": \"This is my customized connector\"\n}"'{
"name": "Custom connector",
"description": "This is my customized connector"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_native' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"is_native":true}'When you create a new connector, the configuration of an ingest pipeline is populated with default settings.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_pipeline' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\": {\n \"extract_binary_content\": true,\n \"name\": \"my-connector-pipeline\",\n \"reduce_whitespace\": true,\n \"run_ml_inference\": true\n }\n}"'{
"pipeline": {
"extract_binary_content": true,
"name": "my-connector-pipeline",
"reduce_whitespace": true,
"run_ml_inference": true
}
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_scheduling' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scheduling\": {\n \"access_control\": {\n \"enabled\": true,\n \"interval\": \"0 10 0 * * ?\"\n },\n \"full\": {\n \"enabled\": true,\n \"interval\": \"0 20 0 * * ?\"\n },\n \"incremental\": {\n \"enabled\": false,\n \"interval\": \"0 30 0 * * ?\"\n }\n }\n}"'{
"scheduling": {
"access_control": {
"enabled": true,
"interval": "0 10 0 * * ?"
},
"full": {
"enabled": true,
"interval": "0 20 0 * * ?"
},
"incremental": {
"enabled": false,
"interval": "0 30 0 * * ?"
}
}
}{
"scheduling": {
"full": {
"enabled": true,
"interval": "0 10 0 * * ?"
}
}
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_service_type' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service_type\": \"sharepoint_online\"\n}"'{
"service_type": "sharepoint_online"
}{
"result": "updated"
}The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_status' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"status\": \"needs_configuration\"\n}"'{
"status": "needs_configuration"
}{
"result": "updated"
}Get information about one or more data streams.
Comma-separated list of data stream names used to limit the request.
Wildcard (*) expressions are supported. If omitted, all data streams are returned.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns all relevant default configurations for the index template.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Whether the maximum timestamp for each data stream should be calculated and returned.
curl \
--request GET 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"{
"data_streams": [
{
"name": "my-data-stream",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-2099.03.07-000001",
"index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
},
{
"index_name": ".ds-my-data-stream-2099.03.08-000002",
"index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 2,
"_meta": {
"my-meta-field": "foo"
},
"status": "GREEN",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
},
{
"name": "my-data-stream-two",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-two-2099.03.08-000001",
"index_uuid": "3liBu2SYS5axasRt6fUIpA",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 1,
"_meta": {
"my-meta-field": "foo"
},
"status": "YELLOW",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
}
]
}You must have a matching index template with data stream enabled.
Name of the data stream, which must meet the following criteria:
Lowercase only;
Cannot include \, /, *, ?, ", <, >, |, ,, #, :, or a space character;
Cannot start with -, _, +, or .ds-;
Cannot be . or ..;
Cannot be longer than 255 bytes. Multi-byte characters count towards this limit faster.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request PUT 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"Deletes one or more data streams and their backing indices.
Comma-separated list of data streams to delete. Wildcard (*) expressions are supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Type of data stream that wildcard patterns can match. Supports comma-separated values,such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.curl \
--request DELETE 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"Get information about an index or data stream's current data stream lifecycle status, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any errors encountered during lifecycle execution.
The name of the index to explain
indicates if the API should return the default values the system uses for the index's lifecycle
Specify timeout for connection to master
curl \
--request GET 'http://api.example.com/{index}/_lifecycle/explain' \
--header "Authorization: $API_KEY"{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"rollover_date_millis" : 1679475564293,
"time_since_rollover" : "121ms",
"lifecycle" : { },
"generation_time" : "121ms"
}
}{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"lifecycle" : {
"enabled": true
},
"error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"
}
}Get the data stream lifecycle configuration of one or more data streams.
Comma-separated list of data streams to limit the request.
Supports wildcards (*).
To target all data streams, omit this parameter or use * or _all.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, return all default settings in the response.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_data_stream/{name}/_lifecycle' \
--header "Authorization: $API_KEY"{
"data_streams": [
{
"name": "my-data-stream-1",
"lifecycle": {
"enabled": true,
"data_retention": "7d"
}
},
{
"name": "my-data-stream-2",
"lifecycle": {
"enabled": true,
"data_retention": "7d"
}
}
]
}Update the data stream lifecycle of the specified data streams.
Comma-separated list of data streams used to limit the request.
Supports wildcards (*).
To target all data streams use * or _all.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Valid values are: all, hidden, open, closed, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle
that's disabled (enabled: false) will have no effect on the data stream.
curl \
--request PUT 'http://api.example.com/_data_stream/{name}/_lifecycle' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"data_retention\": \"7d\"\n}"'{
"data_retention": "7d"
}{
"downsampling": [
{
"after": "1d",
"fixed_interval": "10m"
},
{
"after": "7d",
"fixed_interval": "1d"
}
]
}{
"acknowledged": true
}Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns all relevant default configurations for the index template.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Whether the maximum timestamp for each data stream should be calculated and returned.
curl \
--request GET 'http://api.example.com/_data_stream' \
--header "Authorization: $API_KEY"{
"data_streams": [
{
"name": "my-data-stream",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-2099.03.07-000001",
"index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
},
{
"index_name": ".ds-my-data-stream-2099.03.08-000002",
"index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 2,
"_meta": {
"my-meta-field": "foo"
},
"status": "GREEN",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
},
{
"name": "my-data-stream-two",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream-two-2099.03.08-000001",
"index_uuid": "3liBu2SYS5axasRt6fUIpA",
"prefer_ilm": true,
"ilm_policy": "my-lifecycle-policy",
"managed_by": "Index Lifecycle Management"
}
],
"generation": 1,
"_meta": {
"my-meta-field": "foo"
},
"status": "YELLOW",
"next_generation_managed_by": "Index Lifecycle Management",
"prefer_ilm": true,
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy",
"hidden": false,
"system": false,
"allow_custom_routing": false,
"replicated": false,
"rollover_on_write": false
}
]
}Converts an index alias to a data stream.
You must have a matching index template that is data stream enabled.
The alias must meet the following criteria:
The alias must have a write index;
All indices for the alias must have a @timestamp field mapping of a date or date_nanos field type;
The alias must not have any filters;
The alias must not use custom routing.
If successful, the request removes the alias and creates a data stream with the same name.
The indices for the alias become hidden backing indices for the stream.
The write index for the alias becomes the write index for the stream.
Name of the index alias to convert to a data stream.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_data_stream/_migrate/{name}' \
--header "Authorization: $API_KEY"Performs one or more data stream modification actions in a single atomic operation.
curl \
--request POST 'http://api.example.com/_data_stream/_modify' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"actions":[{"add_backing_index":{"data_stream":"string","index":"string"},"remove_backing_index":{"data_stream":"string","index":"string"}}]}'Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request PUT 'http://api.example.com/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request POST 'http://api.example.com/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
The name of the data stream, index, or index alias to perform bulk actions on.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request PUT 'http://api.example.com/{index}/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}Perform multiple index, create, delete, and update actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.index action, you must have the create, index, or write index privilege.delete action, you must have the delete or write index privilege.update action, you must have the index or write index privilege.auto_configure, create_index, or manage index privilege.refresh parameter, you must have the maintenance or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API.
A create action fails if a document with the same ID already exists in the target
An index action adds or replaces a document as necessary.
NOTE: Data streams support only the create action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n).
Each newline character may be preceded by a carriage return (\r).
When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson.
Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
esutil.BulkIndexerSearch::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scrollelasticsearch.helpers.*client.helpers.*BulkAllObservableSubmitting bulk requests with cURL
If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines.
The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version field.
It automatically follows the behavior of the index or delete operation based on the _version mapping.
It also support the version_type.
Routing
Each bulk item can include the routing value using the routing field.
It automatically follows the behavior of the index or delete operation based on the _routing mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk request at all.
The name of the data stream, index, or index alias to perform bulk actions on.
True or false if to include the document source in the error message in case of parsing errors.
If true, the response will include the ingest pipelines that were run for each index or create.
The pipeline identifier to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, wait for a refresh to make this operation visible to search.
If false, do nothing with refreshes.
Valid values: true, false, wait_for.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or contains a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards.
The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing.
The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default is 1, which waits for each primary shard to be active.
If true, the request's actions must target an index alias.
If true, the request's actions must target a data stream (existing or to be created).
curl \
--request POST 'http://api.example.com/{index}/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs
Using _create guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/ API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Supported values include:
index: Overwrite any documents that already exist.create: Only index documents that do not already exist.Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
If true, the request's actions must target a data stream (existing or to be created).
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
The explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
curl \
--request PUT 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs
Using _create guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/ API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Supported values include:
index: Overwrite any documents that already exist.create: Only index documents that do not already exist.Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
If true, the request's actions must target a data stream (existing or to be created).
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
The explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
curl \
--request POST 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}Get a document and its source or stored fields from an index.
By default, this API is realtime and is not affected by the refresh rate of the index (when data will become visible for search).
In the case where stored fields are requested with the stored_fields parameter and the document has been updated but is not yet refreshed, the API will have to parse and analyze the source to extract the stored fields.
To turn off realtime behavior, set the realtime parameter to false.
Source filtering
By default, the API returns the contents of the _source field unless you have used the stored_fields parameter or the _source field is turned off.
You can turn off _source retrieval by using the _source parameter:
GET my-index-000001/_doc/0?_source=false
If you only need one or two fields from the _source, use the _source_includes or _source_excludes parameters to include or filter out particular fields.
This can be helpful with large documents where partial retrieval can save on network overhead
Both parameters take a comma separated list of fields or wildcard expressions.
For example:
GET my-index-000001/_doc/0?_source_includes=*.id&_source_excludes=entities
If you only want to specify includes, you can use a shorter notation:
GET my-index-000001/_doc/0?_source=*.id
Routing
If routing is used during indexing, the routing value also needs to be specified to retrieve a document. For example:
GET my-index-000001/_doc/2?routing=user1
This request gets the document with ID 2, but it is routed based on the user. The document is not fetched if the correct routing is not specified.
Distributed
The GET operation is hashed into a specific shard ID. It is then redirected to one of the replicas within that shard ID and returns the result. The replicas are the primary shard and its replicas within that shard ID group. This means that the more replicas you have, the better your GET scaling will be.
Versioning support
You can use the version parameter to retrieve the document only if its current version is equal to the specified one.
Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If it is set to _local, the operation will prefer to be run on a local allocated shard when possible.
If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value.
This can help with "jumping values" when hitting different shards in different refresh states.
A sample value can be something like the web session ID or the user name.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of stored fields to return as part of a hit.
If no fields are specified, no stored fields are included in the response.
If this field is specified, the _source parameter defaults to false.
Only leaf fields can be retrieved with the stored_field option.
Object fields can't be returned;if specified, the request fails.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY"{
"_index": "my-index-000001",
"_id": "0",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"found": true,
"_source": {
"@timestamp": "2099-11-15T14:12:12",
"http": {
"request": {
"method": "get"
},
"response": {
"status_code": 200,
"bytes": 1070000
},
"version": "1.1"
},
"source": {
"ip": "127.0.0.1"
},
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no" : 22,
"_primary_term" : 1,
"found": true,
"fields": {
"tags": [
"production"
]
}
}{
"_index": "my-index-000001",
"_id": "2",
"_version": 1,
"_seq_no" : 13,
"_primary_term" : 1,
"_routing": "user1",
"found": true,
"fields": {
"tags": [
"env2"
]
}
}Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters.
If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update API with detect_noop set to true.
The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type should be set to external.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn't match a data stream template, this request creates the index.
You can check for existing targets with the resolve index API.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Supported values include:
index: Overwrite any documents that already exist.create: Only index documents that do not already exist.Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
An explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
If true, the destination must be an index alias.
curl \
--request PUT 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters.
If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update API with detect_noop set to true.
The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type should be set to external.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn't match a data stream template, this request creates the index.
You can check for existing targets with the resolve index API.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Supported values include:
index: Overwrite any documents that already exist.create: Only index documents that do not already exist.Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
An explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
If true, the destination must be an index alias.
curl \
--request POST 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}Remove a JSON document from the specified index.
NOTE: You cannot send deletion requests directly to a data stream. To delete a document in a data stream, you must target the backing index containing the document.
Optimistic concurrency control
Delete operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters.
If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.
Versioning
Each document indexed is versioned.
When deleting a document, the version can be specified to make sure the relevant document you are trying to delete is actually being deleted and it has not changed in the meantime.
Every write operation run on a document, deletes included, causes its version to be incremented.
The version number of a deleted document remains available for a short time after deletion to allow for control of concurrent operations.
The length of time for which a deleted document's version remains available is determined by the index.gc_deletes index setting.
Routing
If routing is used during indexing, the routing value also needs to be specified to delete a document.
If the _routing mapping is set to required and no routing value is specified, the delete API throws a RoutingMissingException and rejects the request.
For example:
DELETE /my-index-000001/_doc/1?routing=shard-1
This request deletes the document with ID 1, but it is routed based on the user. The document is not deleted if the correct routing is not specified.
Distributed
The delete operation gets hashed into a specific shard ID. It then gets redirected into the primary shard within that ID group and replicated (if needed) to shard replicas within that ID group.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
A custom value used to route operations to a specific shard.
The period to wait for active shards.
This parameter is useful for situations where the primary shard assigned to perform the delete operation might not be available when the delete operation runs. Some reasons for this might be that the primary shard is currently recovering from a store or undergoing relocation. By default, the delete operation will wait on the primary shard to become available for up to 1 minute before failing and responding with an error.
An explicit version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The minimum number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
curl \
--request DELETE 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY"{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 2,
"_primary_term": 1,
"_seq_no": 5,
"result": "deleted"
}Verify that a document exists.
For example, check to see if a document with the _id 0 exists:
HEAD my-index-000001/_doc/0
If the document exists, the API returns a status code of 200 - OK.
If the document doesn’t exist, the API returns 404 - Not Found.
Versioning support
You can use the version parameter to check the document only if its current version is equal to the specified one.
Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If it is set to _local, the operation will prefer to be run on a local allocated shard when possible.
If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value.
This can help with "jumping values" when hitting different shards in different refresh states.
A sample value can be something like the web session ID or the user name.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of stored fields to return as part of a hit.
If no fields are specified, no stored fields are included in the response.
If this field is specified, the _source parameter defaults to false.
Explicit version number for concurrency control. The specified version must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
HEAD my-index-000001/_doc/0
curl -I "localhost:9200/my-index-000001/_doc/0?pretty"
const response = await client.exists({
index: "my-index-000001",
id: 0,
});
console.log(response);
resp = client.exists(
index="my-index-000001",
id="0",
)
print(resp)
response = client.exists(
index: 'my-index-000001',
id: 0
)
puts response
Deletes documents that match the specified query.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:
readdelete or writeYou can specify the query criteria in the request URI or the request body using the same syntax as the search API. When you submit a delete by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and deletes matching documents using internal versioning. If a document changes between the time that the snapshot is taken and the delete operation is processed, it results in a version conflict and the delete operation fails.
NOTE: Documents with a version equal to 0 cannot be deleted using delete by query because internal versioning does not support 0 as a valid version number.
While processing a delete by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents to delete. A bulk delete request is performed for each batch of matching documents. If a search or bulk request is rejected, the requests are retried up to 10 times, with exponential back off. If the maximum retry limit is reached, processing halts and all failed requests are returned in the response. Any delete requests that completed successfully still stick, they are not rolled back.
You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed.
Note that if you opt to count version conflicts the operation could attempt to delete more documents from the source than max_docs until it has successfully deleted max_docs documents, or it has gone through every document in the source query.
Throttling delete requests
To control the rate at which delete by query issues batches of delete operations, you can set requests_per_second to any positive decimal number.
This pads each batch with a wait time to throttle the rate.
Set requests_per_second to -1 to disable throttling.
Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set.
This is "bursty" instead of "smooth".
Slicing
Delete by query supports sliced scroll to parallelize the delete process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.
Setting slices to auto lets Elasticsearch choose the number of slices to use.
This setting will use one slice per shard, up to a certain limit.
If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.
Adding slices to the delete by query operation creates sub-requests which means it has some quirks:
slices will rethrottle the unfinished sub-request proportionally.slices will cancel each sub-request.slices each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the earlier point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being deleted.If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:
slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.Whether query or delete performance dominates the runtime depends on the documents being reindexed and cluster resources.
Cancel a delete by query operation
Any delete by query can be canceled using the task cancel API. For example:
POST _tasks/r1A2WoRbTwKZ516z6NEs5A:36619/_cancel
The task ID can be found by using the get tasks API.
Cancellation should happen quickly but might take a few seconds. The get task status API will continue to list the delete by query task until this task checks that it has been cancelled and terminates itself.
A comma-separated list of data streams, indices, and aliases to search.
It supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
Analyzer to use for the query string.
This parameter can be used only when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
This parameter can be used only when the q query string parameter is specified.
What to do if delete by query hits version conflicts: abort or proceed.
Supported values include:
abort: Stop reindexing if there are conflicts.proceed: Continue reindexing even if there are conflicts.Values are abort or proceed.
The default operator for query string query: AND or OR.
This parameter can be used only when the q query string parameter is specified.
Values are and, AND, or, or OR.
The field to use as default where no field prefix is given in the query string.
This parameter can be used only when the q query string parameter is specified.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Skips the specified number of documents.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
This parameter can be used only when the q query string parameter is specified.
The maximum number of documents to process.
Defaults to all documents.
When set to a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.
The node or shard the operation should be performed on. It is random by default.
If true, Elasticsearch refreshes all shards involved in the delete by query after the request completes.
This is different than the delete API's refresh parameter, which causes just the shard that received the delete request to be refreshed.
Unlike the delete API, it does not support wait_for.
If true, the request cache is used for this request.
Defaults to the index-level setting.
The throttle for this request in sub-requests per second.
A custom value used to route operations to a specific shard.
A query in the Lucene query string syntax.
The period to retain the search context for scrolling.
The size of the scroll request that powers the operation.
The explicit timeout for each search request. It defaults to no timeout.
The type of the search operation.
Available options include query_then_fetch and dfs_query_then_fetch.
Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.Values are query_then_fetch or dfs_query_then_fetch.
The number of slices this task should be divided into.
A comma-separated list of <field>:<direction> pairs.
The specific tag of the request for logging and statistical purposes.
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
The period each deletion request waits for active shards.
If true, returns the document version as part of a hit.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The timeout value controls how long each write request waits for unavailable shards to become available.
If true, the request blocks until the operation is complete.
If false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task. Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}. When you are done with a task, you should delete the task document so Elasticsearch can reclaim the space.
The maximum number of documents to delete.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/{index}/_delete_by_query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": {\n \"match_all\": {}\n }\n}"'{
"query": {
"match_all": {}
}
}{
"query": {
"term": {
"user.id": "kimchy"
}
},
"max_docs": 1
}{
"slice": {
"id": 0,
"max": 2
},
"query": {
"range": {
"http.response.bytes": {
"lt": 2000000
}
}
}
}{
"query": {
"range": {
"http.response.bytes": {
"lt": 2000000
}
}
}
}{
"took" : 147,
"timed_out": false,
"total": 119,
"deleted": 119,
"batches": 1,
"version_conflicts": 0,
"noops": 0,
"retries": {
"bulk": 0,
"search": 0
},
"throttled_millis": 0,
"requests_per_second": -1.0,
"throttled_until_millis": 0,
"failures" : [ ]
}Get the source of a document. For example:
GET my-index-000001/_source/1
You can use the source filtering parameters to control which parts of the _source are returned:
GET my-index-000001/_source/1/?_source_includes=*.id&_source_excludes=entities
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude in the response.
A comma-separated list of source fields to include in the response.
A comma-separated list of stored fields to return as part of a hit.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_source/{id}' \
--header "Authorization: $API_KEY"Check whether a document source exists in an index. For example:
HEAD my-index-000001/_source/1
A document's source is not available if it is disabled in the mapping.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude in the response.
A comma-separated list of source fields to include in the response.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request HEAD 'http://api.example.com/{index}/_source/{id}' \
--header "Authorization: $API_KEY"Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters.
If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update API with detect_noop set to true.
The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type should be set to external.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn't match a data stream template, this request creates the index.
You can check for existing targets with the resolve index API.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Supported values include:
index: Overwrite any documents that already exist.create: Only index documents that do not already exist.Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
An explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
If true, the destination must be an index alias.
curl \
--request POST 'http://api.example.com/{index}/_doc' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source field is returned for every document (if stored).
Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document.
You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Specifies the node or shard the operation should be performed on. Random by default.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes relevant shards before retrieving documents.
Custom value used to route operations to a specific shard.
True or false to return the _source field or not, or a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
If true, retrieves the document fields stored in the index rather than the document _source.
curl \
--request GET 'http://api.example.com/_mget' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source field is returned for every document (if stored).
Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document.
You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Specifies the node or shard the operation should be performed on. Random by default.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes relevant shards before retrieving documents.
Custom value used to route operations to a specific shard.
True or false to return the _source field or not, or a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
If true, retrieves the document fields stored in the index rather than the document _source.
curl \
--request POST 'http://api.example.com/_mget' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source field is returned for every document (if stored).
Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document.
You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Name of the index to retrieve documents from when ids are specified, or when a document in the docs array does not specify an index.
Specifies the node or shard the operation should be performed on. Random by default.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes relevant shards before retrieving documents.
Custom value used to route operations to a specific shard.
True or false to return the _source field or not, or a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
If true, retrieves the document fields stored in the index rather than the document _source.
curl \
--request GET 'http://api.example.com/{index}/_mget' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source field is returned for every document (if stored).
Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document.
You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Name of the index to retrieve documents from when ids are specified, or when a document in the docs array does not specify an index.
Specifies the node or shard the operation should be performed on. Random by default.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes relevant shards before retrieving documents.
Custom value used to route operations to a specific shard.
True or false to return the _source field or not, or a list of fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
If true, retrieves the document fields stored in the index rather than the document _source.
curl \
--request POST 'http://api.example.com/{index}/_mget' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index.
A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value used to route operations to a specific shard.
If true, the response includes term frequency and document frequency.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/_mtermvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}{
"ids": [ "1", "2" ],
"fields": [
"message"
],
"term_statistics": true
}{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index.
A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value used to route operations to a specific shard.
If true, the response includes term frequency and document frequency.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request POST 'http://api.example.com/_mtermvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}{
"ids": [ "1", "2" ],
"fields": [
"message"
],
"term_statistics": true
}{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index.
The name of the index that contains the documents.
A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value used to route operations to a specific shard.
If true, the response includes term frequency and document frequency.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_mtermvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}{
"ids": [ "1", "2" ],
"fields": [
"message"
],
"term_statistics": true
}{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index.
The name of the index that contains the documents.
A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value used to route operations to a specific shard.
If true, the response includes term frequency and document frequency.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
curl \
--request POST 'http://api.example.com/{index}/_mtermvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}{
"ids": [ "1", "2" ],
"fields": [
"message"
],
"term_statistics": true
}{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}Copy documents from a source to a destination. You can copy all documents to the destination index or reindex a subset of the documents. The source can be any existing index, alias, or data stream. The destination must differ from the source. For example, you cannot reindex a data stream into itself.
IMPORTANT: Reindex requires _source to be enabled for all documents in the source.
The destination should be configured as wanted before calling the reindex API.
Reindex does not copy the settings from the source or its associated template.
Mappings, shard counts, and replicas, for example, must be configured ahead of time.
If the Elasticsearch security features are enabled, you must have the following security privileges:
read index privilege for the source data stream, index, or alias.write index privilege for the destination data stream, index, or index alias.auto_configure, create_index, or manage index privilege for the destination data stream, index, or alias.source.remote.user must have the monitor cluster privilege and the read index privilege for the source data stream, index, or alias.If reindexing from a remote cluster, you must explicitly allow the remote host in the reindex.remote.whitelist setting.
Automatic data stream creation requires a matching index template with data stream enabled.
The dest element can be configured like the index API to control optimistic concurrency control.
Omitting version_type or setting it to internal causes Elasticsearch to blindly dump documents into the destination, overwriting any that happen to have the same ID.
Setting version_type to external causes Elasticsearch to preserve the version from the source, create any documents that are missing, and update any documents that have an older version in the destination than they do in the source.
Setting op_type to create causes the reindex API to create only missing documents in the destination.
All existing documents will cause a version conflict.
IMPORTANT: Because data streams are append-only, any reindex request to a destination data stream must have an op_type of create.
A reindex can only add new documents to a destination data stream.
It cannot update existing documents in a destination data stream.
By default, version conflicts abort the reindex process.
To continue reindexing if there are conflicts, set the conflicts request body property to proceed.
In this case, the response includes a count of the version conflicts that were encountered.
Note that the handling of other error types is unaffected by the conflicts property.
Additionally, if you opt to count version conflicts, the operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.
NOTE: The reindex API makes no effort to handle ID collisions. The last document written will "win" but the order isn't usually predictable so it is not a good idea to rely on this behavior. Instead, make sure that IDs are unique by using a script.
Running reindex asynchronously
If the request contains wait_for_completion=false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task.
Elasticsearch creates a record of this task as a document at _tasks/<task_id>.
Reindex from multiple sources
If you have many sources to reindex it is generally better to reindex them one at a time rather than using a glob pattern to pick up multiple sources. That way you can resume the process if there are any errors by removing the partially completed source and starting over. It also makes parallelizing the process fairly simple: split the list of sources to reindex and run each list in parallel.
For example, you can use a bash script like this:
for index in i1 i2 i3 i4 i5; do
curl -HContent-Type:application/json -XPOST localhost:9200/_reindex?pretty -d'{
"source": {
"index": "'$index'"
},
"dest": {
"index": "'$index'-reindexed"
}
}'
done
Throttling
Set requests_per_second to any positive decimal number (1.4, 6, 1000, for example) to throttle the rate at which reindex issues batches of index operations.
Requests are throttled by padding each batch with a wait time.
To turn off throttling, set requests_per_second to -1.
The throttling is done by waiting between batches so that the scroll that reindex uses internally can be given a timeout that takes into account the padding.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single bulk request, large batch sizes cause Elasticsearch to create many requests and then wait for a while before starting the next set. This is "bursty" instead of "smooth".
Slicing
Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
You can slice a reindex request manually by providing a slice ID and total number of slices to each request.
You can also let reindex automatically parallelize by using sliced scroll to slice on _id.
The slices parameter specifies the number of slices to use.
Adding slices to the reindex request just automates the manual process, creating sub-requests which means it has some quirks:
slices only contains the status of completed slices.slices will rethrottle the unfinished sub-request proportionally.slices will cancel each sub-request.slices, each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the previous point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being reindexed.If slicing automatically, setting slices to auto will choose a reasonable number for most indices.
If slicing manually or otherwise tuning automatic slicing, use the following guidelines.
Query performance is most efficient when the number of slices is equal to the number of shards in the index.
If that number is large (for example, 500), choose a lower number as too many slices will hurt performance.
Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
Indexing performance scales linearly across available resources with the number of slices.
Whether query or indexing performance dominates the runtime depends on the documents being reindexed and cluster resources.
Modify documents during reindexing
Like _update_by_query, reindex operations support a script that modifies the document.
Unlike _update_by_query, the script is allowed to modify the document's metadata.
Just as in _update_by_query, you can set ctx.op to change the operation that is run on the destination.
For example, set ctx.op to noop if your script decides that the document doesn’t have to be indexed in the destination. This "no operation" will be reported in the noop counter in the response body.
Set ctx.op to delete if your script decides that the document must be deleted from the destination.
The deletion will be reported in the deleted counter in the response body.
Setting ctx.op to anything else will return an error, as will setting any other field in ctx.
Think of the possibilities! Just be careful; you are able to change:
_id_index_version_routingSetting _version to null or clearing it from the ctx map is just like not sending the version in an indexing request.
It will cause the document to be overwritten in the destination regardless of the version on the target or the version type you use in the reindex API.
Reindex from remote
Reindex supports reindexing from a remote Elasticsearch cluster.
The host parameter must contain a scheme, host, port, and optional path.
The username and password parameters are optional and when they are present the reindex operation will connect to the remote Elasticsearch node using basic authentication.
Be sure to use HTTPS when using basic authentication or the password will be sent in plain text.
There are a range of settings available to configure the behavior of the HTTPS connection.
When using Elastic Cloud, it is also possible to authenticate against the remote cluster through the use of a valid API key.
Remote hosts must be explicitly allowed with the reindex.remote.whitelist setting.
It can be set to a comma delimited list of allowed remote host and port combinations.
Scheme is ignored; only the host and port are used.
For example:
reindex.remote.whitelist: [otherhost:9200, another:9200, 127.0.10.*:9200, localhost:*"]
The list of allowed hosts must be configured on any nodes that will coordinate the reindex. This feature should work with remote clusters of any version of Elasticsearch. This should enable you to upgrade from any version of Elasticsearch to the current version by reindexing from a cluster of the old version.
WARNING: Elasticsearch does not support forward compatibility across major versions. For example, you cannot reindex from a 7.x cluster into a 6.x cluster.
To enable queries sent to older versions of Elasticsearch, the query parameter is sent directly to the remote host without validation or modification.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
Reindexing from a remote server uses an on-heap buffer that defaults to a maximum size of 100mb.
If the remote index includes very large documents you'll need to use a smaller batch size.
It is also possible to set the socket read timeout on the remote connection with the socket_timeout field and the connection timeout with the connect_timeout field.
Both default to 30 seconds.
Configuring SSL parameters
Reindex from remote supports configurable SSL settings.
These must be specified in the elasticsearch.yml file, with the exception of the secure settings, which you add in the Elasticsearch keystore.
It is not possible to configure SSL in the body of the reindex request.
If true, the request refreshes affected shards to make this operation visible to search.
The throttle for this request in sub-requests per second. By default, there is no throttle.
The period of time that a consistent view of the index should be maintained for scrolled search.
The number of slices this task should be divided into. It defaults to one slice, which means the task isn't sliced into subtasks.
Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
If set to auto, Elasticsearch chooses the number of slices to use.
This setting will use one slice per shard, up to a certain limit.
If there are multiple sources, it will choose the number of slices based on the index or backing index with the smallest number of shards.
The period each indexing waits for automatic index creation, dynamic mapping updates, and waiting for active shards. By default, Elasticsearch waits for at least one minute before failing. The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value is one, which means it waits for each primary shard to be active.
If true, the request blocks until the operation is complete.
If true, the destination must be an index alias.
Values are abort or proceed.
The maximum number of documents to reindex.
By default, all documents are reindexed.
If it is a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.
If conflicts is set to proceed, the reindex operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.
curl \
--request POST 'http://api.example.com/_reindex' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"source\": {\n \"index\": [\"my-index-000001\", \"my-index-000002\"]\n },\n \"dest\": {\n \"index\": \"my-new-index-000002\"\n }\n}"'{
"source": {
"index": ["my-index-000001", "my-index-000002"]
},
"dest": {
"index": "my-new-index-000002"
}
}{
"source": {
"index": "metricbeat-*"
},
"dest": {
"index": "metricbeat"
},
"script": {
"lang": "painless",
"source": "ctx._index = 'metricbeat-' + (ctx._index.substring('metricbeat-'.length(), ctx._index.length())) + '-1'"
}
}{
"max_docs": 10,
"source": {
"index": "my-index-000001",
"query": {
"function_score" : {
"random_score" : {},
"min_score" : 0.9
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001",
"version_type": "external"
},
"script": {
"source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
"lang": "painless"
}
}{
"source": {
"remote": {
"host": "http://otherhost:9200",
"username": "user",
"password": "pass"
},
"index": "my-index-000001",
"query": {
"match": {
"test": "data"
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001",
"slice": {
"id": 0,
"max": 2
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "source",
"query": {
"match": {
"company": "cat"
}
}
},
"dest": {
"index": "dest",
"routing": "=cat"
}
}{
"source": {
"index": "source"
},
"dest": {
"index": "dest",
"pipeline": "some_ingest_pipeline"
}
}{
"source": {
"index": "my-index-000001",
"query": {
"term": {
"user.id": "kimchy"
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"max_docs": 1,
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001",
"_source": ["user.id", "_doc"]
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
},
"script": {
"source": "ctx._source.tag = ctx._source.remove(\"flag\")"
}
}Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime parameter to false.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
positions: true)offsets: true)payloads: true), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing only to hit a particular shard.
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value that is used to route operations to a specific shard.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_termvectors/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime parameter to false.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
positions: true)offsets: true)payloads: true), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing only to hit a particular shard.
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value that is used to route operations to a specific shard.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
Values are internal, external, external_gte, or force.
curl \
--request POST 'http://api.example.com/{index}/_termvectors/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime parameter to false.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
positions: true)offsets: true)payloads: true), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing only to hit a particular shard.
The name of the index that contains the document.
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value that is used to route operations to a specific shard.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_termvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime parameter to false.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
positions: true)offsets: true)payloads: true), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing only to hit a particular shard.
The name of the index that contains the document.
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value that is used to route operations to a specific shard.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
If true, returns the document version as part of a hit.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
Values are internal, external, external_gte, or force.
curl \
--request POST 'http://api.example.com/{index}/_termvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}Update a document by running a script or passing a partial document.
If the Elasticsearch security features are enabled, you must have the index or write index privilege for the target index or index alias.
The script can update, delete, or skip modifying the document. The API also supports passing a partial document, which is merged into the existing document. To fully replace an existing document, use the index API. This operation:
The document must still be reindexed, but using this API removes some network roundtrips and reduces chances of version conflicts between the GET and the index operation.
The _source field must be enabled to use this API.
In addition to _source, you can access the following variables through the ctx map: _index, _type, _id, _version, _routing, and _now (the current timestamp).
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
The script language.
If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
The number of times the operation should be retried when a conflict occurs.
A custom value used to route operations to a specific shard.
The period to wait for the following operations: dynamic mapping updates and waiting for active shards. Elasticsearch waits for at least the timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
The number of copies of each shard that must be active before proceeding with the operation.
Set to 'all' or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
If false, source retrieval is turned off.
You can also specify a comma-separated list of the fields you want to retrieve.
The source fields you want to exclude.
The source fields you want to retrieve.
If true, the result in the response is set to noop (no operation) when there are no changes to the document.
A partial update to an existing document.
If both doc and script are specified, doc is ignored.
If true, use the contents of 'doc' as the value of 'upsert'.
NOTE: Using ingest pipelines with doc_as_upsert is not supported.
If true, run the script whether or not the document exists.
Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
If the document does not already exist, the contents of 'upsert' are inserted as a new document. If the document exists, the 'script' is run.
curl \
--request POST 'http://api.example.com/{index}/_update/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\" : {\n \"source\": \"ctx._source.counter += params.count\",\n \"lang\": \"painless\",\n \"params\" : {\n \"count\" : 4\n }\n }\n}"'{
"script" : {
"source": "ctx._source.counter += params.count",
"lang": "painless",
"params" : {
"count" : 4
}
}
}{
"scripted_upsert": true,
"script": {
"source": """
if ( ctx.op == 'create' ) {
ctx._source.counter = params.count
} else {
ctx._source.counter += params.count
}
""",
"params": {
"count": 4
}
},
"upsert": {}
}{
"doc": {
"name": "new_name"
},
"doc_as_upsert": true
}{
"script": {
"source": "ctx._source.tags.add(params.tag)",
"lang": "painless",
"params": {
"tag": "blue"
}
}
}{
"script": {
"source": "if (ctx._source.tags.contains(params.tag)) { ctx._source.tags.remove(ctx._source.tags.indexOf(params.tag)) }",
"lang": "painless",
"params": {
"tag": "blue"
}
}
}{
"script" : "ctx._source.new_field = 'value_of_new_field'"
}{
"script" : "ctx._source.remove('new_field')"
}{
"script": "ctx._source['my-object'].remove('my-subfield')"
}{
"script": {
"source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'noop' }",
"lang": "painless",
"params": {
"tag": "green"
}
}
}{
"doc": {
"name": "new_name"
}
}{
"script": {
"source": "ctx._source.counter += params.count",
"lang": "painless",
"params": {
"count": 4
}
},
"upsert": {
"counter": 1
}
}{
"_shards": {
"total": 0,
"successful": 0,
"failed": 0
},
"_index": "test",
"_id": "1",
"_version": 2,
"_primary_term": 1,
"_seq_no": 1,
"result": "noop"
}Updates documents that match the specified query. If no query is specified, performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:
readindex or writeYou can specify the query criteria in the request URI or the request body using the same syntax as the search API.
When you submit an update by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and updates matching documents using internal versioning.
When the versions match, the document is updated and the version number is incremented.
If a document changes between the time that the snapshot is taken and the update operation is processed, it results in a version conflict and the operation fails.
You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed.
Note that if you opt to count version conflicts, the operation could attempt to update more documents from the source than max_docs until it has successfully updated max_docs documents or it has gone through every document in the source query.
NOTE: Documents with a version equal to 0 cannot be updated using update by query because internal versioning does not support 0 as a valid version number.
While processing an update by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents. A bulk update request is performed for each batch of matching documents. Any query or update failures cause the update by query request to fail and the failures are shown in the response. Any update requests that completed successfully still stick, they are not rolled back.
Throttling update requests
To control the rate at which update by query issues batches of update operations, you can set requests_per_second to any positive decimal number.
This pads each batch with a wait time to throttle the rate.
Set requests_per_second to -1 to turn off throttling.
Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".
Slicing
Update by query supports sliced scroll to parallelize the update process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.
Setting slices to auto chooses a reasonable number for most data streams and indices.
This setting will use one slice per shard, up to a certain limit.
If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.
Adding slices to _update_by_query just automates the manual process of creating sub-requests, which means it has some quirks:
slices only contains the status of completed slices.slices will rethrottle the unfinished sub-request proportionally.requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the point above about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being updated.If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:
Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
Update the document source
Update by query supports scripts to update the document source.
As with the update API, you can set ctx.op to change the operation that is performed.
Set ctx.op = "noop" if your script decides that it doesn't have to make any changes.
The update by query operation skips updating the document and increments the noop counter.
Set ctx.op = "delete" if your script decides that the document should be deleted.
The update by query operation deletes the document and increments the deleted counter.
Update by query supports only index, noop, and delete.
Setting ctx.op to anything else is an error.
Setting any other field in ctx is an error.
This API enables you to only modify the source of matching documents; you cannot move them.
A comma-separated list of data streams, indices, and aliases to search.
It supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
The analyzer to use for the query string.
This parameter can be used only when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
This parameter can be used only when the q query string parameter is specified.
The preferred behavior when update by query hits version conflicts: abort or proceed.
Supported values include:
abort: Stop reindexing if there are conflicts.proceed: Continue reindexing even if there are conflicts.Values are abort or proceed.
The default operator for query string query: AND or OR.
This parameter can be used only when the q query string parameter is specified.
Values are and, AND, or, or OR.
The field to use as default where no field prefix is given in the query string.
This parameter can be used only when the q query string parameter is specified.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Skips the specified number of documents.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
This parameter can be used only when the q query string parameter is specified.
The maximum number of documents to process.
It defaults to all documents.
When set to a value less then or equal to scroll_size then a scroll will not be used to retrieve the results for the operation.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
The node or shard the operation should be performed on. It is random by default.
A query in the Lucene query string syntax.
If true, Elasticsearch refreshes affected shards to make the operation visible to search after the request completes.
This is different than the update API's refresh parameter, which causes just the shard that received the request to be refreshed.
If true, the request cache is used for this request.
It defaults to the index-level setting.
The throttle for this request in sub-requests per second.
A custom value used to route operations to a specific shard.
The period to retain the search context for scrolling.
The size of the scroll request that powers the operation.
An explicit timeout for each search request. By default, there is no timeout.
The type of the search operation. Available options include query_then_fetch and dfs_query_then_fetch.
Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.Values are query_then_fetch or dfs_query_then_fetch.
The number of slices this task should be divided into.
A comma-separated list of : pairs.
The specific tag of the request for logging and statistical purposes.
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
The period each update request waits for the following operations: dynamic mapping updates, waiting for active shards. By default, it is one minute. This guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.
If true, returns the document version as part of a hit.
Should the document increment the version number (internal) on hit or not (reindex)
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The timeout parameter controls how long each write request waits for unavailable shards to become available.
Both work exactly the way they work in the bulk API.
If true, the request blocks until the operation is complete.
If false, Elasticsearch performs some preflight checks, launches the request, and returns a task ID that you can use to cancel or get the status of the task.
Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}.
curl \
--request POST 'http://api.example.com/{index}/_update_by_query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": { \n \"term\": {\n \"user.id\": \"kimchy\"\n }\n }\n}"'{
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"script": {
"source": "ctx._source.count++",
"lang": "painless"
},
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"slice": {
"id": 0,
"max": 2
},
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}{
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}Returns information about an enrich policy.
Comma-separated list of enrich policy names used to limit the request. To return information for all enrich policies, omit this parameter.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_enrich/policy/{name}' \
--header "Authorization: $API_KEY"Name of the enrich policy to create or update.
Period to wait for a connection to the master node.
curl \
--request PUT 'http://api.example.com/_enrich/policy/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"additionalProperty1":{"enrich_fields":"string","indices":"string","match_field":"string","query":{},"name":"string","elasticsearch_version":"string"},"additionalProperty2":{"enrich_fields":"string","indices":"string","match_field":"string","query":{},"name":"string","elasticsearch_version":"string"}}'Deletes an existing enrich policy and its enrich index.
Enrich policy to delete.
Period to wait for a connection to the master node.
curl \
--request DELETE 'http://api.example.com/_enrich/policy/{name}' \
--header "Authorization: $API_KEY"Create the enrich index for an existing enrich policy.
Enrich policy to execute.
Period to wait for a connection to the master node.
If true, the request blocks other enrich policy execution requests until complete.
curl \
--request PUT 'http://api.example.com/_enrich/policy/{name}/_execute' \
--header "Authorization: $API_KEY"Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_enrich/policy' \
--header "Authorization: $API_KEY"Event Query Language (EQL) is a query language for event-based time series data, such as logs, metrics, and traces.
Get the current status and available results for an async EQL search or a stored synchronous EQL search.
Identifier for the search.
Period for which the search and its results are stored on the cluster. Defaults to the keep_alive value set by the search’s EQL search API request.
Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
curl \
--request GET 'http://api.example.com/_eql/search/{id}' \
--header "Authorization: $API_KEY"Delete an async EQL search or a stored synchronous EQL search. The API also deletes results for the search.
Identifier for the search to delete.
A search ID is provided in the EQL search API's response for an async search.
A search ID is also provided if the request’s keep_on_completion parameter is true.
curl \
--request DELETE 'http://api.example.com/_eql/search/{id}' \
--header "Authorization: $API_KEY"Get the current status for an async EQL search or a stored synchronous EQL search without returning results.
Identifier for the search.
curl \
--request GET 'http://api.example.com/_eql/search/status/{id}' \
--header "Authorization: $API_KEY"{
"id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
"is_running" : true,
"is_partial" : true,
"start_time_in_millis" : 1611690235000,
"expiration_time_in_millis" : 1611690295000
}Returns search results for an Event Query Language (EQL) query. EQL assumes each document in a data stream or index corresponds to an event.
The name of the index to scope the operation
If true, returns partial results if there are shard failures. If false, returns an error with no partial results.
If true, sequence queries will return partial results in case of shard failures. If false, they will return no results at all. This flag has effect only if allow_partial_search_results is true.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period for which the search and its results are stored on the cluster.
If true, the search and its results are stored on the cluster.
Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
EQL query you wish to run.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Query, written in Query DSL, used to filter the events on which the EQL query runs.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Allow query execution also in case of shard failures. If true, the query will keep running and will return results based on the available shards. For sequences, the behavior can be further refined using allow_partial_sequence_results
This flag applies only to sequences and has effect only if allow_partial_search_results=true. If true, the sequence query will return results based on the available shards, ignoring the others. If false, the sequence query will return successfully, but will always have empty results.
Array of wildcard (*) patterns. The response returns values for field names matching these patterns in the fields property of each hit.
Values are tail or head.
By default, the response of a sample query contains up to 10 samples, with one sample per unique set of join keys. Use the size
parameter to get a smaller or larger set of samples. To retrieve more than one sample per set of join keys, use the
max_samples_per_key parameter. Pipes are not supported for sample queries.
curl \
--request GET 'http://api.example.com/{index}/_eql/search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": \"\"\"\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n \"\"\"\n}"'{
"query": """
process where (process.name == "cmd.exe" and process.pid != 2013)
"""
}{
"query": """
sequence by process.pid
[ file where file.name == "cmd.exe" and process.pid != 2013 ]
[ process where stringContains(process.executable, "regsvr32") ]
"""
}{
"is_partial": false,
"is_running": false,
"took": 6,
"timed_out": false,
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"sequences": [
{
"join_keys": [
2012
],
"events": [
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "AtOJ4UjUBAAx3XR5kcCM",
"_source": {
"@timestamp": "2099-12-06T11:04:07.000Z",
"event": {
"category": "file",
"id": "dGCHwoeS",
"sequence": 2
},
"file": {
"accessed": "2099-12-07T11:07:08.000Z",
"name": "cmd.exe",
"path": "C:\\Windows\\System32\\cmd.exe",
"type": "file",
"size": 16384
},
"process": {
"pid": 2012,
"name": "cmd.exe",
"executable": "C:\\Windows\\System32\\cmd.exe"
}
}
},
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "OQmfCaduce8zoHT93o4H",
"_source": {
"@timestamp": "2099-12-07T11:07:09.000Z",
"event": {
"category": "process",
"id": "aR3NWVOs",
"sequence": 4
},
"process": {
"pid": 2012,
"name": "regsvr32.exe",
"command_line": "regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll",
"executable": "C:\\Windows\\System32\\regsvr32.exe"
}
}
}
]
}
]
}
}Returns search results for an Event Query Language (EQL) query. EQL assumes each document in a data stream or index corresponds to an event.
The name of the index to scope the operation
If true, returns partial results if there are shard failures. If false, returns an error with no partial results.
If true, sequence queries will return partial results in case of shard failures. If false, they will return no results at all. This flag has effect only if allow_partial_search_results is true.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period for which the search and its results are stored on the cluster.
If true, the search and its results are stored on the cluster.
Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
EQL query you wish to run.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Query, written in Query DSL, used to filter the events on which the EQL query runs.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Allow query execution also in case of shard failures. If true, the query will keep running and will return results based on the available shards. For sequences, the behavior can be further refined using allow_partial_sequence_results
This flag applies only to sequences and has effect only if allow_partial_search_results=true. If true, the sequence query will return results based on the available shards, ignoring the others. If false, the sequence query will return successfully, but will always have empty results.
Array of wildcard (*) patterns. The response returns values for field names matching these patterns in the fields property of each hit.
Values are tail or head.
By default, the response of a sample query contains up to 10 samples, with one sample per unique set of join keys. Use the size
parameter to get a smaller or larger set of samples. To retrieve more than one sample per set of join keys, use the
max_samples_per_key parameter. Pipes are not supported for sample queries.
curl \
--request POST 'http://api.example.com/{index}/_eql/search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": \"\"\"\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n \"\"\"\n}"'{
"query": """
process where (process.name == "cmd.exe" and process.pid != 2013)
"""
}{
"query": """
sequence by process.pid
[ file where file.name == "cmd.exe" and process.pid != 2013 ]
[ process where stringContains(process.executable, "regsvr32") ]
"""
}{
"is_partial": false,
"is_running": false,
"took": 6,
"timed_out": false,
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"sequences": [
{
"join_keys": [
2012
],
"events": [
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "AtOJ4UjUBAAx3XR5kcCM",
"_source": {
"@timestamp": "2099-12-06T11:04:07.000Z",
"event": {
"category": "file",
"id": "dGCHwoeS",
"sequence": 2
},
"file": {
"accessed": "2099-12-07T11:07:08.000Z",
"name": "cmd.exe",
"path": "C:\\Windows\\System32\\cmd.exe",
"type": "file",
"size": 16384
},
"process": {
"pid": 2012,
"name": "cmd.exe",
"executable": "C:\\Windows\\System32\\cmd.exe"
}
}
},
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "OQmfCaduce8zoHT93o4H",
"_source": {
"@timestamp": "2099-12-07T11:07:09.000Z",
"event": {
"category": "process",
"id": "aR3NWVOs",
"sequence": 4
},
"process": {
"pid": 2012,
"name": "regsvr32.exe",
"command_line": "regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll",
"executable": "C:\\Windows\\System32\\regsvr32.exe"
}
}
}
]
}
]
}
}The Elasticsearch Query Language (ES|QL) provides a powerful way to filter, transform, and analyze data stored in Elasticsearch, and in the future in other runtimes.
Returns an object extended information about a running ES|QL query.
The query ID
curl \
--request GET 'http://api.example.com/_query/queries/{id}' \
--header "Authorization: $API_KEY"Returns an object containing IDs and other information about the running ES|QL queries.
curl \
--request GET 'http://api.example.com/_query/queries' \
--header "Authorization: $API_KEY"Get search results for an ES|QL (Elasticsearch query language) query.
A short version of the Accept header, e.g. json, yaml.
Values are csv, json, tsv, txt, yaml, cbor, smile, or arrow.
The character to use between values within a CSV row. Only valid for the CSV format.
Should columns that are entirely null be removed from the columns and values portion of the results?
Defaults to false. If true then the response will include an extra section under the name all_columns which has the name of all columns.
If true, partial results will be returned if there are shard failures, but the query can continue to execute on other clusters and shards.
If false, the query will fail if there are any failures.
To override the default behavior, you can set the esql.query.allow_partial_results cluster setting to false.
By default, ES|QL returns results as rows. For example, FROM returns each individual document as one row. For the JSON, YAML, CBOR and smile formats, ES|QL can return the results in a columnar fashion where one row represents all the values of a certain column in the results.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
To avoid any attempts of hacking or code injection, extract the values in a separate list of parameters. Use question mark placeholders (?) in the query string for each of the parameters.
If provided and true the response will include an extra profile object
with information on how the query was executed. This information is for human debugging
and its format can change at any time but it can give some insight into the performance
of each part of the query.
The ES|QL query API accepts an ES|QL query string in the query parameter, runs it, and returns the results.
Tables to use with the LOOKUP operation. The top level key is the table name and the next level key is the column name.
When set to true and performing a cross-cluster query, the response will include an extra _clusters
object with information about the clusters that participated in the search along with info such as shards
count.
curl \
--request POST 'http://api.example.com/_query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": \"\"\"\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n \"\"\",\n \"include_ccs_metadata\": true\n}"'{
"query": """
FROM library,remote-*:library
| EVAL year = DATE_TRUNC(1 YEARS, release_date)
| STATS MAX(page_count) BY year
| SORT year
| LIMIT 5
""",
"include_ccs_metadata": true
}The graph explore API enables you to extract and summarize information about the documents and terms in an Elasticsearch data stream or index.
Extract and summarize information about the documents and terms in an Elasticsearch data stream or index.
The easiest way to understand the behavior of this API is to use the Graph UI to explore connections.
An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph.
Subsequent requests enable you to spider out from one more vertices of interest.
You can exclude vertices that have already been returned.
Name of the index.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Specifies one or more fields that contain the terms you want to include in the graph as vertices.
curl \
--request GET 'http://api.example.com/{index}/_graph/explore' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": {\n \"match\": {\n \"query.raw\": \"midi\"\n }\n },\n \"vertices\": [\n {\n \"field\": \"product\"\n }\n ],\n \"connections\": {\n \"vertices\": [\n {\n \"field\": \"query.raw\"\n }\n ]\n }\n}"'{
"query": {
"match": {
"query.raw": "midi"
}
},
"vertices": [
{
"field": "product"
}
],
"connections": {
"vertices": [
{
"field": "query.raw"
}
]
}
}Extract and summarize information about the documents and terms in an Elasticsearch data stream or index.
The easiest way to understand the behavior of this API is to use the Graph UI to explore connections.
An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph.
Subsequent requests enable you to spider out from one more vertices of interest.
You can exclude vertices that have already been returned.
Name of the index.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Specifies one or more fields that contain the terms you want to include in the graph as vertices.
curl \
--request POST 'http://api.example.com/{index}/_graph/explore' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": {\n \"match\": {\n \"query.raw\": \"midi\"\n }\n },\n \"vertices\": [\n {\n \"field\": \"product\"\n }\n ],\n \"connections\": {\n \"vertices\": [\n {\n \"field\": \"query.raw\"\n }\n ]\n }\n}"'{
"query": {
"match": {
"query.raw": "midi"
}
},
"vertices": [
{
"field": "product"
}
],
"connections": {
"vertices": [
{
"field": "query.raw"
}
]
}
}Index APIs enable you to manage individual indices, index settings, aliases, mappings, and index templates.
Get information about component templates.
Comma-separated list of component template names used to limit the request.
Wildcard (*) expressions are supported.
If true, returns settings in flat format.
Return all default configurations for the component template (default: false)
If true, the request retrieves information from the local node only.
If false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY"Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
An index template can be composed of multiple component templates.
To use a component template, specify it in an index template’s composed_of list.
Component templates are only applied to new data streams and indices as part of a matching index template.
Settings and mappings specified directly in the index template or the create index request override any settings or mappings specified in a component template.
Component templates are only used during index creation. For data streams, this includes data stream creation and the creation of a stream’s backing indices. Changes to component templates do not affect existing indices, including a stream’s backing indices.
You can use C-style /* *\/ block comments in component templates.
You can include comments anywhere in the request body except before the opening curly bracket.
Applying component templates
You cannot directly apply a component template to a data stream or index.
To be applied, a component template must be included in an index template's composed_of list.
Name of the component template to create.
Elasticsearch includes the following built-in component templates: logs-mappings; logs-settings; metrics-mappings; metrics-settings;synthetics-mapping; synthetics-settings.
Elastic Agent uses these templates to configure backing indices for its data streams.
If you use Elastic Agent and want to overwrite one of these templates, set the version for your replacement template higher than the current version.
If you don’t use Elastic Agent and want to disable all built-in component and index templates, set stack.templates.enabled to false using the cluster update settings API.
If true, this request cannot replace or update existing component templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request PUT 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"template\": null,\n \"settings\": {\n \"number_of_shards\": 1\n },\n \"mappings\": {\n \"_source\": {\n \"enabled\": false\n },\n \"properties\": {\n \"host_name\": {\n \"type\": \"keyword\"\n },\n \"created_at\": {\n \"type\": \"date\",\n \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n }\n }\n }\n}"'{
"template": null,
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}{
"template": null,
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
An index template can be composed of multiple component templates.
To use a component template, specify it in an index template’s composed_of list.
Component templates are only applied to new data streams and indices as part of a matching index template.
Settings and mappings specified directly in the index template or the create index request override any settings or mappings specified in a component template.
Component templates are only used during index creation. For data streams, this includes data stream creation and the creation of a stream’s backing indices. Changes to component templates do not affect existing indices, including a stream’s backing indices.
You can use C-style /* *\/ block comments in component templates.
You can include comments anywhere in the request body except before the opening curly bracket.
Applying component templates
You cannot directly apply a component template to a data stream or index.
To be applied, a component template must be included in an index template's composed_of list.
Name of the component template to create.
Elasticsearch includes the following built-in component templates: logs-mappings; logs-settings; metrics-mappings; metrics-settings;synthetics-mapping; synthetics-settings.
Elastic Agent uses these templates to configure backing indices for its data streams.
If you use Elastic Agent and want to overwrite one of these templates, set the version for your replacement template higher than the current version.
If you don’t use Elastic Agent and want to disable all built-in component and index templates, set stack.templates.enabled to false using the cluster update settings API.
If true, this request cannot replace or update existing component templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"template\": null,\n \"settings\": {\n \"number_of_shards\": 1\n },\n \"mappings\": {\n \"_source\": {\n \"enabled\": false\n },\n \"properties\": {\n \"host_name\": {\n \"type\": \"keyword\"\n },\n \"created_at\": {\n \"type\": \"date\",\n \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n }\n }\n }\n}"'{
"template": null,
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}{
"template": null,
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
Comma-separated list or wildcard expression of component template names used to limit the request.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY"Returns information about whether a particular component template exists.
Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
curl \
--request HEAD 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY"Get information about component templates.
If true, returns settings in flat format.
Return all default configurations for the component template (default: false)
If true, the request retrieves information from the local node only.
If false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_component_template' \
--header "Authorization: $API_KEY"Add an index block to an index. Index blocks limit the operations allowed on an index by blocking specific operation types.
A comma-separated list or wildcard expression of index names used to limit the request.
By default, you must explicitly name the indices you are adding blocks to.
To allow the adding of blocks to indices with _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false.
You can update this setting in the elasticsearch.yml file or by using the cluster update settings API.
The block type to add to the index.
Supported values include:
metadata: Disable metadata changes, such as closing the index.read: Disable read operations.read_only: Disable write operations and metadata changes.write: Disable write operations. However, metadata changes are still allowed.Values are metadata, read, read_only, or write.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.The period to wait for the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata.
If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request PUT 'http://api.example.com/{index}/_block/{block}' \
--header "Authorization: $API_KEY"{
"acknowledged" : true,
"shards_acknowledged" : true,
"indices" : [ {
"name" : "my-index-000001",
"blocked" : true
} ]
}The analyze API performs analysis on a text string and returns the resulting tokens.
Generating excessive amount of tokens may cause a node to run out of memory.
The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The _analyze endpoint without a specified index will always use 10000 as its limit.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
The name of the analyzer that should be applied to the provided text.
This could be a built-in analyzer, or an analyzer that’s been configured in the index.
Array of token attributes used to filter the output of the explain parameter.
Array of character filters used to preprocess characters before the tokenizer.
If true, the response includes token attributes and additional details.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Array of token filters used to apply after the tokenizer.
Normalizer to use to convert text into a single token.
curl \
--request GET 'http://api.example.com/_analyze' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analyzer\": \"standard\",\n \"text\": \"this is a test\"\n}"'{
"analyzer": "standard",
"text": "this is a test"
}{
"analyzer": "standard",
"text": [
"this is a test",
"the second text"
]
}{
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"char_filter": [
"html_strip"
],
"text": "this is a <b>test</b>"
}{
"tokenizer": "whitespace",
"filter": [
"lowercase",
{
"type": "stop",
"stopwords": [
"a",
"is",
"this"
]
}
],
"text": "this is a test"
}{
"field": "obj1.field1",
"text": "this is a test"
}{
"normalizer": "my_normalizer",
"text": "BaR"
}{
"tokenizer": "standard",
"filter": [
"snowball"
],
"text": "detailed output",
"explain": true,
"attributes": [
"keyword"
]
}{
"detail": {
"custom_analyzer": true,
"charfilters": [],
"tokenizer": {
"name": "standard",
"tokens": [
{
"token": "detailed",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1
}
]
},
"tokenfilters": [
{
"name": "snowball",
"tokens": [
{
"token": "detail",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0,
"keyword": false
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1,
"keyword": false
}
]
}
]
}
}The analyze API performs analysis on a text string and returns the resulting tokens.
Generating excessive amount of tokens may cause a node to run out of memory.
The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The _analyze endpoint without a specified index will always use 10000 as its limit.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
The name of the analyzer that should be applied to the provided text.
This could be a built-in analyzer, or an analyzer that’s been configured in the index.
Array of token attributes used to filter the output of the explain parameter.
Array of character filters used to preprocess characters before the tokenizer.
If true, the response includes token attributes and additional details.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Array of token filters used to apply after the tokenizer.
Normalizer to use to convert text into a single token.
curl \
--request POST 'http://api.example.com/_analyze' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analyzer\": \"standard\",\n \"text\": \"this is a test\"\n}"'{
"analyzer": "standard",
"text": "this is a test"
}{
"analyzer": "standard",
"text": [
"this is a test",
"the second text"
]
}{
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"char_filter": [
"html_strip"
],
"text": "this is a <b>test</b>"
}{
"tokenizer": "whitespace",
"filter": [
"lowercase",
{
"type": "stop",
"stopwords": [
"a",
"is",
"this"
]
}
],
"text": "this is a test"
}{
"field": "obj1.field1",
"text": "this is a test"
}{
"normalizer": "my_normalizer",
"text": "BaR"
}{
"tokenizer": "standard",
"filter": [
"snowball"
],
"text": "detailed output",
"explain": true,
"attributes": [
"keyword"
]
}{
"detail": {
"custom_analyzer": true,
"charfilters": [],
"tokenizer": {
"name": "standard",
"tokens": [
{
"token": "detailed",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1
}
]
},
"tokenfilters": [
{
"name": "snowball",
"tokens": [
{
"token": "detail",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0,
"keyword": false
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1,
"keyword": false
}
]
}
]
}
}The analyze API performs analysis on a text string and returns the resulting tokens.
Generating excessive amount of tokens may cause a node to run out of memory.
The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The _analyze endpoint without a specified index will always use 10000 as its limit.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
The name of the analyzer that should be applied to the provided text.
This could be a built-in analyzer, or an analyzer that’s been configured in the index.
Array of token attributes used to filter the output of the explain parameter.
Array of character filters used to preprocess characters before the tokenizer.
If true, the response includes token attributes and additional details.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Array of token filters used to apply after the tokenizer.
Normalizer to use to convert text into a single token.
curl \
--request GET 'http://api.example.com/{index}/_analyze' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analyzer\": \"standard\",\n \"text\": \"this is a test\"\n}"'{
"analyzer": "standard",
"text": "this is a test"
}{
"analyzer": "standard",
"text": [
"this is a test",
"the second text"
]
}{
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"char_filter": [
"html_strip"
],
"text": "this is a <b>test</b>"
}{
"tokenizer": "whitespace",
"filter": [
"lowercase",
{
"type": "stop",
"stopwords": [
"a",
"is",
"this"
]
}
],
"text": "this is a test"
}{
"field": "obj1.field1",
"text": "this is a test"
}{
"normalizer": "my_normalizer",
"text": "BaR"
}{
"tokenizer": "standard",
"filter": [
"snowball"
],
"text": "detailed output",
"explain": true,
"attributes": [
"keyword"
]
}{
"detail": {
"custom_analyzer": true,
"charfilters": [],
"tokenizer": {
"name": "standard",
"tokens": [
{
"token": "detailed",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1
}
]
},
"tokenfilters": [
{
"name": "snowball",
"tokens": [
{
"token": "detail",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0,
"keyword": false
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1,
"keyword": false
}
]
}
]
}
}The analyze API performs analysis on a text string and returns the resulting tokens.
Generating excessive amount of tokens may cause a node to run out of memory.
The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The _analyze endpoint without a specified index will always use 10000 as its limit.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
Index used to derive the analyzer.
If specified, the analyzer or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
The name of the analyzer that should be applied to the provided text.
This could be a built-in analyzer, or an analyzer that’s been configured in the index.
Array of token attributes used to filter the output of the explain parameter.
Array of character filters used to preprocess characters before the tokenizer.
If true, the response includes token attributes and additional details.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Array of token filters used to apply after the tokenizer.
Normalizer to use to convert text into a single token.
curl \
--request POST 'http://api.example.com/{index}/_analyze' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analyzer\": \"standard\",\n \"text\": \"this is a test\"\n}"'{
"analyzer": "standard",
"text": "this is a test"
}{
"analyzer": "standard",
"text": [
"this is a test",
"the second text"
]
}{
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"char_filter": [
"html_strip"
],
"text": "this is a <b>test</b>"
}{
"tokenizer": "whitespace",
"filter": [
"lowercase",
{
"type": "stop",
"stopwords": [
"a",
"is",
"this"
]
}
],
"text": "this is a test"
}{
"field": "obj1.field1",
"text": "this is a test"
}{
"normalizer": "my_normalizer",
"text": "BaR"
}{
"tokenizer": "standard",
"filter": [
"snowball"
],
"text": "detailed output",
"explain": true,
"attributes": [
"keyword"
]
}{
"detail": {
"custom_analyzer": true,
"charfilters": [],
"tokenizer": {
"name": "standard",
"tokens": [
{
"token": "detailed",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1
}
]
},
"tokenfilters": [
{
"name": "snowball",
"tokens": [
{
"token": "detail",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0,
"keyword": false
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1,
"keyword": false
}
]
}
]
}
}Get information about one or more indices. For data streams, the API returns information about the stream’s backing indices.
Comma-separated list of data streams, indices, and index aliases used to limit the request. Wildcard expressions (*) are supported.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Return only information on specified index features
Supported values include: aliases, mappings, settings
curl \
--request GET 'http://api.example.com/{index}' \
--header "Authorization: $API_KEY"You can use the create index API to add a new index to an Elasticsearch cluster. When creating an index, you can specify the following:
Wait for active shards
By default, index creation will only return a response to the client when the primary copies of each shard have been started, or the request times out.
The index creation response will indicate what happened.
For example, acknowledged indicates whether the index was successfully created in the cluster, while shards_acknowledged indicates whether the requisite number of shard copies were started for each shard in the index before timing out.
Note that it is still possible for either acknowledged or shards_acknowledged to be false, but for the index creation to be successful.
These values simply indicate whether the operation completed before the timeout.
If acknowledged is false, the request timed out before the cluster state was updated with the newly created index, but it probably will be created sometime soon.
If shards_acknowledged is false, then the request timed out before the requisite number of shards were started (by default just the primaries), even if the cluster state was successfully updated to reflect the newly created index (that is to say, acknowledged is true).
You can change the default of only waiting for the primary shards to start through the index setting index.write.wait_for_active_shards.
Note that changing this setting will also affect the wait_for_active_shards value on all subsequent write operations.
Name of the index you wish to create. Index names must meet the following criteria:
\, /, *, ?, ", <, >, |, ,, or #:), but that has been deprecated and will not be supported in later versions-, _, or +. or ... are deprecated, except for hidden indices and internal indices managed by pluginsPeriod to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
curl \
--request PUT 'http://api.example.com/{index}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"settings\": {\n \"number_of_shards\": 3,\n \"number_of_replicas\": 2\n }\n}"'{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 2
}
}{
"settings": {
"number_of_shards": 1
},
"mappings": {
"properties": {
"field1": { "type": "text" }
}
}
}{
"aliases": {
"alias_1": {},
"alias_2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
}
}
}Deleting an index deletes its documents, shards, and metadata. It does not delete related Kibana components, such as data views, visualizations, or dashboards.
You cannot delete the current write index of a data stream. To delete the index, you must roll over the data stream so a new write index is created. You can then use the delete index API to delete the previous write index.
Comma-separated list of indices to delete.
You cannot specify index aliases.
By default, this parameter does not support wildcards (*) or _all.
To use wildcards or _all, set the action.destructive_requires_name cluster setting to false.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/{index}' \
--header "Authorization: $API_KEY"Comma-separated list of data streams, indices, and aliases. Supports wildcards (*).
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only.
curl \
--request HEAD 'http://api.example.com/{index}' \
--header "Authorization: $API_KEY"Retrieves information for one or more data stream or index aliases.
Comma-separated list of data streams or indices used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
Comma-separated list of aliases to retrieve.
Supports wildcards (*).
To retrieve all aliases, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/{index}/_alias/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of data streams or indices to add.
Supports wildcards (*).
Wildcard patterns that match both data streams and indices return an error.
Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
If true, sets the write index or data stream for the alias.
If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests.
If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index.
Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.
curl \
--request PUT 'http://api.example.com/{index}/_alias/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"actions\": [\n {\n \"add\": {\n \"index\": \"my-data-stream\",\n \"alias\": \"my-alias\"\n }\n }\n ]\n}"'{
"actions": [
{
"add": {
"index": "my-data-stream",
"alias": "my-alias"
}
}
]
}Comma-separated list of data streams or indices to add.
Supports wildcards (*).
Wildcard patterns that match both data streams and indices return an error.
Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
If true, sets the write index or data stream for the alias.
If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests.
If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index.
Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.
curl \
--request POST 'http://api.example.com/{index}/_alias/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"actions\": [\n {\n \"add\": {\n \"index\": \"my-data-stream\",\n \"alias\": \"my-alias\"\n }\n }\n ]\n}"'{
"actions": [
{
"add": {
"index": "my-data-stream",
"alias": "my-alias"
}
}
]
}Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/{index}/_alias/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of data streams or indices used to limit the request. Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
Comma-separated list of aliases to check. Supports wildcards (*).
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request HEAD 'http://api.example.com/{index}/_alias/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of data streams or indices to add.
Supports wildcards (*).
Wildcard patterns that match both data streams and indices return an error.
Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
If true, sets the write index or data stream for the alias.
If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests.
If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index.
Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.
curl \
--request PUT 'http://api.example.com/{index}/_aliases/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"actions\": [\n {\n \"add\": {\n \"index\": \"my-data-stream\",\n \"alias\": \"my-alias\"\n }\n }\n ]\n}"'{
"actions": [
{
"add": {
"index": "my-data-stream",
"alias": "my-alias"
}
}
]
}Comma-separated list of data streams or indices to add.
Supports wildcards (*).
Wildcard patterns that match both data streams and indices return an error.
Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
If true, sets the write index or data stream for the alias.
If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests.
If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index.
Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.
curl \
--request POST 'http://api.example.com/{index}/_aliases/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"actions\": [\n {\n \"add\": {\n \"index\": \"my-data-stream\",\n \"alias\": \"my-alias\"\n }\n }\n ]\n}"'{
"actions": [
{
"add": {
"index": "my-data-stream",
"alias": "my-alias"
}
}
]
}Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/{index}/_aliases/{name}' \
--header "Authorization: $API_KEY"Get information about one or more index templates.
Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, returns all relevant default configurations for the index template.
curl \
--request GET 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY"Index templates define settings, mappings, and aliases that can be applied automatically to new indices.
Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.
You can use C-style /* *\/ block comments in index templates.
You can include comments anywhere in the request body, except before the opening curly bracket.
Multiple matching templates
If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.
Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.
Composing aliases, mappings, and settings
When multiple component templates are specified in the composed_of field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates.
Any mappings, settings, or aliases from the parent index template are merged in next.
Finally, any configuration on the index request itself is merged.
Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration.
If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one.
This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates and meta.
If an earlier component contains a dynamic_templates block, then by default new dynamic_templates entries are appended onto the end.
If an entry already exists with the same key, then it is overwritten by the new definition.
Index or template name
If true, this request cannot replace or update existing index templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
User defined reason for creating/updating the index template
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
This setting overrides the value of the action.auto_create_index cluster setting.
If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index.
If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request PUT 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\" : [\"template*\"],\n \"priority\" : 1,\n \"template\": {\n \"settings\" : {\n \"number_of_shards\" : 2\n }\n }\n}"'{
"index_patterns" : ["template*"],
"priority" : 1,
"template": {
"settings" : {
"number_of_shards" : 2
}
}
}{
"index_patterns": [
"template*"
],
"template": {
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}
}Index templates define settings, mappings, and aliases that can be applied automatically to new indices.
Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.
You can use C-style /* *\/ block comments in index templates.
You can include comments anywhere in the request body, except before the opening curly bracket.
Multiple matching templates
If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.
Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.
Composing aliases, mappings, and settings
When multiple component templates are specified in the composed_of field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates.
Any mappings, settings, or aliases from the parent index template are merged in next.
Finally, any configuration on the index request itself is merged.
Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration.
If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one.
This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates and meta.
If an earlier component contains a dynamic_templates block, then by default new dynamic_templates entries are appended onto the end.
If an entry already exists with the same key, then it is overwritten by the new definition.
Index or template name
If true, this request cannot replace or update existing index templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
User defined reason for creating/updating the index template
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
This setting overrides the value of the action.auto_create_index cluster setting.
If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index.
If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\" : [\"template*\"],\n \"priority\" : 1,\n \"template\": {\n \"settings\" : {\n \"number_of_shards\" : 2\n }\n }\n}"'{
"index_patterns" : ["template*"],
"priority" : 1,
"template": {
"settings" : {
"number_of_shards" : 2
}
}
}{
"index_patterns": [
"template*"
],
"template": {
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}
}The provided may contain multiple template names separated by a comma. If multiple template names are specified then there is no wildcard support and the provided names should match completely with existing templates.
Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request HEAD 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of aliases to retrieve.
Supports wildcards (*).
To retrieve all aliases, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_alias/{name}' \
--header "Authorization: $API_KEY"Comma-separated list of aliases to check. Supports wildcards (*).
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request HEAD 'http://api.example.com/_alias/{name}' \
--header "Authorization: $API_KEY"If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_alias' \
--header "Authorization: $API_KEY"Comma-separated list of data streams or indices used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/{index}/_alias' \
--header "Authorization: $API_KEY"Get information about one or more index templates.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, returns all relevant default configurations for the index template.
curl \
--request GET 'http://api.example.com/_index_template' \
--header "Authorization: $API_KEY"For data streams, the API retrieves mappings for the stream’s backing indices.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the request retrieves information from the local node only.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_mapping' \
--header "Authorization: $API_KEY"For data streams, the API retrieves mappings for the stream’s backing indices.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the request retrieves information from the local node only.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/{index}/_mapping' \
--header "Authorization: $API_KEY"Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.
Add multi-fields to an existing field
Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.
Change supported mapping parameters for an existing field
The documentation for each mapping parameter indicates whether you can update it for an existing field using this API.
For example, you can use the update mapping API to update the ignore_above parameter.
Change the mapping of an existing field
Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.
If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.
Rename a field
Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.
A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
If true, the mappings are applied only to the current write index for the target.
Controls whether dynamic date detection is enabled.
Values are strict, runtime, true, or false.
If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.
Specify dynamic templates for the mapping.
Automatically map strings into numeric data types for all fields.
Mapping for a field. For new fields, this mapping can include:
curl \
--request PUT 'http://api.example.com/{index}/_mapping' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"properties\": {\n \"user\": {\n \"properties\": {\n \"name\": {\n \"type\": \"keyword\"\n }\n }\n }\n }\n}"'{
"properties": {
"user": {
"properties": {
"name": {
"type": "keyword"
}
}
}
}
}Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.
Add multi-fields to an existing field
Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.
Change supported mapping parameters for an existing field
The documentation for each mapping parameter indicates whether you can update it for an existing field using this API.
For example, you can use the update mapping API to update the ignore_above parameter.
Change the mapping of an existing field
Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.
If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.
Rename a field
Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.
A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
If true, the mappings are applied only to the current write index for the target.
Controls whether dynamic date detection is enabled.
Values are strict, runtime, true, or false.
If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.
Specify dynamic templates for the mapping.
Automatically map strings into numeric data types for all fields.
Mapping for a field. For new fields, this mapping can include:
curl \
--request POST 'http://api.example.com/{index}/_mapping' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"properties\": {\n \"user\": {\n \"properties\": {\n \"name\": {\n \"type\": \"keyword\"\n }\n }\n }\n }\n}"'{
"properties": {
"user": {
"properties": {
"name": {
"type": "keyword"
}
}
}
}
}Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only. If
false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_settings' \
--header "Authorization: $API_KEY"Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.
To revert a setting to the default value, use a null value.
The list of per-index settings that can be updated dynamically on live indices can be found in index module documentation.
To preserve existing settings from being updated, set the preserve_existing parameter to true.
NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match. If the request can target
data streams, this argument determines whether wildcard expressions match
hidden data streams. Supports comma-separated values, such as
open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, existing index settings remain unchanged.
Whether to close and reopen the index to apply non-dynamic settings.
If set to true the indices to which the settings are being applied
will be closed temporarily and then reopened in order to apply the changes.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are true, false, or checksum.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
Time unit for milliseconds
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Configure custom similarity settings to customize how search results are scored.
curl \
--request PUT 'http://api.example.com/_settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\" : {\n \"number_of_replicas\" : 2\n }\n}"'{
"index" : {
"number_of_replicas" : 2
}
}{
"index" : {
"refresh_interval" : null
}
}{
"analysis" : {
"analyzer":{
"content":{
"type":"custom",
"tokenizer":"whitespace"
}
}
}
}
POST /my-index-000001/_openGet setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.
Comma-separated list of data streams, indices, and aliases used to limit
the request. Supports wildcards (*). To target all data streams and
indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only. If
false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/{index}/_settings' \
--header "Authorization: $API_KEY"Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.
To revert a setting to the default value, use a null value.
The list of per-index settings that can be updated dynamically on live indices can be found in index module documentation.
To preserve existing settings from being updated, set the preserve_existing parameter to true.
NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.
Comma-separated list of data streams, indices, and aliases used to limit
the request. Supports wildcards (*). To target all data streams and
indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match. If the request can target
data streams, this argument determines whether wildcard expressions match
hidden data streams. Supports comma-separated values, such as
open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, existing index settings remain unchanged.
Whether to close and reopen the index to apply non-dynamic settings.
If set to true the indices to which the settings are being applied
will be closed temporarily and then reopened in order to apply the changes.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are true, false, or checksum.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
Time unit for milliseconds
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Configure custom similarity settings to customize how search results are scored.
curl \
--request PUT 'http://api.example.com/{index}/_settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\" : {\n \"number_of_replicas\" : 2\n }\n}"'{
"index" : {
"number_of_replicas" : 2
}
}{
"index" : {
"refresh_interval" : null
}
}{
"analysis" : {
"analyzer":{
"content":{
"type":"custom",
"tokenizer":"whitespace"
}
}
}
}
POST /my-index-000001/_openGet setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.
Comma-separated list of data streams, indices, and aliases used to limit
the request. Supports wildcards (*). To target all data streams and
indices, omit this parameter or use * or _all.
Comma-separated list or wildcard expression of settings to retrieve.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only. If
false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/{index}/_settings/{name}' \
--header "Authorization: $API_KEY"Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.
Comma-separated list or wildcard expression of settings to retrieve.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, returns settings in flat format.
If true, return all default settings in the response.
If true, the request retrieves information from the local node only. If
false, information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_settings/{name}' \
--header "Authorization: $API_KEY"A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.curl \
--request GET 'http://api.example.com/_refresh' \
--header "Authorization: $API_KEY"A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.curl \
--request POST 'http://api.example.com/_refresh' \
--header "Authorization: $API_KEY"A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.curl \
--request GET 'http://api.example.com/{index}/_refresh' \
--header "Authorization: $API_KEY"A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.curl \
--request POST 'http://api.example.com/{index}/_refresh' \
--header "Authorization: $API_KEY"Resolve the names and/or index patterns for indices, aliases, and data streams. Multiple patterns and remote clusters are supported.
Comma-separated name(s) or index pattern(s) of the indices, aliases, and data streams to resolve.
Resources on remote clusters can be specified using the <cluster>:<name> syntax.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
curl \
--request GET 'http://api.example.com/_resolve/index/{name}' \
--header "Authorization: $API_KEY"{
"indices": [
{
"name": "foo_closed",
"attributes": [
"closed"
]
},
{
"name": "freeze-index",
"aliases": [
"f-alias"
],
"attributes": [
"open"
]
},
{
"name": "remoteCluster1:bar-01",
"attributes": [
"open"
]
}
],
"aliases": [
{
"name": "f-alias",
"indices": [
"freeze-index",
"my-index-000001"
]
}
],
"data_streams": [
{
"name": "foo",
"backing_indices": [
".ds-foo-2099.03.07-000001"
],
"timestamp_field": "@timestamp"
}
]
}TIP: It is recommended to use the index lifecycle rollover action to automate rollovers.
The rollover API creates a new index for a data stream or index alias. The API behavior depends on the rollover target.
Roll over a data stream
If you roll over a data stream, the API creates a new write index for the stream. The stream's previous write index becomes a regular backing index. A rollover also increments the data stream's generation.
Roll over an index alias with a write index
TIP: Prior to Elasticsearch 7.9, you'd typically use an index alias with a write index to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with data tiers.
If an index alias points to multiple indices, one of the indices must be a write index.
The rollover API creates a new write index for the alias with is_write_index set to true.
The API also sets is_write_index to false for the previous write index.
Roll over an index alias with one index
If you roll over an index alias that points to only one index, the API creates a new index for the alias and removes the original index from the alias.
NOTE: A rollover creates a new index and is subject to the wait_for_active_shards setting.
Increment index names for an alias
When you roll over an index alias, you can specify a name for the new index.
If you don't specify a name and the current index ends with - and a number, such as my-index-000001 or my-index-3, the new index name increments that number.
For example, if you roll over an alias with a current index of my-index-000001, the rollover creates a new index named my-index-000002.
This number is always six characters and zero-padded, regardless of the previous index's name.
If you use an index alias for time series data, you can use date math in the index name to track the rollover date.
For example, you can create an alias that points to an index named <my-index-{now/d}-000001>.
If you create the index on May 6, 2099, the index's name is my-index-2099.05.06-000001.
If you roll over the alias on May 7, 2099, the new index's name is my-index-2099.05.07-000002.
Name of the data stream or index alias to roll over.
If true, checks whether the current index satisfies the specified conditions but does not perform a rollover.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
If set to true, the rollover action will only mark a data stream to signal that it needs to be rolled over at the next write. Only allowed on data streams.
Aliases for the target index. Data streams do not support this parameter.
Configuration options for the index. Data streams do not support this parameter.
curl \
--request POST 'http://api.example.com/{alias}/_rollover' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"conditions\": {\n \"max_age\": \"7d\",\n \"max_docs\": 1000,\n \"max_primary_shard_size\": \"50gb\",\n \"max_primary_shard_docs\": \"2000\"\n }\n}"'{
"conditions": {
"max_age": "7d",
"max_docs": 1000,
"max_primary_shard_size": "50gb",
"max_primary_shard_docs": "2000"
}
}{
"_shards": {},
"indices": {
"test": {
"shards": {
"0": [
{
"routing": {
"state": "STARTED",
"primary": true,
"node": "zDC_RorJQCao9xf9pg3Fvw"
},
"num_committed_segments": 0,
"num_search_segments": 1,
"segments": {
"_0": {
"generation": 0,
"num_docs": 1,
"deleted_docs": 0,
"size_in_bytes": 3800,
"committed": false,
"search": true,
"version": "7.0.0",
"compound": true,
"attributes": {}
}
}
}
]
}
}
}
}TIP: It is recommended to use the index lifecycle rollover action to automate rollovers.
The rollover API creates a new index for a data stream or index alias. The API behavior depends on the rollover target.
Roll over a data stream
If you roll over a data stream, the API creates a new write index for the stream. The stream's previous write index becomes a regular backing index. A rollover also increments the data stream's generation.
Roll over an index alias with a write index
TIP: Prior to Elasticsearch 7.9, you'd typically use an index alias with a write index to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with data tiers.
If an index alias points to multiple indices, one of the indices must be a write index.
The rollover API creates a new write index for the alias with is_write_index set to true.
The API also sets is_write_index to false for the previous write index.
Roll over an index alias with one index
If you roll over an index alias that points to only one index, the API creates a new index for the alias and removes the original index from the alias.
NOTE: A rollover creates a new index and is subject to the wait_for_active_shards setting.
Increment index names for an alias
When you roll over an index alias, you can specify a name for the new index.
If you don't specify a name and the current index ends with - and a number, such as my-index-000001 or my-index-3, the new index name increments that number.
For example, if you roll over an alias with a current index of my-index-000001, the rollover creates a new index named my-index-000002.
This number is always six characters and zero-padded, regardless of the previous index's name.
If you use an index alias for time series data, you can use date math in the index name to track the rollover date.
For example, you can create an alias that points to an index named <my-index-{now/d}-000001>.
If you create the index on May 6, 2099, the index's name is my-index-2099.05.06-000001.
If you roll over the alias on May 7, 2099, the new index's name is my-index-2099.05.07-000002.
If true, checks whether the current index satisfies the specified conditions but does not perform a rollover.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
If set to true, the rollover action will only mark a data stream to signal that it needs to be rolled over at the next write. Only allowed on data streams.
Aliases for the target index. Data streams do not support this parameter.
Configuration options for the index. Data streams do not support this parameter.
curl \
--request POST 'http://api.example.com/{alias}/_rollover/{new_index}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"conditions\": {\n \"max_age\": \"7d\",\n \"max_docs\": 1000,\n \"max_primary_shard_size\": \"50gb\",\n \"max_primary_shard_docs\": \"2000\"\n }\n}"'{
"conditions": {
"max_age": "7d",
"max_docs": 1000,
"max_primary_shard_size": "50gb",
"max_primary_shard_docs": "2000"
}
}{
"_shards": {},
"indices": {
"test": {
"shards": {
"0": [
{
"routing": {
"state": "STARTED",
"primary": true,
"node": "zDC_RorJQCao9xf9pg3Fvw"
},
"num_committed_segments": 0,
"num_search_segments": 1,
"segments": {
"_0": {
"generation": 0,
"num_docs": 1,
"deleted_docs": 0,
"size_in_bytes": 3800,
"committed": false,
"search": true,
"version": "7.0.0",
"compound": true,
"attributes": {}
}
}
}
]
}
}
}
}Get the index configuration that would be applied to the specified index from an existing index template.
Name of the index to simulate
Whether the index template we optionally defined in the body should only be dry-run added if new or can also replace an existing one
User defined reason for dry-run creating the new template for simulation purposes
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, returns all relevant default configurations for the index template.
curl \
--request POST 'http://api.example.com/_index_template/_simulate_index/{name}' \
--header "Authorization: $API_KEY"{
"template" : {
"settings" : {
"index" : {
"number_of_shards" : "2",
"number_of_replicas" : "0",
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
}
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
}
}
},
"aliases" : { }
},
"overlapping" : [
{
"name" : "template_1",
"index_patterns" : [
"my-index-*"
]
}
]
}Get the index configuration that would be applied by a particular index template.
If true, the template passed in the body is only used if no existing templates match the same index patterns. If false, the simulation uses the template with the highest priority. Note that the template is not permanently added or updated in either case; it is only used for the simulation.
User defined reason for dry-run creating the new template for simulation purposes
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, returns all relevant default configurations for the index template.
This setting overrides the value of the action.auto_create_index cluster setting.
If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index.
If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST 'http://api.example.com/_index_template/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\": [\"my-index-*\"],\n \"composed_of\": [\"ct2\"],\n \"priority\": 10,\n \"template\": {\n \"settings\": {\n \"index.number_of_replicas\": 1\n }\n }\n}"'{
"index_patterns": ["my-index-*"],
"composed_of": ["ct2"],
"priority": 10,
"template": {
"settings": {
"index.number_of_replicas": 1
}
}
}{
"template" : {
"settings" : {
"index" : {
"number_of_replicas" : "1",
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
}
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
}
}
},
"aliases" : { }
},
"overlapping" : [
{
"name" : "final-template",
"index_patterns" : [
"my-index-*"
]
}
]
}Get the index configuration that would be applied by a particular index template.
Name of the index template to simulate. To test a template configuration before you add it to the cluster, omit this parameter and specify the template configuration in the request body.
If true, the template passed in the body is only used if no existing templates match the same index patterns. If false, the simulation uses the template with the highest priority. Note that the template is not permanently added or updated in either case; it is only used for the simulation.
User defined reason for dry-run creating the new template for simulation purposes
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, returns all relevant default configurations for the index template.
This setting overrides the value of the action.auto_create_index cluster setting.
If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index.
If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST 'http://api.example.com/_index_template/_simulate/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\": [\"my-index-*\"],\n \"composed_of\": [\"ct2\"],\n \"priority\": 10,\n \"template\": {\n \"settings\": {\n \"index.number_of_replicas\": 1\n }\n }\n}"'{
"index_patterns": ["my-index-*"],
"composed_of": ["ct2"],
"priority": 10,
"template": {
"settings": {
"index.number_of_replicas": 1
}
}
}{
"template" : {
"settings" : {
"index" : {
"number_of_replicas" : "1",
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
}
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
}
}
},
"aliases" : { }
},
"overlapping" : [
{
"name" : "final-template",
"index_patterns" : [
"my-index-*"
]
}
]
}Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_aliases' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"actions":[{"add":{"alias":"string","aliases":"string","filter":{},"index":"string","indices":"string","index_routing":"string","is_hidden":true,"is_write_index":true,"routing":"string","search_routing":"string","must_exist":true},"remove":{"alias":"string","aliases":"string","index":"string","indices":"string","must_exist":true},"remove_index":{"index":"string","indices":"string","must_exist":true}}]}'If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
If true, the validation is executed on all shards instead of one random shard per index.
Analyzer to use for the query string.
This parameter can only be used when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
The default operator for query string query: AND or OR.
Values are and, AND, or, or OR.
Field to use as default where no field prefix is given in the query string.
This parameter can only be used when the q query string parameter is specified.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the response returns detailed information if an error has occurred.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
Query in the Lucene query string syntax.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request GET 'http://api.example.com/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
If true, the validation is executed on all shards instead of one random shard per index.
Analyzer to use for the query string.
This parameter can only be used when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
The default operator for query string query: AND or OR.
Values are and, AND, or, or OR.
Field to use as default where no field prefix is given in the query string.
This parameter can only be used when the q query string parameter is specified.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the response returns detailed information if an error has occurred.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
Query in the Lucene query string syntax.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'Comma-separated list of data streams, indices, and aliases to search.
Supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
If true, the validation is executed on all shards instead of one random shard per index.
Analyzer to use for the query string.
This parameter can only be used when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
The default operator for query string query: AND or OR.
Values are and, AND, or, or OR.
Field to use as default where no field prefix is given in the query string.
This parameter can only be used when the q query string parameter is specified.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the response returns detailed information if an error has occurred.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
Query in the Lucene query string syntax.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request GET 'http://api.example.com/{index}/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'Comma-separated list of data streams, indices, and aliases to search.
Supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
If true, the validation is executed on all shards instead of one random shard per index.
Analyzer to use for the query string.
This parameter can only be used when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
The default operator for query string query: AND or OR.
Values are and, AND, or, or OR.
Field to use as default where no field prefix is given in the query string.
This parameter can only be used when the q query string parameter is specified.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, the response returns detailed information if an error has occurred.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
Query in the Lucene query string syntax.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/{index}/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'Inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation.
It only works with the chat_completion task type for openai and elastic inference services.
IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
NOTE: The chat_completion task type is only available within the _stream API and only supports streaming.
The Chat completion inference API and the Stream inference API differ in their response structure and capabilities.
The Chat completion inference API provides more comprehensive customization options through more fields and function calling support.
If you use the openai service or the elastic service, use the Chat completion inference API.
The inference Id
Specifies the amount of time to wait for the inference request to complete.
A list of objects representing the conversation.
Requests should generally only add new messages from the user (role user).
The other message roles (assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.
The ID of the model to use.
The upper bound limit for the number of tokens that can be generated for a completion request.
A sequence of strings to control when the model should stop generating additional tokens.
The sampling temperature to use.
A list of tools that the model can call.
Nucleus sampling, an alternative to sampling with temperature.
curl \
--request POST 'http://api.example.com/_inference/chat_completion/{inference_id}/_stream' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"model\": \"gpt-4o\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"What is Elastic?\"\n }\n ]\n}"'{
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "What is Elastic?"
}
]
}{
"messages": [
{
"role": "assistant",
"content": "Let's find out what the weather is",
"tool_calls": [
{
"id": "call_KcAjWtAww20AihPHphUh46Gd",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\":\"Boston, MA\"}"
}
}
]
},
{
"role": "tool",
"content": "The weather is cold",
"tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
}
]
}{
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What's the price of a scarf?"
}
]
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_price",
"description": "Get the current price of a item",
"parameters": {
"type": "object",
"properties": {
"item": {
"id": "123"
}
}
}
}
}
],
"tool_choice": {
"type": "function",
"function": {
"name": "get_current_price"
}
}
}event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}
(...)
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}}
event: message
data: [DONE]The inference Id
Specifies the amount of time to wait for the inference request to complete.
Inference input. Either a string or an array of strings.
curl \
--request POST 'http://api.example.com/_inference/completion/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": \"What is Elastic?\"\n}"'{
"input": "What is Elastic?"
}{
"completion": [
{
"result": "Elastic is a company that provides a range of software solutions for search, logging, security, and analytics. Their flagship product is Elasticsearch, an open-source, distributed search engine that allows users to search, analyze, and visualize large volumes of data in real-time. Elastic also offers products such as Kibana, a data visualization tool, and Logstash, a log management and pipeline tool, as well as various other tools and solutions for data analysis and management."
}
]
}The inference Id
curl \
--request GET 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY"When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The inference Id
The service type
curl \
--request PUT 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"chunking_settings":{"max_chunk_size":42.0,"overlap":42.0,"sentence_overlap":42.0,"strategy":"string"},"service":"string","service_settings":{},"task_settings":{}}'This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.
For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.
The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The unique identifier for the inference endpoint.
The amount of time to wait for the inference request to complete.
The query input, which is required only for the rerank task.
It is not required for other tasks.
The text on which you want to perform the inference task. It can be a single string or an array.
Inference endpoints for the completion task type currently only support a single string as input.
curl \
--request POST 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":"string","input":"string","task_settings":{}}'The inference identifier.
curl \
--request DELETE 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY"The task type
Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
The inference Id
curl \
--request GET 'http://api.example.com/_inference/{task_type}/{inference_id}' \
--header "Authorization: $API_KEY"When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The task type
Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
The inference Id
The service type
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"chunking_settings":{"max_chunk_size":42.0,"overlap":42.0,"sentence_overlap":42.0,"strategy":"string"},"service":"string","service_settings":{},"task_settings":{}}'This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.
For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.
The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The type of inference task that the model performs.
Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
The unique identifier for the inference endpoint.
The amount of time to wait for the inference request to complete.
The query input, which is required only for the rerank task.
It is not required for other tasks.
The text on which you want to perform the inference task. It can be a single string or an array.
Inference endpoints for the completion task type currently only support a single string as input.
curl \
--request POST 'http://api.example.com/_inference/{task_type}/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":"string","input":"string","task_settings":{}}'The task type
Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
The inference identifier.
curl \
--request DELETE 'http://api.example.com/_inference/{task_type}/{inference_id}' \
--header "Authorization: $API_KEY"curl \
--request GET 'http://api.example.com/_inference' \
--header "Authorization: $API_KEY"Create an inference endpoint to perform an inference task with the alibabacloud-ai-search service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are completion, rerank, space_embedding, or text_embedding.
The unique identifier of the inference endpoint.
Value is alibabacloud-ai-search.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{alibabacloud_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"host\" : \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-qwen-turbo\",\n \"workspace\" : \"default\"\n }\n}"'{
"service": "alibabacloud-ai-search",
"service_settings": {
"host" : "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-qwen-turbo",
"workspace" : "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-bge-reranker-larger",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-text-sparse-embedding-001",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-text-embedding-001",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}Creates an inference endpoint to perform an inference task with the amazonbedrock service.
You need to provide the access and secret keys only once, during the inference model creation. The get inference API does not retrieve your access or secret keys. After creating the inference model, you cannot change the associated key pairs. If you want to use a different access and secret key pair, delete the inference model and recreate it with the same name and the updated keys.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are completion or text_embedding.
The unique identifier of the inference endpoint.
Value is amazonbedrock.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{amazonbedrock_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"amazonbedrock\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"provider\": \"amazontitan\",\n \"model\": \"amazon.titan-embed-text-v2:0\"\n }\n}"'{
"service": "amazonbedrock",
"service_settings": {
"access_key": "AWS-access-key",
"secret_key": "AWS-secret-key",
"region": "us-east-1",
"provider": "amazontitan",
"model": "amazon.titan-embed-text-v2:0"
}
}{
"service": "openai",
"service_settings": {
"api_key": "OpenAI-API-Key",
"model_id": "gpt-3.5-turbo"
}
}Create an inference endpoint to perform an inference task with the anthropic service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The task type.
The only valid task type for the model to perform is completion.
Value is completion.
The unique identifier of the inference endpoint.
Value is anthropic.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{anthropic_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"anthropic\",\n \"service_settings\": {\n \"api_key\": \"Anthropic-Api-Key\",\n \"model_id\": \"Model-ID\"\n },\n \"task_settings\": {\n \"max_tokens\": 1024\n }\n}"'{
"service": "anthropic",
"service_settings": {
"api_key": "Anthropic-Api-Key",
"model_id": "Model-ID"
},
"task_settings": {
"max_tokens": 1024
}
}Create an inference endpoint to perform an inference task with the azureaistudio service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are completion or text_embedding.
The unique identifier of the inference endpoint.
Value is azureaistudio.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{azureaistudio_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"azureaistudio\",\n \"service_settings\": {\n \"api_key\": \"Azure-AI-Studio-API-key\",\n \"target\": \"Target-Uri\",\n \"provider\": \"openai\",\n \"endpoint_type\": \"token\"\n }\n}"'{
"service": "azureaistudio",
"service_settings": {
"api_key": "Azure-AI-Studio-API-key",
"target": "Target-Uri",
"provider": "openai",
"endpoint_type": "token"
}
}{
"service": "azureaistudio",
"service_settings": {
"api_key": "Azure-AI-Studio-API-key",
"target": "Target-URI",
"provider": "databricks",
"endpoint_type": "realtime"
}
}Create an inference endpoint to perform an inference task with the azureopenai service.
The list of chat completion models that you can choose from in your Azure OpenAI deployment include:
The list of embeddings models that you can choose from in your deployment can be found in the Azure models documentation.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
NOTE: The chat_completion task type only supports streaming and only through the _stream API.
Values are completion or text_embedding.
The unique identifier of the inference endpoint.
Value is azureopenai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{azureopenai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}"'{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}Create an inference endpoint to perform an inference task with the cohere service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are completion, rerank, or text_embedding.
The unique identifier of the inference endpoint.
Value is cohere.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{cohere_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"cohere\",\n \"service_settings\": {\n \"api_key\": \"Cohere-Api-key\",\n \"model_id\": \"embed-english-light-v3.0\",\n \"embedding_type\": \"byte\"\n }\n}"'{
"service": "cohere",
"service_settings": {
"api_key": "Cohere-Api-key",
"model_id": "embed-english-light-v3.0",
"embedding_type": "byte"
}
}{
"service": "cohere",
"service_settings": {
"api_key": "Cohere-API-key",
"model_id": "rerank-english-v3.0"
},
"task_settings": {
"top_n": 10,
"return_documents": true
}
}Create an inference endpoint to perform an inference task with the elasticsearch service.
Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings.
If you use the ELSER or the E5 model through the elasticsearch service, the API request will automatically download and deploy the model if it isn't downloaded yet.
You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are rerank, sparse_embedding, or text_embedding.
The unique identifier of the inference endpoint.
The must not match the model_id.
Value is elasticsearch.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{elasticsearch_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"adaptive_allocations\": { \n \"enabled\": true,\n \"min_number_of_allocations\": 1,\n \"max_number_of_allocations\": 4\n },\n \"num_threads\": 1,\n \"model_id\": \".elser_model_2\" \n }\n}"'{
"service": "elasticsearch",
"service_settings": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 1,
"max_number_of_allocations": 4
},
"num_threads": 1,
"model_id": ".elser_model_2"
}
}{
"service": "elasticsearch",
"service_settings": {
"model_id": ".rerank-v1",
"num_threads": 1,
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 1,
"max_number_of_allocations": 4
}
}
}{
"service": "elasticsearch",
"service_settings": {
"num_allocations": 1,
"num_threads": 1,
"model_id": ".multilingual-e5-small"
}
}{
"service": "elasticsearch",
"service_settings": {
"num_allocations": 1,
"num_threads": 1,
"model_id": "msmarco-MiniLM-L12-cos-v5"
}
}{
"service": "elasticsearch",
"service_settings": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 3,
"max_number_of_allocations": 10
},
"num_threads": 1,
"model_id": ".multilingual-e5-small"
}
}{
"service": "elasticsearch",
"service_settings": {
"deployment_id": ".elser_model_2"
}
}{
"inference_id": "use_existing_deployment",
"task_type": "sparse_embedding",
"service": "elasticsearch",
"service_settings": {
"num_allocations": 2,
"num_threads": 1,
"model_id": ".elser_model_2",
"deployment_id": ".elser_model_2"
},
"chunking_settings": {
"strategy": "sentence",
"max_chunk_size": 250,
"sentence_overlap": 1
}
}Create an inference endpoint to perform an inference task with the elser service.
You can also deploy ELSER by using the Elasticsearch inference integration.
Your Elasticsearch deployment contains a preconfigured ELSER inference endpoint, you only need to create the enpoint using the API if you want to customize the settings.
The API request will automatically download and deploy the ELSER model if it isn't already downloaded.
You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Value is sparse_embedding.
The unique identifier of the inference endpoint.
Value is elser.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{elser_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"elser\",\n \"service_settings\": {\n \"num_allocations\": 1,\n \"num_threads\": 1\n }\n}"'{
"service": "elser",
"service_settings": {
"num_allocations": 1,
"num_threads": 1
}
}{
"service": "elser",
"service_settings": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 3,
"max_number_of_allocations": 10
},
"num_threads": 1
}
}{
"inference_id": "my-elser-model",
"task_type": "sparse_embedding",
"service": "elser",
"service_settings": {
"num_allocations": 1,
"num_threads": 1
},
"task_settings": {}
}Create an inference endpoint to perform an inference task with the googleaistudio service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are completion or text_embedding.
The unique identifier of the inference endpoint.
Value is googleaistudio.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{googleaistudio_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"googleaistudio\",\n \"service_settings\": {\n \"api_key\": \"api-key\",\n \"model_id\": \"model-id\"\n }\n}"'{
"service": "googleaistudio",
"service_settings": {
"api_key": "api-key",
"model_id": "model-id"
}
}Create an inference endpoint to perform an inference task with the googlevertexai service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are rerank or text_embedding.
The unique identifier of the inference endpoint.
Value is googlevertexai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{googlevertexai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"service_account_json\": \"service-account-json\",\n \"model_id\": \"model-id\",\n \"location\": \"location\",\n \"project_id\": \"project-id\"\n }\n}"'{
"service": "googlevertexai",
"service_settings": {
"service_account_json": "service-account-json",
"model_id": "model-id",
"location": "location",
"project_id": "project-id"
}
}{
"service": "googlevertexai",
"service_settings": {
"service_account_json": "service-account-json",
"project_id": "project-id"
}
}Create an inference endpoint to perform an inference task with the hugging_face service.
You must first create an inference endpoint on the Hugging Face endpoint page to get an endpoint URL.
Select the model you want to use on the new endpoint creation page (for example intfloat/e5-small-v2), then select the sentence embeddings task under the advanced configuration section.
Create the endpoint and copy the URL after the endpoint initialization has been finished.
The following models are recommended for the Hugging Face service:
all-MiniLM-L6-v2all-MiniLM-L12-v2all-mpnet-base-v2e5-base-v2e5-small-v2multilingual-e5-basemultilingual-e5-smallWhen you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Value is text_embedding.
The unique identifier of the inference endpoint.
Value is hugging_face.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{huggingface_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"hugging_face\",\n \"service_settings\": {\n \"api_key\": \"hugging-face-access-token\", \n \"url\": \"url-endpoint\" \n }\n}"'{
"service": "hugging_face",
"service_settings": {
"api_key": "hugging-face-access-token",
"url": "url-endpoint"
}
}Create an inference endpoint to perform an inference task with the jinaai service.
To review the available rerank models, refer to https://jina.ai/reranker.
To review the available text_embedding models, refer to the https://jina.ai/embeddings/.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are rerank or text_embedding.
The unique identifier of the inference endpoint.
Value is jinaai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{jinaai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v3\",\n \"api_key\": \"JinaAi-Api-key\"\n }\n}"'{
"service": "jinaai",
"service_settings": {
"model_id": "jina-embeddings-v3",
"api_key": "JinaAi-Api-key"
}
}{
"service": "jinaai",
"service_settings": {
"api_key": "JinaAI-Api-key",
"model_id": "jina-reranker-v2-base-multilingual"
},
"task_settings": {
"top_n": 10,
"return_documents": true
}
}Creates an inference endpoint to perform an inference task with the mistral service.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The task type.
The only valid task type for the model to perform is text_embedding.
Value is text_embedding.
The unique identifier of the inference endpoint.
Value is mistral.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{mistral_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"mistral\",\n \"service_settings\": {\n \"api_key\": \"Mistral-API-Key\",\n \"model\": \"mistral-embed\" \n }\n}"'{
"service": "mistral",
"service_settings": {
"api_key": "Mistral-API-Key",
"model": "mistral-embed"
}
}Create an inference endpoint to perform an inference task with the openai service or openai compatible APIs.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
NOTE: The chat_completion task type only supports streaming and only through the _stream API.
Values are chat_completion, completion, or text_embedding.
The unique identifier of the inference endpoint.
Value is openai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{openai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"openai\",\n \"service_settings\": {\n \"api_key\": \"OpenAI-API-Key\",\n \"model_id\": \"text-embedding-3-small\",\n \"dimensions\": 128\n }\n}"'{
"service": "openai",
"service_settings": {
"api_key": "OpenAI-API-Key",
"model_id": "text-embedding-3-small",
"dimensions": 128
}
}{
"service": "amazonbedrock",
"service_settings": {
"access_key": "AWS-access-key",
"secret_key": "AWS-secret-key",
"region": "us-east-1",
"provider": "amazontitan",
"model": "amazon.titan-text-premier-v1:0"
}
}Create an inference endpoint to perform an inference task with the voyageai service.
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are text_embedding or rerank.
The unique identifier of the inference endpoint.
Value is voyageai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{voyageai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"voyageai\",\n \"service_settings\": {\n \"model_id\": \"voyage-3-large\",\n \"dimensions\": 512\n }\n}"'{
"service": "voyageai",
"service_settings": {
"model_id": "voyage-3-large",
"dimensions": 512
}
}{
"service": "voyageai",
"service_settings": {
"model_id": "rerank-2"
}
}Create an inference endpoint to perform an inference task with the watsonxai service.
You need an IBM Cloud Databases for Elasticsearch deployment to use the watsonxai inference service.
You can provision one through the IBM catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The task type.
The only valid task type for the model to perform is text_embedding.
Value is text_embedding.
The unique identifier of the inference endpoint.
Value is watsonxai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{watsonx_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"watsonxai\",\n \"service_settings\": {\n \"api_key\": \"Watsonx-API-Key\", \n \"url\": \"Wastonx-URL\", \n \"model_id\": \"ibm/slate-30m-english-rtrvr\",\n \"project_id\": \"IBM-Cloud-ID\", \n \"api_version\": \"2024-03-14\"\n }\n}"'{
"service": "watsonxai",
"service_settings": {
"api_key": "Watsonx-API-Key",
"url": "Wastonx-URL",
"model_id": "ibm/slate-30m-english-rtrvr",
"project_id": "IBM-Cloud-ID",
"api_version": "2024-03-14"
}
}The unique identifier for the inference endpoint.
The amount of time to wait for the inference request to complete.
Query input.
The text on which you want to perform the inference task. It can be a single string or an array.
Inference endpoints for the completion task type currently only support a single string as input.
curl \
--request POST 'http://api.example.com/_inference/rerank/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": [\"luke\", \"like\", \"leia\", \"chewy\",\"r2d2\", \"star\", \"wars\"],\n \"query\": \"star wars main character\"\n}"'{
"input": ["luke", "like", "leia", "chewy","r2d2", "star", "wars"],
"query": "star wars main character"
}{
"rerank": [
{
"index": "2",
"relevance_score": "0.011597361",
"text": "leia"
},
{
"index": "0",
"relevance_score": "0.006338922",
"text": "luke"
},
{
"index": "5",
"relevance_score": "0.0016166499",
"text": "star"
},
{
"index": "4",
"relevance_score": "0.0011695103",
"text": "r2d2"
},
{
"index": "1",
"relevance_score": "5.614787E-4",
"text": "like"
},
{
"index": "6",
"relevance_score": "3.7850367E-4",
"text": "wars"
},
{
"index": "3",
"relevance_score": "1.2508839E-5",
"text": "chewy"
}
]
}The inference Id
Specifies the amount of time to wait for the inference request to complete.
Inference input. Either a string or an array of strings.
curl \
--request POST 'http://api.example.com/_inference/sparse_embedding/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\"\n}"'{
"input": "The sky above the port was the color of television tuned to a dead channel."
}{
"sparse_embedding": [
{
"port": 2.1259406,
"sky": 1.7073475,
"color": 1.6922266,
"dead": 1.6247464,
"television": 1.3525393,
"above": 1.2425821,
"tuned": 1.1440028,
"colors": 1.1218185,
"tv": 1.0111054,
"ports": 1.0067928,
"poem": 1.0042328,
"channel": 0.99471164,
"tune": 0.96235967,
"scene": 0.9020516
}
]
}The inference Id
Specifies the amount of time to wait for the inference request to complete.
Inference input. Either a string or an array of strings.
curl \
--request POST 'http://api.example.com/_inference/text_embedding/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\",\n \"task_settings\": {\n \"input_type\": \"ingest\"\n }\n}"'{
"input": "The sky above the port was the color of television tuned to a dead channel.",
"task_settings": {
"input_type": "ingest"
}
}{
"text_embedding": [
{
"embedding": [
{
0.018569946,
-0.036895752,
0.01486969,
-0.0045204163,
-0.04385376,
0.0075950623,
0.04260254,
-0.004005432,
0.007865906,
0.030792236,
-0.050476074,
0.011795044,
-0.011642456,
-0.010070801
}
]
}
]
}curl \
--request GET 'http://api.example.com/' \
--header "Authorization: $API_KEY"{
"name": "instance-0000000000",
"cluster_name": "my_test_cluster",
"cluster_uuid": "5QaxoN0pRZuOmWSxstBBwQ",
"version": {
"build_date": "2024-02-01T13:07:13.727175297Z",
"minimum_wire_compatibility_version": "7.17.0",
"build_hash": "6185ba65d27469afabc9bc951cded6c17c21e3f3",
"number": "8.12.1",
"lucene_version": "9.9.2",
"minimum_index_compatibility_version": "7.0.0",
"build_flavor": "default",
"build_snapshot": false,
"build_type": "docker"
},
"tagline": "You Know, for Search"
}Ingest APIs enable you to manage tasks and resources related to ingest pipelines and processors.
Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.
Comma-separated list of pipeline IDs to retrieve.
Wildcard (*) expressions are supported.
To get all ingest pipelines, omit this parameter or use *.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Return pipelines without their definitions (default: false)
curl \
--request GET 'http://api.example.com/_ingest/pipeline/{id}' \
--header "Authorization: $API_KEY"{
"my-pipeline-id" : {
"description" : "describe pipeline",
"version" : 123,
"processors" : [
{
"set" : {
"field" : "foo",
"value" : "bar"
}
}
]
}
}Changes made using this API take effect immediately.
ID of the ingest pipeline to create or update.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Required version for optimistic concurrency control for pipeline updates
Description of the ingest pipeline.
Processors to run immediately after a processor failure. Each processor supports a processor-level on_failure value. If a processor without an on_failure value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline's remaining processors.
Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
curl \
--request PUT 'http://api.example.com/_ingest/pipeline/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"description\" : \"My optional pipeline description\",\n \"processors\" : [\n {\n \"set\" : {\n \"description\" : \"My optional processor description\",\n \"field\": \"my-keyword-field\",\n \"value\": \"foo\"\n }\n }\n ]\n}"'{
"description" : "My optional pipeline description",
"processors" : [
{
"set" : {
"description" : "My optional processor description",
"field": "my-keyword-field",
"value": "foo"
}
}
]
}{
"description" : "My optional pipeline description",
"processors" : [
{
"set" : {
"description" : "My optional processor description",
"field": "my-keyword-field",
"value": "foo"
}
}
],
"_meta": {
"reason": "set my-keyword-field to foo",
"serialization": {
"class": "MyPipeline",
"id": 10
}
}
}Delete one or more ingest pipelines.
Pipeline ID or wildcard expression of pipeline IDs used to limit the request.
To delete all ingest pipelines in a cluster, use a value of *.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE 'http://api.example.com/_ingest/pipeline/{id}' \
--header "Authorization: $API_KEY"Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Return pipelines without their definitions (default: false)
curl \
--request GET 'http://api.example.com/_ingest/pipeline' \
--header "Authorization: $API_KEY"{
"my-pipeline-id" : {
"description" : "describe pipeline",
"version" : 123,
"processors" : [
{
"set" : {
"field" : "foo",
"value" : "bar"
}
}
]
}
}Extract structured fields out of a single text field within a document. You must choose which field to extract matched fields from, as well as the grok pattern you expect will match. A grok pattern is like a regular expression that supports aliased expressions that can be reused.
curl \
--request GET 'http://api.example.com/_ingest/processor/grok' \
--header "Authorization: $API_KEY"Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.
If true, the response includes output data for each processor in the executed pipeline.
curl \
--request GET 'http://api.example.com/_ingest/pipeline/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\" :\n {\n \"description\": \"_description\",\n \"processors\": [\n {\n \"set\" : {\n \"field\" : \"field2\",\n \"value\" : \"_value\"\n }\n }\n ]\n },\n \"docs\": [\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"pipeline" :
{
"description": "_description",
"processors": [
{
"set" : {
"field" : "field2",
"value" : "_value"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "bar"
}
},
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "bar"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.187Z"
}
}
},
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "rab"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.188Z"
}
}
}
]
}Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.
If true, the response includes output data for each processor in the executed pipeline.
curl \
--request POST 'http://api.example.com/_ingest/pipeline/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\" :\n {\n \"description\": \"_description\",\n \"processors\": [\n {\n \"set\" : {\n \"field\" : \"field2\",\n \"value\" : \"_value\"\n }\n }\n ]\n },\n \"docs\": [\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"pipeline" :
{
"description": "_description",
"processors": [
{
"set" : {
"field" : "field2",
"value" : "_value"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "bar"
}
},
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "bar"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.187Z"
}
}
},
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "rab"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.188Z"
}
}
}
]
}Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.
The pipeline to test.
If you don't specify a pipeline in the request body, this parameter is required.
If true, the response includes output data for each processor in the executed pipeline.
curl \
--request GET 'http://api.example.com/_ingest/pipeline/{id}/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\" :\n {\n \"description\": \"_description\",\n \"processors\": [\n {\n \"set\" : {\n \"field\" : \"field2\",\n \"value\" : \"_value\"\n }\n }\n ]\n },\n \"docs\": [\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"pipeline" :
{
"description": "_description",
"processors": [
{
"set" : {
"field" : "field2",
"value" : "_value"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "bar"
}
},
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "bar"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.187Z"
}
}
},
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "rab"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.188Z"
}
}
}
]
}Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.
The pipeline to test.
If you don't specify a pipeline in the request body, this parameter is required.
If true, the response includes output data for each processor in the executed pipeline.
curl \
--request POST 'http://api.example.com/_ingest/pipeline/{id}/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"pipeline\" :\n {\n \"description\": \"_description\",\n \"processors\": [\n {\n \"set\" : {\n \"field\" : \"field2\",\n \"value\" : \"_value\"\n }\n }\n ]\n },\n \"docs\": [\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_index\": \"index\",\n \"_id\": \"id\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"pipeline" :
{
"description": "_description",
"processors": [
{
"set" : {
"field" : "field2",
"value" : "_value"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "bar"
}
},
{
"_index": "index",
"_id": "id",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "bar"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.187Z"
}
}
},
{
"doc": {
"_id": "id",
"_index": "index",
"_version": "-3",
"_source": {
"field2": "_value",
"foo": "rab"
},
"_ingest": {
"timestamp": "2017-05-04T22:30:03.188Z"
}
}
}
]
}Licensing APIs enable you to manage your licenses.
Get information about your Elastic license including its type, its status, when it was issued, and when it expires.
If the master node is generating a new cluster state, the get license API may return a 404 Not Found response.
If you receive an unexpected 404 response after cluster startup, wait a short period and retry the request.
If true, this parameter returns enterprise for Enterprise license types. If false, this parameter returns platinum for both platinum and enterprise license types. This behavior is maintained for backwards compatibility.
This parameter is deprecated and will always be set to true in 8.x.
Specifies whether to retrieve local information. The default value is false, which means the information is retrieved from the master node.
curl \
--request GET 'http://api.example.com/_license' \
--header "Authorization: $API_KEY"{
"license" : {
"status" : "active",
"uid" : "cbff45e7-c553-41f7-ae4f-9205eabd80xx",
"type" : "trial",
"issue_date" : "2018-10-20T22:05:12.332Z",
"issue_date_in_millis" : 1540073112332,
"expiry_date" : "2018-11-19T22:05:12.332Z",
"expiry_date_in_millis" : 1542665112332,
"max_nodes" : 1000,
"max_resource_units" : null,
"issued_to" : "test",
"issuer" : "elasticsearch",
"start_date_in_millis" : -1
}
}Logstash APIs enable you to manage pipelines that are used by Logstash Central Management.
Get pipelines that are used for Logstash Central Management.
A comma-separated list of pipeline identifiers.
curl \
--request GET 'http://api.example.com/_logstash/pipeline/{id}' \
--header "Authorization: $API_KEY"{
"my_pipeline": {
"description": "Sample pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\\n filter { grok {} }\\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes": "1gb",
"queue.checkpoint.writes": 1024
}
}
}Create a pipeline that is used for Logstash Central Management. If the specified pipeline exists, it is replaced.
An identifier for the pipeline.
A description of the pipeline. This description is not used by Elasticsearch or Logstash.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
The configuration for the pipeline.
The user who last updated the pipeline.
curl \
--request PUT 'http://api.example.com/_logstash/pipeline/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"description\": \"Sample pipeline for illustration purposes\",\n \"last_modified\": \"2021-01-02T02:50:51.250Z\",\n \"pipeline_metadata\": {\n \"type\": \"logstash_pipeline\",\n \"version\": 1\n },\n \"username\": \"elastic\",\n \"pipeline\": \"input {}\\\\n filter { grok {} }\\\\n output {}\",\n \"pipeline_settings\": {\n \"pipeline.workers\": 1,\n \"pipeline.batch.size\": 125,\n \"pipeline.batch.delay\": 50,\n \"queue.type\": \"memory\",\n \"queue.max_bytes\": \"1gb\",\n \"queue.checkpoint.writes\": 1024\n }\n}"'{
"description": "Sample pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": 1
},
"username": "elastic",
"pipeline": "input {}\\n filter { grok {} }\\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes": "1gb",
"queue.checkpoint.writes": 1024
}
}Delete a pipeline that is used for Logstash Central Management. If the request succeeds, you receive an empty response with an appropriate status code.
An identifier for the pipeline.
curl \
--request DELETE 'http://api.example.com/_logstash/pipeline/{id}' \
--header "Authorization: $API_KEY"Get pipelines that are used for Logstash Central Management.
curl \
--request GET 'http://api.example.com/_logstash/pipeline' \
--header "Authorization: $API_KEY"{
"my_pipeline": {
"description": "Sample pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\\n filter { grok {} }\\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes": "1gb",
"queue.checkpoint.writes": 1024
}
}
}A job can be opened and closed multiple times throughout its lifecycle. A closed job cannot receive data or perform analysis operations, but you can still explore and navigate results. When you close a job, it runs housekeeping tasks such as pruning the model history, flushing buffers, calculating final results and persisting the model snapshots. Depending upon the size of the job, it could take several minutes to close and the equivalent time to re-open. After it is closed, the job has a minimal overhead on the cluster except for maintaining its meta data. Therefore it is a best practice to close jobs that are no longer required to process data. If you close an anomaly detection job whose datafeed is running, the request first tries to stop the datafeed. This behavior is equivalent to calling stop datafeed API with the same timeout and force parameters as the close job request. When a datafeed that has a specified end date stops, it automatically closes its associated job.
Identifier for the anomaly detection job. It can be a job identifier, a group name, or a wildcard expression. You can close multiple anomaly detection jobs in a single API request by using a group name, a comma-separated list of jobs, or a wildcard expression. You can close all jobs by using _all or by specifying * as the job identifier.
Specifies what to do when the request: contains wildcard expressions and there are no jobs that match; contains the _all string or no identifiers and there are no matches; or contains wildcard expressions and there are only partial matches. By default, it returns an empty jobs array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Use to close a failed job, or to forcefully close a job which has not responded to its initial close request; the request returns without performing the associated actions such as flushing buffers and persisting the model snapshots.
If you want the job to be in a consistent state after the close job API returns, do not set to true. This parameter should be used only in situations where the job has already failed or where you are not interested in results the job might have recently produced or might produce in the future.
Controls the time to wait until a job has closed.
Refer to the description for the allow_no_match query parameter.
Refer to the descriptiion for the force query parameter.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_close' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_no_match":true,"force":true,"timeout":"string"}'{
"closed": true
}A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.
curl \
--request GET 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'A string that uniquely identifies a calendar.
An array of anomaly detection job identifiers.
A description of the calendar.
curl \
--request PUT 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"job_ids":["string"],"description":"string"}'A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.
curl \
--request POST 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'Remove all scheduled events from a calendar, then delete it.
A string that uniquely identifies a calendar.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}A string that uniquely identifies a calendar.
Identifier for the scheduled event. You can obtain this identifier by using the get calendar events API.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}/events/{event_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}A string that uniquely identifies a calendar.
An identifier for the anomaly detection jobs. It can be a job identifier, a group name, or a comma-separated list of jobs or groups.
curl \
--request PUT 'http://api.example.com/_ml/calendars/{calendar_id}/jobs/{job_id}' \
--header "Authorization: $API_KEY"A string that uniquely identifies a calendar.
An identifier for the anomaly detection jobs. It can be a job identifier, a group name, or a comma-separated list of jobs or groups.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}/jobs/{job_id}' \
--header "Authorization: $API_KEY"{
"calendar_id": "planned-outages",
"job_ids": []
}You can get information for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get information for all datafeeds by using _all, by specifying * as the
<feed_id>, or by omitting the <feed_id>.
This API returns a maximum of 10,000 datafeeds.
Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value is true, which returns an empty datafeeds array
when there are no matches and the subset of results when there are
partial matches. If this parameter is false, the request returns a
404 status code when there are no matches or only partial matches.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY"Datafeeds retrieve data from Elasticsearch for analysis by an anomaly detection job.
You can associate only one datafeed with each anomaly detection job.
The datafeed contains a query that runs at a defined interval (frequency).
If you are concerned about delayed data, you can add a delay (query_delay') at each interval.
By default, the datafeed uses the following query:{"match_all": {"boost": 1}}`.
When Elasticsearch security features are enabled, your datafeed remembers which roles the user who created it had
at the time of creation and runs the query using those same roles. If you provide secondary authorization headers,
those credentials are used instead.
You must use Kibana, this API, or the create anomaly detection jobs API to create a datafeed. Do not add a datafeed
directly to the .ml-config index. Do not give users write privileges on the .ml-config index.
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the _all
string or when no indices are specified.
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, concrete, expanded, or aliased indices are ignored when frozen.
If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
If a real-time datafeed has never seen any data (including during any initial training period), it automatically
stops and closes the associated job after this many real-time searches return no documents. In other words,
it stops after frequency times max_empty_searches of real-time operation. If not set, a datafeed with no
end time that sees no data remains started until it is explicitly stopped. By default, it is not set.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.
The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations.
The maximum value is the value of index.max_result_window, which is 10,000 by default.
curl \
--request PUT 'http://api.example.com/_ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0,"headers":{}}'A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Use to forcefully delete a started datafeed; this method is quicker than stopping and deleting the datafeed.
curl \
--request DELETE 'http://api.example.com/_ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}A string that uniquely identifies a filter.
curl \
--request GET 'http://api.example.com/_ml/filters/{filter_id}' \
--header "Authorization: $API_KEY"A filter contains a list of strings. It can be used by one or more anomaly detection jobs.
Specifically, filters are referenced in the custom_rules property of detector configuration objects.
A string that uniquely identifies a filter.
A description of the filter.
The items of the filter. A wildcard * can be used at the beginning or the end of an item.
Up to 10000 items are allowed in each filter.
curl \
--request PUT 'http://api.example.com/_ml/filters/{filter_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"description":"string","items":["string"]}'If an anomaly detection job references the filter, you cannot delete the filter. You must update or delete the job before you can delete the filter.
A string that uniquely identifies a filter.
curl \
--request DELETE 'http://api.example.com/_ml/filters/{filter_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}You can get information for multiple anomaly detection jobs in a single API
request by using a group name, a comma-separated list of jobs, or a wildcard
expression. You can get information for all anomaly detection jobs by using
_all, by specifying * as the <job_id>, or by omitting the <job_id>.
Identifier for the anomaly detection job. It can be a job identifier, a group name, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.
Specifies what to do when the request:
The default value is true, which returns an empty jobs array when
there are no matches and the subset of results when there are partial
matches. If this parameter is false, the request returns a 404 status
code when there are no matches or only partial matches.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}' \
--header "Authorization: $API_KEY"If you include a datafeed_config, you must have read index privileges on the source index.
If you include a datafeed_config but do not provide a query, the datafeed uses {"match_all": {"boost": 1}}.
The identifier for the anomaly detection job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the
_all string or when no indices are specified.
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values. Valid values are:
all: Match any data stream or index, including hidden ones.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard patterns are not accepted.open: Match open, non-hidden indices. Also matches any non-hidden data stream.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, concrete, expanded or aliased indices are ignored when frozen.
Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node. By default, if a machine learning node with capacity to run the job cannot immediately be found, the open anomaly detection jobs API returns an error. However, this is also subject to the cluster-wide xpack.ml.max_lazy_ml_nodes setting. If this option is set to true, the open anomaly detection jobs API does not return an error and the job waits in the opening state until sufficient machine learning node capacity is available.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Custom metadata about the job
Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job. Valid values range from 0 to model_snapshot_retention_days.
A description of the job.
A list of job groups. A job can belong to no groups or many.
Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job. By default, snapshots ten days older than the newest snapshot are deleted.
Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen. The default value is the longer of 30 days or 100 bucket spans.
Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained. Annotations generated by the system also count as results for retention purposes; they are deleted after the same number of days as results. Annotations added by users are retained forever.
curl \
--request PUT 'http://api.example.com/_ml/anomaly_detectors/{job_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analysis_config\": {\n \"bucket_span\": \"15m\",\n \"detectors\": [\n {\n \"detector_description\": \"Sum of bytes\",\n \"function\": \"sum\",\n \"field_name\": \"bytes\"\n }\n ]\n },\n \"data_description\": {\n \"time_field\": \"timestamp\",\n \"time_format\": \"epoch_ms\"\n },\n \"analysis_limits\": {\n \"model_memory_limit\": \"11MB\"\n },\n \"model_plot_config\": {\n \"enabled\": true,\n \"annotations_enabled\": true\n },\n \"results_index_name\": \"test-job1\",\n \"datafeed_config\": {\n \"indices\": [\n \"kibana_sample_data_logs\"\n ],\n \"query\": {\n \"bool\": {\n \"must\": [\n {\n \"match_all\": {}\n }\n ]\n }\n },\n \"runtime_mappings\": {\n \"hour_of_day\": {\n \"type\": \"long\",\n \"script\": {\n \"source\": \"emit(doc['timestamp'].value.getHour());\"\n }\n }\n },\n \"datafeed_id\": \"datafeed-test-job1\"\n }\n}"'{
"analysis_config": {
"bucket_span": "15m",
"detectors": [
{
"detector_description": "Sum of bytes",
"function": "sum",
"field_name": "bytes"
}
]
},
"data_description": {
"time_field": "timestamp",
"time_format": "epoch_ms"
},
"analysis_limits": {
"model_memory_limit": "11MB"
},
"model_plot_config": {
"enabled": true,
"annotations_enabled": true
},
"results_index_name": "test-job1",
"datafeed_config": {
"indices": [
"kibana_sample_data_logs"
],
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"runtime_mappings": {
"hour_of_day": {
"type": "long",
"script": {
"source": "emit(doc['timestamp'].value.getHour());"
}
}
},
"datafeed_id": "datafeed-test-job1"
}
}{
"job_id": "test-job1",
"job_type": "anomaly_detector",
"job_version": "8.4.0",
"create_time": 1656087283340,
"datafeed_config": {
"datafeed_id": "datafeed-test-job1",
"job_id": "test-job1",
"authorization": {
"roles": [
"superuser"
]
},
"query_delay": "61499ms",
"chunking_config": {
"mode": "auto"
},
"indices_options": {
"expand_wildcards": [
"open"
],
"ignore_unavailable": false,
"allow_no_indices": true,
"ignore_throttled": true
},
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"indices": [
"kibana_sample_data_logs"
],
"scroll_size": 1000,
"delayed_data_check_config": {
"enabled": true
},
"runtime_mappings": {
"hour_of_day": {
"type": "long",
"script": {
"source": "emit(doc['timestamp'].value.getHour());"
}
}
}
},
"analysis_config": {
"bucket_span": "15m",
"detectors": [
{
"detector_description": "Sum of bytes",
"function": "sum",
"field_name": "bytes",
"detector_index": 0
}
],
"influencers": [],
"model_prune_window": "30d"
},
"analysis_limits": {
"model_memory_limit": "11mb",
"categorization_examples_limit": 4
},
"data_description": {
"time_field": "timestamp",
"time_format": "epoch_ms"
},
"model_plot_config": {
"enabled": true,
"annotations_enabled": true
},
"model_snapshot_retention_days": 10,
"daily_model_snapshot_retention_after_days": 1,
"results_index_name": "custom-test-job1",
"allow_lazy_open": false
}All job configuration, model state and results are deleted. It is not currently possible to delete multiple jobs using wildcards or a comma separated list. If you delete a job that has a datafeed, the request first tries to delete the datafeed. This behavior is equivalent to calling the delete datafeed API with the same timeout and force parameters as the delete job request.
Identifier for the anomaly detection job.
Use to forcefully delete an opened job; this method is quicker than closing and deleting the job.
Specifies whether annotations that have been added by the user should be deleted along with any auto-generated annotations when the job is reset.
Specifies whether the request should return immediately or wait until the job deletion completes.
curl \
--request DELETE 'http://api.example.com/_ml/anomaly_detectors/{job_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}{
"task": "oTUltX4IQMOUUVeiohTt8A:39"
}Make an estimation of the memory usage for an anomaly detection job model. The estimate is based on analysis configuration details for the job and cardinality estimates for the fields it references.
Estimates of the highest cardinality in a single bucket that is observed
for influencer fields over the time period that the job analyzes data.
To produce a good answer, values must be provided for all influencer
fields. Providing values for fields that are not listed as influencers
has no effect on the estimation.
Estimates of the cardinality that is observed for fields over the whole
time period that the job analyzes data. To produce a good answer, values
must be provided for fields referenced in the by_field_name,
over_field_name and partition_field_name of any detectors. Providing
values for other fields has no effect on the estimation. It can be
omitted from the request if no detectors have a by_field_name,
over_field_name or partition_field_name.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/_estimate_model_memory' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analysis_config\": {\n \"bucket_span\": \"5m\",\n \"detectors\": [\n {\n \"function\": \"sum\",\n \"field_name\": \"bytes\",\n \"by_field_name\": \"status\",\n \"partition_field_name\": \"app\"\n }\n ],\n \"influencers\": [\n \"source_ip\",\n \"dest_ip\"\n ]\n },\n \"overall_cardinality\": {\n \"status\": 10,\n \"app\": 50\n },\n \"max_bucket_cardinality\": {\n \"source_ip\": 300,\n \"dest_ip\": 30\n }\n}"'{
"analysis_config": {
"bucket_span": "5m",
"detectors": [
{
"function": "sum",
"field_name": "bytes",
"by_field_name": "status",
"partition_field_name": "app"
}
],
"influencers": [
"source_ip",
"dest_ip"
]
},
"overall_cardinality": {
"status": 10,
"app": 50
},
"max_bucket_cardinality": {
"source_ip": 300,
"dest_ip": 30
}
}{
"model_memory_estimate": "21mb"
}The flush jobs API is only applicable when sending data for analysis using the post data API. Depending on the content of the buffer, then it might additionally calculate new results. Both flush and close operations are similar, however the flush is more efficient if you are expecting to send more data for analysis. When flushing, the job remains open and is available to continue analyzing data. A close operation additionally prunes and persists the model state to disk and the job must be opened again before analyzing further data.
Identifier for the anomaly detection job.
Specifies to advance to a particular time value. Results are generated and the model is updated for data from the specified time interval.
If true, calculates the interim results for the most recent bucket or all buckets within the latency period.
When used in conjunction with calc_interim and start, specifies the
range of buckets on which to calculate interim results.
Specifies to skip to a particular time value. Results are not generated and the model is not updated for data from the specified time interval.
When used in conjunction with calc_interim, specifies the range of
buckets on which to calculate interim results.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Refer to the description for the calc_interim query parameter.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_flush' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"":"string","calc_interim":true}'A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.
Specifies to get events with timestamps earlier than this time.
Skips the specified number of events.
Specifies to get events for a specific anomaly detection job identifier or job group. It must be used with a calendar identifier of _all or *.
Specifies the maximum number of events to obtain.
Specifies to get events with timestamps after this time.
curl \
--request GET 'http://api.example.com/_ml/calendars/{calendar_id}/events' \
--header "Authorization: $API_KEY"A string that uniquely identifies a calendar.
curl \
--request POST 'http://api.example.com/_ml/calendars/{calendar_id}/events' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"events":[{"calendar_id":"string","event_id":"string","description":"string","":"string","skip_result":true,"skip_model_update":true,"force_time_shift":42.0}]}'curl \
--request GET 'http://api.example.com/_ml/calendars' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'curl \
--request POST 'http://api.example.com/_ml/calendars' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'You can get statistics for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get statistics for all datafeeds by using _all, by specifying * as the
<feed_id>, or by omitting the <feed_id>. If the datafeed is stopped, the
only information you receive is the datafeed_id and the state.
This API returns a maximum of 10,000 datafeeds.
Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value is true, which returns an empty datafeeds array
when there are no matches and the subset of results when there are
partial matches. If this parameter is false, the request returns a
404 status code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_stats' \
--header "Authorization: $API_KEY"You can get statistics for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get statistics for all datafeeds by using _all, by specifying * as the
<feed_id>, or by omitting the <feed_id>. If the datafeed is stopped, the
only information you receive is the datafeed_id and the state.
This API returns a maximum of 10,000 datafeeds.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value is true, which returns an empty datafeeds array
when there are no matches and the subset of results when there are
partial matches. If this parameter is false, the request returns a
404 status code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/_stats' \
--header "Authorization: $API_KEY"You can get information for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get information for all datafeeds by using _all, by specifying * as the
<feed_id>, or by omitting the <feed_id>.
This API returns a maximum of 10,000 datafeeds.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value is true, which returns an empty datafeeds array
when there are no matches and the subset of results when there are
partial matches. If this parameter is false, the request returns a
404 status code when there are no matches or only partial matches.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/datafeeds' \
--header "Authorization: $API_KEY"curl \
--request GET 'http://api.example.com/_ml/filters' \
--header "Authorization: $API_KEY"Specifies what to do when the request:
If true, the API returns an empty jobs array when
there are no matches and the subset of results when there are partial
matches. If false, the API returns a 404 status
code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/_stats' \
--header "Authorization: $API_KEY"Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.
Specifies what to do when the request:
If true, the API returns an empty jobs array when
there are no matches and the subset of results when there are partial
matches. If false, the API returns a 404 status
code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_stats' \
--header "Authorization: $API_KEY"You can get information for multiple anomaly detection jobs in a single API
request by using a group name, a comma-separated list of jobs, or a wildcard
expression. You can get information for all anomaly detection jobs by using
_all, by specifying * as the <job_id>, or by omitting the <job_id>.
Specifies what to do when the request:
The default value is true, which returns an empty jobs array when
there are no matches and the subset of results when there are partial
matches. If this parameter is false, the request returns a 404 status
code when there are no matches or only partial matches.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors' \
--header "Authorization: $API_KEY"Retrievs overall bucket results that summarize the bucket results of multiple anomaly detection jobs.
The overall_score is calculated by combining the scores of all the
buckets within the overall bucket span. First, the maximum
anomaly_score per anomaly detection job in the overall bucket is
calculated. Then the top_n of those scores are averaged to result in
the overall_score. This means that you can fine-tune the
overall_score so that it is more or less sensitive to the number of
jobs that detect an anomaly at the same time. For example, if you set
top_n to 1, the overall_score is the maximum bucket score in the
overall bucket. Alternatively, if you set top_n to the number of jobs,
the overall_score is high only when all jobs detect anomalies in that
overall bucket. If you set the bucket_span parameter (to a value
greater than its default), the overall_score is the maximum
overall_score of the overall buckets that have a span equal to the
jobs' largest bucket span.
Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs or groups, or a wildcard expression.
You can summarize the bucket results for all anomaly detection jobs by
using _all or by specifying * as the <job_id>.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the request returns an empty jobs array when there are no
matches and the subset of results when there are partial matches. If this
parameter is false, the request returns a 404 status code when there
are no matches or only partial matches.
The span of the overall buckets. Must be greater or equal to the largest bucket span of the specified anomaly detection jobs, which is the default value.
By default, an overall bucket has a span equal to the largest bucket span
of the specified anomaly detection jobs. To override that behavior, use
the optional bucket_span parameter.
Returns overall buckets with timestamps earlier than this time.
If true, the output excludes interim results.
Returns overall buckets with overall scores greater than or equal to this value.
Returns overall buckets with timestamps after this time.
The number of top anomaly detection job bucket scores to be used in the
overall_score calculation.
Refer to the description for the allow_no_match query parameter.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Refer to the description for the exclude_interim query parameter.
Refer to the description for the overall_score query parameter.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Refer to the description for the top_n query parameter.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/overall_buckets' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_no_match":true,"bucket_span":"string","":"string","exclude_interim":true,"overall_score":42.0,"top_n":42.0}'Retrievs overall bucket results that summarize the bucket results of multiple anomaly detection jobs.
The overall_score is calculated by combining the scores of all the
buckets within the overall bucket span. First, the maximum
anomaly_score per anomaly detection job in the overall bucket is
calculated. Then the top_n of those scores are averaged to result in
the overall_score. This means that you can fine-tune the
overall_score so that it is more or less sensitive to the number of
jobs that detect an anomaly at the same time. For example, if you set
top_n to 1, the overall_score is the maximum bucket score in the
overall bucket. Alternatively, if you set top_n to the number of jobs,
the overall_score is high only when all jobs detect anomalies in that
overall bucket. If you set the bucket_span parameter (to a value
greater than its default), the overall_score is the maximum
overall_score of the overall buckets that have a span equal to the
jobs' largest bucket span.
Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs or groups, or a wildcard expression.
You can summarize the bucket results for all anomaly detection jobs by
using _all or by specifying * as the <job_id>.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the request returns an empty jobs array when there are no
matches and the subset of results when there are partial matches. If this
parameter is false, the request returns a 404 status code when there
are no matches or only partial matches.
The span of the overall buckets. Must be greater or equal to the largest bucket span of the specified anomaly detection jobs, which is the default value.
By default, an overall bucket has a span equal to the largest bucket span
of the specified anomaly detection jobs. To override that behavior, use
the optional bucket_span parameter.
Returns overall buckets with timestamps earlier than this time.
If true, the output excludes interim results.
Returns overall buckets with overall scores greater than or equal to this value.
Returns overall buckets with timestamps after this time.
The number of top anomaly detection job bucket scores to be used in the
overall_score calculation.
Refer to the description for the allow_no_match query parameter.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Refer to the description for the exclude_interim query parameter.
Refer to the description for the overall_score query parameter.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Refer to the description for the top_n query parameter.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/overall_buckets' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_no_match":true,"bucket_span":"string","":"string","exclude_interim":true,"overall_score":42.0,"top_n":42.0}'An anomaly detection job must be opened to be ready to receive and analyze data. It can be opened and closed multiple times throughout its lifecycle. When you open a new job, it starts with an empty model. When you open an existing job, the most recent model state is automatically loaded. The job is ready to resume its analysis from where it left off, once new data is received.
Identifier for the anomaly detection job.
Controls the time to wait until a job has opened.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_open' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"timeout\": \"35m\"\n}"'{
"timeout": "35m"
}{
"opened": true,
"node": "node-1"
}This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters. NOTE: If you use this path parameter, you cannot provide datafeed or anomaly detection job configuration details in the request body.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters. NOTE: If you use this path parameter, you cannot provide datafeed or anomaly detection job configuration details in the request body.
curl \
--request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
curl \
--request POST 'http://api.example.com/_ml/datafeeds/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'All model state and results are deleted. The job is ready to start over as if it had just been created. It is not currently possible to reset multiple jobs using wildcards or a comma separated list.
The ID of the job to reset.
Should this request wait until the operation has completed before returning.
Specifies whether annotations that have been added by the user should be deleted along with any auto-generated annotations when the job is reset.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_reset' \
--header "Authorization: $API_KEY"A datafeed must be started in order to retrieve data from Elasticsearch. A datafeed can be started and stopped multiple times throughout its lifecycle.
Before you can start a datafeed, the anomaly detection job must be open. Otherwise, an error occurs.
If you restart a stopped datafeed, it continues processing input data from the next millisecond after it was stopped. If new data was indexed for that exact millisecond between stopping and starting, it will be ignored.
When Elasticsearch security features are enabled, your datafeed remembers which roles the last user to create or update it had at the time of creation or update and runs the query using those same roles. If you provided secondary authorization headers when you created or updated the datafeed, those credentials are used instead.
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
The time that the datafeed should end, which can be specified by using one of the following formats:
2017-01-22T06:00:00.000Z2017-01-22T06:00:00+00:001485061200000Date-time arguments using either of the ISO 8601 formats must have a time zone designator, where Z is accepted
as an abbreviation for UTC time. When a URL is expected (for example, in browsers), the + used in time zone
designators must be encoded as %2B.
The end time value is exclusive. If you do not specify an end time, the datafeed
runs continuously.
The time that the datafeed should begin, which can be specified by using the same formats as the end parameter.
This value is inclusive.
If you do not specify a start time and the datafeed is associated with a new anomaly detection job, the analysis
starts from the earliest time for which data is available.
If you restart a stopped datafeed and specify a start value that is earlier than the timestamp of the latest
processed record, the datafeed continues from 1 millisecond after the timestamp of the latest processed record.
Specifies the amount of time to wait until a datafeed starts.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
curl \
--request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_start' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"":"string","timeout":"string"}'A datafeed that is stopped ceases to retrieve data from Elasticsearch. A datafeed can be started and stopped multiple times throughout its lifecycle.
Identifier for the datafeed. You can stop multiple datafeeds in a single API request by using a comma-separated
list of datafeeds or a wildcard expression. You can close all datafeeds by using _all or by specifying * as
the identifier.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
If true, the datafeed is stopped forcefully.
Specifies the amount of time to wait until a datafeed stops.
Refer to the description for the allow_no_match query parameter.
Refer to the description for the force query parameter.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
curl \
--request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_stop' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_no_match":true,"force":true,"timeout":"string"}'You must stop and start the datafeed for the changes to be applied. When Elasticsearch security features are enabled, your datafeed remembers which roles the user who updated it had at the time of the update and runs the query using those same roles. If you provide secondary authorization headers, those credentials are used instead.
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the
_all string or when no indices are specified.
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values. Valid values are:
all: Match any data stream or index, including hidden ones.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard patterns are not accepted.open: Match open, non-hidden indices. Also matches any non-hidden data stream.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, concrete, expanded or aliased indices are ignored when frozen.
If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
An array of index names. Wildcards are supported. If any of the indices are in remote clusters, the machine
learning nodes must have the remote_cluster_client role.
If a real-time datafeed has never seen any data (including during any initial training period), it automatically
stops and closes the associated job after this many real-time searches return no documents. In other words,
it stops after frequency times max_empty_searches of real-time operation. If not set, a datafeed with no
end time that sees no data remains started until it is explicitly stopped. By default, it is not set.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.
The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations.
The maximum value is the value of index.max_result_window.
curl \
--request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":["string"],"indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0}'Updates the description of a filter, adds items, or removes items from the list.
A string that uniquely identifies a filter.
The items to add to the filter.
A description for the filter.
The items to remove from the filter.
curl \
--request POST 'http://api.example.com/_ml/filters/{filter_id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"add_items":["string"],"description":"string","remove_items":["string"]}'Updates certain properties of an anomaly detection job.
Identifier for the job.
Advanced configuration option. Specifies whether this job can open when
there is insufficient machine learning node capacity for it to be
immediately assigned to a node. If false and a machine learning node
with capacity to run the job cannot immediately be found, the open
anomaly detection jobs API returns an error. However, this is also
subject to the cluster-wide xpack.ml.max_lazy_ml_nodes setting. If this
option is set to true, the open anomaly detection jobs API does not
return an error and the job waits in the opening state until sufficient
machine learning node capacity is available.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Advanced configuration option. Contains custom meta data about the job. For example, it can contain custom URL information as shown in Adding custom URLs to machine learning results.
A description of the job.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Advanced configuration option, which affects the automatic removal of old
model snapshots for this job. It specifies a period of time (in days)
after which only the first snapshot per day is retained. This period is
relative to the timestamp of the most recent snapshot for this job. Valid
values range from 0 to model_snapshot_retention_days. For jobs created
before version 7.8.0, the default value matches
model_snapshot_retention_days.
Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job.
Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen.
Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained.
A list of job groups. A job can belong to no groups or many.
An array of detector update objects.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_lazy_open":true,"analysis_limits":{"model_memory_limit":"string"},"background_persist_interval":"string","custom_settings":{"additionalProperty1":{},"additionalProperty2":{}},"categorization_filters":["string"],"description":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_prune_window":"string","daily_model_snapshot_retention_after_days":42.0,"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_retention_days":42.0,"groups":["string"],"detectors":[{"detector_index":42.0,"description":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{"applies_to":"actual","operator":"gt","value":42.0}],"scope":{"additionalProperty1":{"filter_id":"string","filter_type":"include"},"additionalProperty2":{"filter_id":"string","filter_type":"include"}}}]}],"per_partition_categorization":{"enabled":true,"stop_on_warn":true}}'You can get information for multiple data frame analytics jobs in a single API request by using a comma-separated list of data frame analytics jobs or a wildcard expression.
Identifier for the data frame analytics job. If you do not specify this option, the API returns information for the first hundred data frame analytics jobs.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value returns an empty data_frame_analytics array when there
are no matches and the subset of results when there are partial matches.
If this parameter is false, the request returns a 404 status code when
there are no matches or only partial matches.
Skips the specified number of data frame analytics jobs.
Specifies the maximum number of data frame analytics jobs to obtain.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY"This API creates a data frame analytics job that performs an analysis on the
source indices and stores the outcome in a destination index.
By default, the query used in the source configuration is {"match_all": {}}.
If the destination index does not exist, it is created automatically when you start the job.
If you supply only a subset of the regression or classification parameters, hyperparameter optimization occurs. It determines a value for each of the undefined parameters.
Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Specifies whether this job can start when there is insufficient machine
learning node capacity for it to be immediately assigned to a node. If
set to false and a machine learning node with capacity to run the job
cannot be immediately found, the API returns an error. If set to true,
the API does not return an error; the job waits in the starting state
until sufficient machine learning node capacity is available. This
behavior is also affected by the cluster-wide
xpack.ml.max_lazy_ml_nodes setting.
A description of the job.
The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.
The approximate maximum amount of memory resources that are permitted for
analytical processing. If your elasticsearch.yml file contains an
xpack.ml.max_model_memory_limit setting, an error occurs when you try
to create data frame analytics jobs that have model_memory_limit values
greater than that setting.
curl \
--request PUT 'http://api.example.com/_ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"allow_lazy_start":true,"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{"feature_name":"string","field":"string","frequency_map":{}},"multi_encoding":{"processors":[42.0]},"n_gram_encoding":{"feature_prefix":"string","field":"string","length":42.0,"n_grams":[42.0],"start":42.0,"custom":true},"one_hot_encoding":{"field":"string","hot_map":"string"},"target_mean_encoding":{"default_value":42.0,"feature_name":"string","field":"string","target_map":{}}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{"feature_name":"string","field":"string","frequency_map":{}},"multi_encoding":{"processors":[42.0]},"n_gram_encoding":{"feature_prefix":"string","field":"string","length":42.0,"n_grams":[42.0],"start":42.0,"custom":true},"one_hot_encoding":{"field":"string","hot_map":"string"},"target_mean_encoding":{"default_value":42.0,"feature_name":"string","field":"string","target_map":{}}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"analyzed_fields":{"includes":["string"],"excludes":["string"]},"description":"string","dest":{"index":"string","results_field":"string"},"max_num_threads":42.0,"_meta":{"additionalProperty1":{},"additionalProperty2":{}},"model_memory_limit":"string","source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"headers":{},"version":"string"}'Identifier for the data frame analytics job.
curl \
--request DELETE 'http://api.example.com/_ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/_ml/data_frame/_evaluate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\": \"animal_classification\",\n \"evaluation\": {\n \"classification\": {\n \"actual_field\": \"animal_class\",\n \"predicted_field\": \"ml.animal_class_prediction\",\n \"metrics\": {\n \"multiclass_confusion_matrix\": {}\n }\n }\n }\n}"'{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"predicted_field": "ml.animal_class_prediction",
"metrics": {
"multiclass_confusion_matrix": {}
}
}
}
}{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"metrics": {
"auc_roc": {
"class_name": "dog"
}
}
}
}
}{
"index": "my_analytics_dest_index",
"evaluation": {
"outlier_detection": {
"actual_field": "is_outlier",
"predicted_probability_field": "ml.outlier_score"
}
}
}{
"index": "house_price_predictions",
"query": {
"bool": {
"filter": [
{
"term": {
"ml.is_training": false
}
}
]
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {
"offset": 10
},
"huber": {
"delta": 1.5
}
}
}
}
}{
"index": "house_price_predictions",
"query": {
"term": {
"ml.is_training": {
"value": true
}
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {},
"huber": {}
}
}
}
}{
"classification": {
"multiclass_confusion_matrix": {
"confusion_matrix": [
{
"actual_class": "cat",
"actual_class_doc_count": 12,
"predicted_classes": [
{
"predicted_class": "cat",
"count": 12
},
{
"predicted_class": "dog",
"count": 0
}
],
"other_predicted_class_doc_count": 0
},
{
"actual_class": "dog",
"actual_class_doc_count": 11,
"predicted_classes": [
{
"predicted_class": "dog",
"count": 7
},
{
"predicted_class": "cat",
"count": 4
}
],
"other_predicted_class_doc_count": 0
}
],
"other_actual_class_count": 0
}
}
}{
"classification": {
"auc_roc": {
"value": 0.8941788639536681
}
}
}{
"outlier_detection": {
"auc_roc": {
"value": 0.9258475774641445
},
"confusion_matrix": {
"0.25": {
"tp": 5,
"fp": 9,
"tn": 204,
"fn": 5
},
"0.5": {
"tp": 1,
"fp": 5,
"tn": 208,
"fn": 9
},
"0.75": {
"tp": 0,
"fp": 4,
"tn": 209,
"fn": 10
}
},
"precision": {
"0.25": 0.35714285714285715,
"0.5": 0.16666666666666666,
"0.75": 0
},
"recall": {
"0.25": 0.5,
"0.5": 0.1,
"0.75": 0
}
}
}You can get information for multiple data frame analytics jobs in a single API request by using a comma-separated list of data frame analytics jobs or a wildcard expression.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value returns an empty data_frame_analytics array when there
are no matches and the subset of results when there are partial matches.
If this parameter is false, the request returns a 404 status code when
there are no matches or only partial matches.
Skips the specified number of data frame analytics jobs.
Specifies the maximum number of data frame analytics jobs to obtain.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics' \
--header "Authorization: $API_KEY"Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value returns an empty data_frame_analytics array when there
are no matches and the subset of results when there are partial matches.
If this parameter is false, the request returns a 404 status code when
there are no matches or only partial matches.
Skips the specified number of data frame analytics jobs.
Specifies the maximum number of data frame analytics jobs to obtain.
Defines whether the stats response should be verbose.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/_stats' \
--header "Authorization: $API_KEY"Identifier for the data frame analytics job. If you do not specify this option, the API returns information for the first hundred data frame analytics jobs.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value returns an empty data_frame_analytics array when there
are no matches and the subset of results when there are partial matches.
If this parameter is false, the request returns a 404 status code when
there are no matches or only partial matches.
Skips the specified number of data frame analytics jobs.
Specifies the maximum number of data frame analytics jobs to obtain.
Defines whether the stats response should be verbose.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/{id}/_stats' \
--header "Authorization: $API_KEY"Preview the extracted features used by a data frame analytics config.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'Preview the extracted features used by a data frame analytics config.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'Preview the extracted features used by a data frame analytics config.
Identifier for the data frame analytics job.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/{id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'Preview the extracted features used by a data frame analytics config.
Identifier for the data frame analytics job.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'A data frame analytics job can be started and stopped multiple times
throughout its lifecycle.
If the destination index does not exist, it is created automatically the
first time you start the data frame analytics job. The
index.number_of_shards and index.number_of_replicas settings for the
destination index are copied from the source index. If there are multiple
source indices, the destination index copies the highest setting values. The
mappings for the destination index are also copied from the source indices.
If there are any mapping conflicts, the job fails to start.
If the destination index exists, it is used as is. You can therefore set up
the destination index in advance with custom settings and mappings.
Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Controls the amount of time to wait until the data frame analytics job starts.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_start' \
--header "Authorization: $API_KEY"A data frame analytics job can be started and stopped multiple times throughout its lifecycle.
Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Specifies what to do when the request:
The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.
If true, the data frame analytics job is stopped forcefully.
Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_stop' \
--header "Authorization: $API_KEY"Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
A description of the job.
The approximate maximum amount of memory resources that are permitted for
analytical processing. If your elasticsearch.yml file contains an
xpack.ml.max_model_memory_limit setting, an error occurs when you try
to create data frame analytics jobs that have model_memory_limit values
greater than that setting.
The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.
Specifies whether this job can start when there is insufficient machine learning node capacity for it to be immediately assigned to a node.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"description":"string","model_memory_limit":"string","max_num_threads":42.0,"allow_lazy_start":true}'The unique identifier of the trained model or a model alias.
You can get information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.
Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Specifies whether the included model definition should be returned as a JSON map (true) or in a custom compressed format (false).
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
Skips the specified number of models.
A comma delimited string of optional fields to include in the response body.
Supported values include:
definition: Includes the model definition.feature_importance_baseline: Includes the baseline for feature importance values.hyperparameters: Includes the information about hyperparameters used to train the model.
This information consists of the value, the absolute and relative
importance of the hyperparameter as well as an indicator of whether it was
specified by the user or tuned during hyperparameter optimization.total_feature_importance: Includes the total feature importance for the training data set. The
baseline and total feature importance values are returned in the metadata
field in the response body.definition_status: Includes the model definition status.Values are definition, feature_importance_baseline, hyperparameters, total_feature_importance, or definition_status.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models/{model_id}' \
--header "Authorization: $API_KEY"Enable you to supply a trained model that is not created by data frame analytics.
The unique identifier of the trained model.
If set to true and a compressed_definition is provided,
the request defers definition decompression and skips relevant
validations.
Whether to wait for all child operations (e.g. model download) to complete.
The compressed (GZipped and Base64 encoded) inference definition of the model. If compressed_definition is specified, then definition cannot be specified.
A human-readable description of the inference trained model.
Inference configuration provided when storing the model config
An object map that contains metadata about the model.
Values are tree_ensemble, lang_ident, or pytorch.
The estimated memory usage in bytes to keep the trained model in memory. This property is supported only if defer_definition_decompression is true or the model definition is not supplied.
The platform architecture (if applicable) of the trained mode. If the model
only works on one platform, because it is heavily optimized for a particular
processor architecture and OS combination, then this field specifies which.
The format of the string must match the platform identifiers used by Elasticsearch,
so one of, linux-x86_64, linux-aarch64, darwin-x86_64, darwin-aarch64,
or windows-x86_64. For portable models (those that work independent of processor
architecture or OS features), leave this field unset.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"compressed_definition":"string","definition":{"preprocessors":[{"frequency_encoding":{"field":"string","feature_name":"string","frequency_map":{"additionalProperty1":42.0,"additionalProperty2":42.0}},"one_hot_encoding":{"field":"string","hot_map":{"additionalProperty1":"string","additionalProperty2":"string"}},"target_mean_encoding":{"field":"string","feature_name":"string","target_map":{"additionalProperty1":42.0,"additionalProperty2":42.0},"default_value":42.0}}],"trained_model":{"tree":{"classification_labels":["string"],"feature_names":["string"],"target_type":"string","tree_structure":[{"decision_type":"string","default_left":true,"leaf_value":42.0,"left_child":42.0,"node_index":42.0,"right_child":42.0,"split_feature":42.0,"split_gain":42.0,"threshold":42.0}]},"tree_node":{"decision_type":"string","default_left":true,"leaf_value":42.0,"left_child":42.0,"node_index":42.0,"right_child":42.0,"split_feature":42.0,"split_gain":42.0,"threshold":42.0},"ensemble":{"aggregate_output":{"logistic_regression":{"weights":42.0},"weighted_sum":{"weights":42.0},"weighted_mode":{"weights":42.0},"exponent":{"weights":42.0}},"classification_labels":["string"],"feature_names":["string"],"target_type":"string","trained_models":[{}]}}},"description":"string","inference_config":{"regression":{"results_field":"string","num_top_feature_importance_values":42.0},"classification":{"num_top_classes":42.0,"num_top_feature_importance_values":42.0,"prediction_field_type":"string","results_field":"string","top_classes_results_field":"string"},"text_classification":{"num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","classification_labels":["string"],"vocabulary":{"index":"string"}},"zero_shot_classification":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"hypothesis_template":"string","classification_labels":["string"],"results_field":"string","multi_label":true,"labels":["string"]},"fill_mask":{"mask_token":"string","num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"learning_to_rank":{"default_params":{"additionalProperty1":{},"additionalProperty2":{}},"feature_extractors":[{}],"num_top_feature_importance_values":42.0},"ner":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","classification_labels":["string"],"vocabulary":{"index":"string"}},"pass_through":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"text_embedding":{"embedding_size":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"text_expansion":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"question_answering":{"num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","max_answer_length":42.0}},"input":{"field_names":"string"},"metadata":{},"model_type":"tree_ensemble","model_size_bytes":42.0,"platform_architecture":"string","tags":["string"],"prefix_strings":{"ingest":"string","search":"string"}}'The request deletes a trained inference model that is not referenced by an ingest pipeline.
The unique identifier of the trained model.
curl \
--request DELETE 'http://api.example.com/_ml/trained_models/{model_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}A trained model alias is a logical name used to reference a single trained model. You can use aliases instead of trained model identifiers to make it easier to reference your models. For example, you can use aliases in inference aggregations and processors. An alias must be unique and refer to only a single trained model. However, you can have multiple aliases for each trained model. If you use this API to update an alias such that it references a different trained model ID and the model uses a different type of data frame analytics, an error occurs. For example, this situation occurs if you have a trained model for regression analysis and a trained model for classification analysis; you cannot reassign an alias from one type of trained model to another. If you use this API to update an alias and there are very few input fields in common between the old and new trained models for the model alias, the API returns a warning.
The identifier for the trained model that the alias refers to.
The alias to create or update. This value cannot end in numbers.
Specifies whether the alias gets reassigned to the specified trained model if it is already assigned to a different model. If the alias is already assigned and this parameter is false, the API returns an error.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/model_aliases/{model_alias}' \
--header "Authorization: $API_KEY"This API deletes an existing model alias that refers to a trained model. If
the model alias is missing or refers to a model other than the one identified
by the model_id, this API returns an error.
The trained model ID to which the model alias refers.
The model alias to delete.
curl \
--request DELETE 'http://api.example.com/_ml/trained_models/{model_id}/model_aliases/{model_alias}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Specifies whether the included model definition should be returned as a JSON map (true) or in a custom compressed format (false).
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
Skips the specified number of models.
A comma delimited string of optional fields to include in the response body.
Supported values include:
definition: Includes the model definition.feature_importance_baseline: Includes the baseline for feature importance values.hyperparameters: Includes the information about hyperparameters used to train the model.
This information consists of the value, the absolute and relative
importance of the hyperparameter as well as an indicator of whether it was
specified by the user or tuned during hyperparameter optimization.total_feature_importance: Includes the total feature importance for the training data set. The
baseline and total feature importance values are returned in the metadata
field in the response body.definition_status: Includes the model definition status.Values are definition, feature_importance_baseline, hyperparameters, total_feature_importance, or definition_status.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models' \
--header "Authorization: $API_KEY"You can get usage information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.
The unique identifier of the trained model or a model alias. It can be a comma-separated list or a wildcard expression.
Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Skips the specified number of models.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models/{model_id}/_stats' \
--header "Authorization: $API_KEY"You can get usage information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.
Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Skips the specified number of models.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models/_stats' \
--header "Authorization: $API_KEY"The unique identifier of the trained model.
Controls the amount of time to wait for inference results.
An array of objects to pass to the model for inference. The objects should contain a fields matching your
configured trained model input. Typically, for NLP models, the field name is text_field.
Currently, for NLP models, only a single value is allowed.
curl \
--request POST 'http://api.example.com/_ml/trained_models/{model_id}/_infer' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"docs":[{"additionalProperty1":{},"additionalProperty2":{}}],"inference_config":{"regression":{"results_field":"string","num_top_feature_importance_values":42.0},"classification":{"num_top_classes":42.0,"num_top_feature_importance_values":42.0,"prediction_field_type":"string","results_field":"string","top_classes_results_field":"string"},"text_classification":{"num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string","classification_labels":["string"]},"zero_shot_classification":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string","multi_label":true,"labels":["string"]},"fill_mask":{"num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"ner":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"pass_through":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"text_embedding":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"text_expansion":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"question_answering":{"question":"string","num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string","max_answer_length":42.0}}}'The definition part for the model. Must be a base64 encoded string.
The total uncompressed definition length in bytes. Not base64 encoded.
The total number of parts that will be uploaded. Must be greater than 0.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/definition/{part}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"definition":"string","total_definition_length":42.0,"total_parts":42.0}'This API is supported only for natural language processing (NLP) models.
The vocabulary is stored in the index as described in inference_config.*.vocabulary of the trained model definition.
The unique identifier of the trained model.
The model vocabulary, which must not be empty.
The optional model merges if required by the tokenizer.
The optional vocabulary value scores if required by the tokenizer.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/vocabulary' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"vocabulary":["string"],"merges":["string"],"scores":[42.0]}'It allocates the model to every machine learning node.
The unique identifier of the trained model. Currently, only PyTorch models are supported.
The inference cache size (in memory outside the JVM heap) per node for the model.
The default value is the same size as the model_size_bytes. To disable the cache,
0b can be provided.
The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.
The deployment priority.
Values are normal or low.
Specifies the number of inference requests that are allowed in the queue. After the number of requests exceeds this value, new requests are rejected with a 429 error.
Sets the number of threads used by each model allocation during inference. This generally increases the inference speed. The inference process is a compute-bound process; any number greater than the number of available hardware threads on the machine does not increase the inference speed. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.
Specifies the amount of time to wait for the model to deploy.
Specifies the allocation status to wait for before returning.
Supported values include:
started: The trained model is started on at least one node.starting: Trained model deployment is starting but it is not yet deployed on any nodes.fully_allocated: Trained model deployment has started on all valid nodes.Values are started, starting, or fully_allocated.
curl \
--request POST 'http://api.example.com/_ml/trained_models/{model_id}/deployment/_start' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"adaptive_allocations":{"enabled":true,"min_number_of_allocations":42.0,"max_number_of_allocations":42.0}}'The unique identifier of the trained model.
Specifies what to do when the request: contains wildcard expressions and there are no deployments that match;
contains the _all string or no identifiers and there are no matches; or contains wildcard expressions and
there are only partial matches. By default, it returns an empty array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Forcefully stops the deployment, even if it is used by ingest pipelines. You can't use these pipelines until you restart the model deployment.
curl \
--request POST 'http://api.example.com/_ml/trained_models/{model_id}/deployment/_stop' \
--header "Authorization: $API_KEY"The unique identifier of the trained model. Currently, only PyTorch models are supported.
The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.
The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.
curl \
--request POST 'http://api.example.com/_ml/trained_models/{model_id}/deployment/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"number_of_allocations":42.0,"adaptive_allocations":{"enabled":true,"min_number_of_allocations":42.0,"max_number_of_allocations":42.0}}'Query rules enable you to configure per-query rules that are applied at query time to queries that match the specific rule. Query rules are organized into rulesets, collections of query rules that are matched against incoming queries. Query rules are applied using the rule query. If a query matches one or more rules in the ruleset, the query is re-written to apply the rules before searching. This allows pinning documents for only queries that match a specific term.
Get details about a query rule within a query ruleset.
The unique identifier of the query ruleset containing the rule to retrieve
The unique identifier of the query rule within the specified ruleset to retrieve
curl \
--request GET 'http://api.example.com/_query_rules/{ruleset_id}/_rule/{rule_id}' \
--header "Authorization: $API_KEY"{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "query_string",
"values": [
"pugs",
"puggles"
]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
}Create or update a query rule within a query ruleset.
IMPORTANT: Due to limitations within pinned queries, you can only pin documents using ids or docs, but cannot use both in single rule. It is advised to use one or the other in query rulesets, to avoid errors. Additionally, pinned queries have a maximum limit of 100 pinned hits. If multiple matching rules pin more than 100 documents, only the first 100 documents are pinned in the order they are specified in the ruleset.
The unique identifier of the query ruleset containing the rule to be created or updated.
The unique identifier of the query rule within the specified ruleset to be created or updated.
curl \
--request PUT 'http://api.example.com/_query_rules/{ruleset_id}/_rule/{rule_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"match_criteria\": {\n \"query_string\": \"puggles\"\n }\n}"'{
"match_criteria": {
"query_string": "puggles"
}
}Delete a query rule within a query ruleset. This is a destructive action that is only recoverable by re-adding the same rule with the create or update query rule API.
The unique identifier of the query ruleset containing the rule to delete
The unique identifier of the query rule within the specified ruleset to delete
curl \
--request DELETE 'http://api.example.com/_query_rules/{ruleset_id}/_rule/{rule_id}' \
--header "Authorization: $API_KEY"Get details about a query ruleset.
The unique identifier of the query ruleset
curl \
--request GET 'http://api.example.com/_query_rules/{ruleset_id}' \
--header "Authorization: $API_KEY"{
"ruleset_id": "my-ruleset",
"rules": [
{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "query_string",
"values": [ "pugs", "puggles" ]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
},
{
"rule_id": "my-rule2",
"type": "pinned",
"criteria": [
{
"type": "fuzzy",
"metadata": "query_string",
"values": [ "rescue dogs" ]
}
],
"actions": {
"docs": [
{
"_index": "index1",
"_id": "id3"
},
{
"_index": "index2",
"_id": "id4"
}
]
}
}
]
}There is a limit of 100 rules per ruleset.
This limit can be increased by using the xpack.applications.rules.max_rules_per_ruleset cluster setting.
IMPORTANT: Due to limitations within pinned queries, you can only select documents using ids or docs, but cannot use both in single rule.
It is advised to use one or the other in query rulesets, to avoid errors.
Additionally, pinned queries have a maximum limit of 100 pinned hits.
If multiple matching rules pin more than 100 documents, only the first 100 documents are pinned in the order they are specified in the ruleset.
The unique identifier of the query ruleset to be created or updated.
curl \
--request PUT 'http://api.example.com/_query_rules/{ruleset_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"rules\": [\n {\n \"rule_id\": \"my-rule1\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"contains\",\n \"metadata\": \"user_query\",\n \"values\": [ \"pugs\", \"puggles\" ]\n },\n {\n \"type\": \"exact\",\n \"metadata\": \"user_country\",\n \"values\": [ \"us\" ]\n }\n ],\n \"actions\": {\n \"ids\": [\n \"id1\",\n \"id2\"\n ]\n }\n },\n {\n \"rule_id\": \"my-rule2\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"fuzzy\",\n \"metadata\": \"user_query\",\n \"values\": [ \"rescue dogs\" ]\n }\n ],\n \"actions\": {\n \"docs\": [\n {\n \"_index\": \"index1\",\n \"_id\": \"id3\"\n },\n {\n \"_index\": \"index2\",\n \"_id\": \"id4\"\n }\n ]\n }\n }\n ]\n}"'{
"rules": [
{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "user_query",
"values": [ "pugs", "puggles" ]
},
{
"type": "exact",
"metadata": "user_country",
"values": [ "us" ]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
},
{
"rule_id": "my-rule2",
"type": "pinned",
"criteria": [
{
"type": "fuzzy",
"metadata": "user_query",
"values": [ "rescue dogs" ]
}
],
"actions": {
"docs": [
{
"_index": "index1",
"_id": "id3"
},
{
"_index": "index2",
"_id": "id4"
}
]
}
}
]
}Remove a query ruleset and its associated data. This is a destructive action that is not recoverable.
The unique identifier of the query ruleset to delete
curl \
--request DELETE 'http://api.example.com/_query_rules/{ruleset_id}' \
--header "Authorization: $API_KEY"Get summarized information about the query rulesets.
curl \
--request GET 'http://api.example.com/_query_rules' \
--header "Authorization: $API_KEY"{
"count": 3,
"results": [
{
"ruleset_id": "ruleset-1",
"rule_total_count": 1,
"rule_criteria_types_counts": {
"exact": 1
}
},
{
"ruleset_id": "ruleset-2",
"rule_total_count": 2,
"rule_criteria_types_counts": {
"exact": 1,
"fuzzy": 1
}
},
{
"ruleset_id": "ruleset-3",
"rule_total_count": 3,
"rule_criteria_types_counts": {
"exact": 1,
"fuzzy": 2
}
}
]
}Evaluate match criteria against a query ruleset to identify the rules that would match that criteria.
The unique identifier of the query ruleset to be created or updated
The match criteria to apply to rules in the given query ruleset.
Match criteria should match the keys defined in the criteria.metadata field of the rule.
curl \
--request POST 'http://api.example.com/_query_rules/{ruleset_id}/_test' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"rules\": [\n {\n \"rule_id\": \"my-rule1\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"contains\",\n \"metadata\": \"user_query\",\n \"values\": [ \"pugs\", \"puggles\" ]\n },\n {\n \"type\": \"exact\",\n \"metadata\": \"user_country\",\n \"values\": [ \"us\" ]\n }\n ],\n \"actions\": {\n \"ids\": [\n \"id1\",\n \"id2\"\n ]\n }\n },\n {\n \"rule_id\": \"my-rule2\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"fuzzy\",\n \"metadata\": \"user_query\",\n \"values\": [ \"rescue dogs\" ]\n }\n ],\n \"actions\": {\n \"docs\": [\n {\n \"_index\": \"index1\",\n \"_id\": \"id3\"\n },\n {\n \"_index\": \"index2\",\n \"_id\": \"id4\"\n }\n ]\n }\n }\n ]\n}"'{
"rules": [
{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "user_query",
"values": [ "pugs", "puggles" ]
},
{
"type": "exact",
"metadata": "user_country",
"values": [ "us" ]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
},
{
"rule_id": "my-rule2",
"type": "pinned",
"criteria": [
{
"type": "fuzzy",
"metadata": "user_query",
"values": [ "rescue dogs" ]
}
],
"actions": {
"docs": [
{
"_index": "index1",
"_id": "id3"
},
{
"_index": "index2",
"_id": "id4"
}
]
}
}
]
}{
"total_matched_rules": 1,
"matched_rules": [
{
"ruleset_id": "my-ruleset",
"rule_id": "my-rule1"
}
]
}Use the script support APIs to get a list of supported script contexts and languages. Use the stored script APIs to manage stored scripts and search templates.
The identifier for the stored script or search template.
The period to wait for the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request GET 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY"Creates or updates a stored script or search template.
The identifier for the stored script or search template. It must be unique within the cluster.
The context in which the script or search template should run.
To prevent errors, the API immediately compiles the script or template in this context.
If you specify both this and the <context> path parameter, the API uses the request path parameter.
The period to wait for a connection to the master node.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request PUT 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"lang\": \"mustache\",\n \"source\": {\n \"query\": {\n \"match\": {\n \"message\": \"{{query_string}}\"\n }\n },\n \"from\": \"{{from}}\",\n \"size\": \"{{size}}\"\n }\n }\n}"'{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2) + params['my_modifier']"
}
}Creates or updates a stored script or search template.
The identifier for the stored script or search template. It must be unique within the cluster.
The context in which the script or search template should run.
To prevent errors, the API immediately compiles the script or template in this context.
If you specify both this and the <context> path parameter, the API uses the request path parameter.
The period to wait for a connection to the master node.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"lang\": \"mustache\",\n \"source\": {\n \"query\": {\n \"match\": {\n \"message\": \"{{query_string}}\"\n }\n },\n \"from\": \"{{from}}\",\n \"size\": \"{{size}}\"\n }\n }\n}"'{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2) + params['my_modifier']"
}
}Deletes a stored script or search template.
The identifier for the stored script or search template.
The period to wait for a connection to the master node.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request DELETE 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY"Creates or updates a stored script or search template.
The context in which the script or search template should run.
To prevent errors, the API immediately compiles the script or template in this context.
If you specify both this and the <context> path parameter, the API uses the request path parameter.
The period to wait for a connection to the master node.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request PUT 'http://api.example.com/_scripts/{id}/{context}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"lang\": \"mustache\",\n \"source\": {\n \"query\": {\n \"match\": {\n \"message\": \"{{query_string}}\"\n }\n },\n \"from\": \"{{from}}\",\n \"size\": \"{{size}}\"\n }\n }\n}"'{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2) + params['my_modifier']"
}
}Creates or updates a stored script or search template.
The context in which the script or search template should run.
To prevent errors, the API immediately compiles the script or template in this context.
If you specify both this and the <context> path parameter, the API uses the request path parameter.
The period to wait for a connection to the master node.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
The period to wait for a response.
If no response is received before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_scripts/{id}/{context}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"lang\": \"mustache\",\n \"source\": {\n \"query\": {\n \"match\": {\n \"message\": \"{{query_string}}\"\n }\n },\n \"from\": \"{{from}}\",\n \"size\": \"{{size}}\"\n }\n }\n}"'{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2) + params['my_modifier']"
}
}Runs a script and returns a result. Use this API to build and test scripts, such as when defining a script for a runtime field. This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.
The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.
Each context requires a script, but additional parameters depend on the context you're using for that script.
Values are painless_test, filter, score, boolean_field, date_field, double_field, geo_point_field, ip_field, keyword_field, long_field, or composite_field.
curl \
--request GET 'http://api.example.com/_scripts/painless/_execute' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"source\": \"params.count / params.total\",\n \"params\": {\n \"count\": 100.0,\n \"total\": 1000.0\n }\n }\n}"'{
"script": {
"source": "params.count / params.total",
"params": {
"count": 100.0,
"total": 1000.0
}
}
}{
"script": {
"source": "doc['field'].value.length() <= params.max_length",
"params": {
"max_length": 4
}
},
"context": "filter",
"context_setup": {
"index": "my-index-000001",
"document": {
"field": "four"
}
}
}{
"script": {
"source": "doc['rank'].value / params.max_rank",
"params": {
"max_rank": 5.0
}
},
"context": "score",
"context_setup": {
"index": "my-index-000001",
"document": {
"rank": 4
}
}
}{
"result": "0.1"
}{
"result": true
}{
"result": 0.8
}Runs a script and returns a result. Use this API to build and test scripts, such as when defining a script for a runtime field. This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.
The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.
Each context requires a script, but additional parameters depend on the context you're using for that script.
Values are painless_test, filter, score, boolean_field, date_field, double_field, geo_point_field, ip_field, keyword_field, long_field, or composite_field.
curl \
--request POST 'http://api.example.com/_scripts/painless/_execute' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"source\": \"params.count / params.total\",\n \"params\": {\n \"count\": 100.0,\n \"total\": 1000.0\n }\n }\n}"'{
"script": {
"source": "params.count / params.total",
"params": {
"count": 100.0,
"total": 1000.0
}
}
}{
"script": {
"source": "doc['field'].value.length() <= params.max_length",
"params": {
"max_length": 4
}
},
"context": "filter",
"context_setup": {
"index": "my-index-000001",
"document": {
"field": "four"
}
}
}{
"script": {
"source": "doc['rank'].value / params.max_rank",
"params": {
"max_rank": 5.0
}
},
"context": "score",
"context_setup": {
"index": "my-index-000001",
"document": {
"rank": 4
}
}
}{
"result": "0.1"
}{
"result": true
}{
"result": 0.8
}Retrieve the results of a previously submitted asynchronous search request. If the Elasticsearch security features are enabled, access to the results of a specific async search is restricted to the user or API key that submitted it.
A unique identifier for the async search.
The length of time that the async search should be available in the cluster.
When not specified, the keep_alive set with the corresponding submit async request will be used.
Otherwise, it is possible to override the value and extend the validity of the request.
When this period expires, the search, if still running, is cancelled.
If the search is completed, its saved results are deleted.
Specify whether aggregation and suggester names should be prefixed by their respective types in the response
Specifies to wait for the search to be completed up until the provided timeout. Final results will be returned if available before the timeout expires, otherwise the currently available results will be returned once the timeout expires. By default no timeout is set meaning that the currently available results will be returned without any additional wait.
curl \
--request GET 'http://api.example.com/_async_search/{id}' \
--header "Authorization: $API_KEY"{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_partial" : false,
"is_running" : false,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"completion_time_in_millis" : 1583945903130,
"response" : {
"took" : 12144,
"timed_out" : false,
"num_reduce_phases" : 46,
"_shards" : {
"total" : 562,
"successful" : 188,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 456433,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"sale_date" : {
"buckets" : []
}
}
}
}If the asynchronous search is still running, it is cancelled.
Otherwise, the saved search results are deleted.
If the Elasticsearch security features are enabled, the deletion of a specific async search is restricted to: the authenticated user that submitted the original search request; users that have the cancel_task cluster privilege.
A unique identifier for the async search.
curl \
--request DELETE 'http://api.example.com/_async_search/{id}' \
--header "Authorization: $API_KEY"Get the status of a previously submitted async search request given its identifier, without retrieving search results. If the Elasticsearch security features are enabled, the access to the status of a specific async search is restricted to:
monitor cluster privilege or greater privileges.A unique identifier for the async search.
The length of time that the async search needs to be available. Ongoing async searches and any saved search results are deleted after this period.
curl \
--request GET 'http://api.example.com/_async_search/status/{id}' \
--header "Authorization: $API_KEY"{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_running" : true,
"is_partial" : true,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"_shards" : {
"total" : 562,
"successful" : 188,
"skipped" : 0,
"failed" : 0
}
}{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_running" : false,
"is_partial" : false,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"_shards" : {
"total" : 562,
"successful" : 562,
"skipped" : 0,
"failed" : 0
},
"completion_status" : 200
}{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_running" : false,
"is_partial" : true,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"_shards" : {
"total" : 562,
"successful" : 450,
"skipped" : 0,
"failed" : 112
},
"completion_status" : 503
}When the primary sort of the results is an indexed field, shards get sorted based on minimum and maximum value that they hold for that field. Partial results become available following the sort criteria that was requested.
Warning: Asynchronous search does not support scroll or search requests that include only the suggest section.
By default, Elasticsearch does not allow you to store an async search response larger than 10Mb and an attempt to do this results in an error.
The maximum allowed size for a stored async search response can be set by changing the search.max_async_search_response_size cluster level setting.
Blocks and waits until the search is completed up to a certain timeout. When the async search completes within the timeout, the response won’t include the ID as the results are not stored in the cluster.
Specifies how long the async search needs to be available. Ongoing async searches and any saved search results are deleted after this period.
If true, results are stored for later retrieval when the search completes within the wait_for_completion_timeout.
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Indicate if an error should be returned if there is a partial search failure or timeout
The analyzer to use for the query string
Specify whether wildcard and prefix queries should be analyzed (default: false)
Affects how often partial results become available, which happens whenever shard results are reduced. A partial reduction is performed every time the coordinating node has received a certain number of new shard responses (5 by default).
The default value is the only supported value.
The default operator for query string query (AND or OR)
Values are and, AND, or, or OR.
The field to use as default where no field prefix is given in the query string
A comma-separated list of fields to return as the docvalue representation of a field for each hit
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Specify whether to return detailed information about score computation as part of a hit
Whether specified concrete, expanded or aliased indices should be ignored when throttled
Specify whether format-based query failures (such as providing text to a numeric field) should be ignored
The number of concurrent shard requests per node this search executes concurrently. This value should be used to limit the impact of the search on the cluster in order to limit the number of concurrent shard requests
Specify the node or shard the operation should be performed on (default: random)
Specify if request cache should be used for this request or not, defaults to true
A comma-separated list of specific routing values
Search operation type
Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.Values are query_then_fetch or dfs_query_then_fetch.
Specific 'tag' of the request for logging and statistical purposes
A comma-separated list of stored fields to return as part of a hit
Specifies which field to use for suggestions.
Specify suggest mode
Supported values include:
missing: Only generate suggestions for terms that are not in the shard.popular: Only suggest terms that occur in more docs on the shard than the original term.always: Suggest any matching suggestions based on terms in the suggest text.Values are missing, popular, or always.
How many suggestions to return in response
The source text for which the suggestions should be returned.
The maximum number of documents to collect for each shard, upon reaching which the query execution will terminate early.
Explicit operation timeout
Indicate if the number of documents that match the query should be tracked. A number can also be specified, to accurately track the total hit count up to the number.
Whether to calculate and return scores even if they are not used for sorting
Specify whether aggregation and suggester names should be prefixed by their respective types in the response
Indicates whether hits.total should be rendered as an integer or an object in the rest search response
Specify whether to return document version as part of a hit
True or false to return the _source field or not, or a list of fields to return
A list of fields to exclude from the returned _source field
A list of fields to extract and return from the _source field
Specify whether to return sequence number and primary term of the last modification of each hit
Query in the Lucene query string syntax
Number of hits to return (default: 10)
Starting offset (default: 0)
A comma-separated list of : pairs
If true, returns detailed information about score computation as part of a hit.
Configuration of search extensions defined by Elasticsearch plugins.
Starting document offset. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
Boosts the _score of documents from specified indices.
Array of wildcard (*) patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.
Defines the approximate kNN search to run.
Minimum _score for matching documents. Documents with a lower _score are not included in search results and results collected by aggregations.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Retrieve a script evaluation (based on different fields) for each hit.
A field value.
The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
Array of wildcard (*) patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.
Maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting. Defaults to 0, which does not terminate query execution early.
Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
If true, calculate and return document scores, even if the scores are not used for sorting.
If true, returns document version as part of a hit.
If true, returns sequence number and primary term of the last modification of each hit. See Optimistic concurrency control.
Stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.
curl \
--request POST 'http://api.example.com/_async_search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"sort\": [\n { \"date\": { \"order\": \"asc\" } }\n ],\n \"aggs\": {\n \"sale_date\": {\n \"date_histogram\": {\n \"field\": \"date\",\n \"calendar_interval\": \"1d\"\n }\n }\n }\n}"'{
"sort": [
{ "date": { "order": "asc" } }
],
"aggs": {
"sale_date": {
"date_histogram": {
"field": "date",
"calendar_interval": "1d"
}
}
}
}{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_partial" : true,
"is_running" : true,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"response" : {
"took" : 1122,
"timed_out" : false,
"num_reduce_phases" : 0,
"_shards" : {
"total" : 562,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 157483,
"relation" : "gte"
},
"max_score" : null,
"hits" : [ ]
}
}
}When the primary sort of the results is an indexed field, shards get sorted based on minimum and maximum value that they hold for that field. Partial results become available following the sort criteria that was requested.
Warning: Asynchronous search does not support scroll or search requests that include only the suggest section.
By default, Elasticsearch does not allow you to store an async search response larger than 10Mb and an attempt to do this results in an error.
The maximum allowed size for a stored async search response can be set by changing the search.max_async_search_response_size cluster level setting.
A comma-separated list of index names to search; use _all or empty string to perform the operation on all indices
Blocks and waits until the search is completed up to a certain timeout. When the async search completes within the timeout, the response won’t include the ID as the results are not stored in the cluster.
Specifies how long the async search needs to be available. Ongoing async searches and any saved search results are deleted after this period.
If true, results are stored for later retrieval when the search completes within the wait_for_completion_timeout.
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Indicate if an error should be returned if there is a partial search failure or timeout
The analyzer to use for the query string
Specify whether wildcard and prefix queries should be analyzed (default: false)
Affects how often partial results become available, which happens whenever shard results are reduced. A partial reduction is performed every time the coordinating node has received a certain number of new shard responses (5 by default).
The default value is the only supported value.
The default operator for query string query (AND or OR)
Values are and, AND, or, or OR.
The field to use as default where no field prefix is given in the query string
A comma-separated list of fields to return as the docvalue representation of a field for each hit
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Specify whether to return detailed information about score computation as part of a hit
Whether specified concrete, expanded or aliased indices should be ignored when throttled
Specify whether format-based query failures (such as providing text to a numeric field) should be ignored
The number of concurrent shard requests per node this search executes concurrently. This value should be used to limit the impact of the search on the cluster in order to limit the number of concurrent shard requests
Specify the node or shard the operation should be performed on (default: random)
Specify if request cache should be used for this request or not, defaults to true
A comma-separated list of specific routing values
Search operation type
Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.Values are query_then_fetch or dfs_query_then_fetch.
Specific 'tag' of the request for logging and statistical purposes
A comma-separated list of stored fields to return as part of a hit
Specifies which field to use for suggestions.
Specify suggest mode
Supported values include:
missing: Only generate suggestions for terms that are not in the shard.popular: Only suggest terms that occur in more docs on the shard than the original term.always: Suggest any matching suggestions based on terms in the suggest text.Values are missing, popular, or always.
How many suggestions to return in response
The source text for which the suggestions should be returned.
The maximum number of documents to collect for each shard, upon reaching which the query execution will terminate early.
Explicit operation timeout
Indicate if the number of documents that match the query should be tracked. A number can also be specified, to accurately track the total hit count up to the number.
Whether to calculate and return scores even if they are not used for sorting
Specify whether aggregation and suggester names should be prefixed by their respective types in the response
Indicates whether hits.total should be rendered as an integer or an object in the rest search response
Specify whether to return document version as part of a hit
True or false to return the _source field or not, or a list of fields to return
A list of fields to exclude from the returned _source field
A list of fields to extract and return from the _source field
Specify whether to return sequence number and primary term of the last modification of each hit
Query in the Lucene query string syntax
Number of hits to return (default: 10)
Starting offset (default: 0)
A comma-separated list of : pairs
If true, returns detailed information about score computation as part of a hit.
Configuration of search extensions defined by Elasticsearch plugins.
Starting document offset. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
Boosts the _score of documents from specified indices.
Array of wildcard (*) patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.
Defines the approximate kNN search to run.
Minimum _score for matching documents. Documents with a lower _score are not included in search results and results collected by aggregations.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Retrieve a script evaluation (based on different fields) for each hit.
A field value.
The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
Array of wildcard (*) patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.
Maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting. Defaults to 0, which does not terminate query execution early.
Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
If true, calculate and return document scores, even if the scores are not used for sorting.
If true, returns document version as part of a hit.
If true, returns sequence number and primary term of the last modification of each hit. See Optimistic concurrency control.
Stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.
curl \
--request POST 'http://api.example.com/{index}/_async_search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"sort\": [\n { \"date\": { \"order\": \"asc\" } }\n ],\n \"aggs\": {\n \"sale_date\": {\n \"date_histogram\": {\n \"field\": \"date\",\n \"calendar_interval\": \"1d\"\n }\n }\n }\n}"'{
"sort": [
{ "date": { "order": "asc" } }
],
"aggs": {
"sale_date": {
"date_histogram": {
"field": "date",
"calendar_interval": "1d"
}
}
}
}{
"id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
"is_partial" : true,
"is_running" : true,
"start_time_in_millis" : 1583945890986,
"expiration_time_in_millis" : 1584377890986,
"response" : {
"took" : 1122,
"timed_out" : false,
"num_reduce_phases" : 0,
"_shards" : {
"total" : 562,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 157483,
"relation" : "gte"
},
"max_score" : null,
"hits" : [ ]
}
}
}IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
The scroll API gets large sets of results from a single scrolling search request.
To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter.
The scroll parameter indicates how long Elasticsearch should retain the search context for the request.
The search response returns a scroll ID in the _scroll_id response body parameter.
You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request.
If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.
You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.
IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
The period to retain the search context for scrolling.
The scroll ID for scrolled search
If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.
curl \
--request GET 'http://api.example.com/_search/scroll' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
The scroll API gets large sets of results from a single scrolling search request.
To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter.
The scroll parameter indicates how long Elasticsearch should retain the search context for the request.
The search response returns a scroll ID in the _scroll_id response body parameter.
You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request.
If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.
You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.
IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
The period to retain the search context for scrolling.
The scroll ID for scrolled search
If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.
curl \
--request POST 'http://api.example.com/_search/scroll' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}Clear the search context and results for a scrolling search.
curl \
--request DELETE 'http://api.example.com/_search/scroll' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\": \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
The scroll API gets large sets of results from a single scrolling search request.
To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter.
The scroll parameter indicates how long Elasticsearch should retain the search context for the request.
The search response returns a scroll ID in the _scroll_id response body parameter.
You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request.
If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.
You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.
IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
The scroll ID
The period to retain the search context for scrolling.
The scroll ID for scrolled search
If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.
curl \
--request GET 'http://api.example.com/_search/scroll/{scroll_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
The scroll API gets large sets of results from a single scrolling search request.
To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter.
The scroll parameter indicates how long Elasticsearch should retain the search context for the request.
The search response returns a scroll ID in the _scroll_id response body parameter.
You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request.
If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.
You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.
IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
The scroll ID
The period to retain the search context for scrolling.
The scroll ID for scrolled search
If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.
curl \
--request POST 'http://api.example.com/_search/scroll/{scroll_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}Clear the search context and results for a scrolling search.
A comma-separated list of scroll IDs to clear.
To clear all scroll IDs, use _all.
IMPORTANT: Scroll IDs can be long. It is recommended to specify scroll IDs in the request body parameter.
curl \
--request DELETE 'http://api.example.com/_search/scroll/{scroll_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"scroll_id\": \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'{
"scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}A point in time must be opened explicitly before being used in search requests.
The keep_alive parameter tells Elasticsearch how long it should persist.
A point in time is automatically closed when the keep_alive period has elapsed.
However, keeping points in time has a cost; close them as soon as they are no longer required for search requests.
curl \
--request DELETE 'http://api.example.com/_pit' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"id\": \"46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==\"\n}"'{
"id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}{
"succeeded": true,
"num_freed": 3
}Get the number of documents matching a query.
The query can be provided either by using a simple query string as a parameter, or by defining Query DSL within the request body.
The query is optional. When no query is provided, the API uses match_all to count all the documents.
The count API supports multi-target syntax. You can run a single count API search across multiple data streams and indices.
The operation is broadcast across all shards. For each shard ID group, a replica is chosen and the search is run against it. This means that replicas increase the scalability of the count.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
The analyzer to use for the query string.
This parameter can be used only when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
This parameter can be used only when the q query string parameter is specified.
The default operator for query string query: AND or OR.
This parameter can be used only when the q query string parameter is specified.
Values are and, AND, or, or OR.
The field to use as a default when no field prefix is given in the query string.
This parameter can be used only when the q query string parameter is specified.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, concrete, expanded, or aliased indices are ignored when frozen.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
This parameter can be used only when the q query string parameter is specified.
The minimum _score value that documents must have to be included in the result.
The node or shard the operation should be performed on. By default, it is random.
A custom value used to route operations to a specific shard.
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
The query in Lucene query string syntax. This parameter cannot be used with a request body.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request GET 'http://api.example.com/_count' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\" : {\n \"term\" : { \"user.id\" : \"kimchy\" }\n }\n}"'{
"query" : {
"term" : { "user.id" : "kimchy" }
}
}{
"count": 1,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
}
}Get the number of documents matching a query.
The query can be provided either by using a simple query string as a parameter, or by defining Query DSL within the request body.
The query is optional. When no query is provided, the API uses match_all to count all the documents.
The count API supports multi-target syntax. You can run a single count API search across multiple data streams and indices.
The operation is broadcast across all shards. For each shard ID group, a replica is chosen and the search is run against it. This means that replicas increase the scalability of the count.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
The analyzer to use for the query string.
This parameter can be used only when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
This parameter can be used only when the q query string parameter is specified.
The default operator for query string query: AND or OR.
This parameter can be used only when the q query string parameter is specified.
Values are and, AND, or, or OR.
The field to use as a default when no field prefix is given in the query string.
This parameter can be used only when the q query string parameter is specified.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.If true, concrete, expanded, or aliased indices are ignored when frozen.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
This parameter can be used only when the q query string parameter is specified.
The minimum _score value that documents must have to be included in the result.
The node or shard the operation should be performed on. By default, it is random.
A custom value used to route operations to a specific shard.
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
The query in Lucene query string syntax. This parameter cannot be used with a request body.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/_count' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\" : {\n \"term\" : { \"user.id\" : \"kimchy\" }\n }\n}"'{
"query" : {
"term" : { "user.id" : "kimchy" }
}
}{
"count": 1,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
}
}