Autoscaling

Get an autoscaling policy Added in 7.11.0

GET /_autoscaling/policy/{name}
Api key auth Basic auth Bearer auth

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

External documentation

Path parameters

  • name string Required

    the name of the autoscaling policy

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
GET /_autoscaling/policy/{name} 
curl \
 --request GET 'http://api.example.com/_autoscaling/policy/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
This may be a response to `GET /_autoscaling/policy/my_autoscaling_policy`. 
{
   "roles": <roles>,
   "deciders": <deciders>
}

Get behavioral analytics collections Deprecated Technical preview

GET /_application/analytics/{name}
Api key auth Basic auth Bearer auth

Path parameters

  • name array[string] Required

    A list of analytics collections to limit the returned information

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • event_data_stream object Required
        Hide event_data_stream attribute Show event_data_stream attribute object
GET /_application/analytics/{name} 
curl \
 --request GET 'http://api.example.com/_application/analytics/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _application/analytics/my*` 
{
  "my_analytics_collection": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection"
      }
  },
  "my_analytics_collection2": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection2"
      }
  }
}

Get behavioral analytics collections Deprecated Technical preview

GET /_application/analytics
Api key auth Basic auth Bearer auth

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • event_data_stream object Required
        Hide event_data_stream attribute Show event_data_stream attribute object
GET /_application/analytics 
curl \
 --request GET 'http://api.example.com/_application/analytics' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _application/analytics/my*` 
{
  "my_analytics_collection": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection"
      }
  },
  "my_analytics_collection2": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection2"
      }
  }
}

Get index information

GET /_cat/indices
Api key auth Basic auth Bearer auth

Get high-level information about indices in a cluster, including backing indices for data streams.

Use this request to get the following information for each index in a cluster:

  • shard count
  • document count
  • deleted document count
  • primary store size
  • total store size of all shards, including shard replicas

These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.

CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match.

  • health string

    The health status used to limit returned indices. By default, the response includes indices of any health status.

    Values are green, GREEN, yellow, YELLOW, red, or RED.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

  • pri boolean

    If true, the response only includes information from primary shards.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • master_timeout string

    Period to wait for a connection to the master node.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

Responses

GET /_cat/indices 
curl \
 --request GET 'http://api.example.com/_cat/indices' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/indices/my-index-*?v=true&s=index&format=json`. 
[
  {
    "health": "yellow",
    "status": "open",
    "index": "my-index-000001",
    "uuid": "u8FNjxh8Rfy_awN11oDKYQ",
    "pri": "1",
    "rep": "1",
    "docs.count": "1200",
    "docs.deleted": "0",
    "store.size": "88.1kb",
    "pri.store.size": "88.1kb",
    "dataset.size": "88.1kb"
  },
  {
    "health": "green",
    "status": "open",
    "index": "my-index-000002",
    "uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
    "pri": "1",
    "rep": "0",
    "docs.count": "0",
    "docs.deleted": "0",
    "store.size": "260b",
    "pri.store.size": "260b",
    "dataset.size": "260b"
  }
]

Get anomaly detection jobs Added in 7.7.0

GET /_cat/ml/anomaly_detectors
Api key auth Basic auth Bearer auth

Get configuration and usage information for anomaly detection jobs. This API returns a maximum of 10,000 jobs. If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage cluster privileges to use this API.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get anomaly detection job statistics API.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no jobs that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    Comma-separated list of column names to display.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • state string

      Values are closing, closed, opened, failed, or opening.

    • opened_time string

      For open jobs only, the amount of time the job has been opened.

    • assignment_explanation string

      For open anomaly detection jobs only, contains messages relating to the selection of a node to run the job.

    • data.processed_records string

      The number of input documents that have been processed by the anomaly detection job. This value includes documents with missing fields, since they are nonetheless analyzed. If you use datafeeds and have aggregations in your search query, the processed_record_count is the number of aggregation results processed, not the number of Elasticsearch documents.

    • data.processed_fields string

      The total number of fields in all the documents that have been processed by the anomaly detection job. Only fields that are specified in the detector configuration object contribute to this count. The timestamp is not included in this count.

    • data.input_bytes number | string

      One of:
      ByteSize number ByteSize string
    • data.input_records string

      The number of input documents posted to the anomaly detection job.

    • data.input_fields string

      The total number of fields in input documents posted to the anomaly detection job. This count includes fields that are not used in the analysis. However, be aware that if you are using a datafeed, it extracts only the required fields from the documents it retrieves before posting them to the job.

    • data.invalid_dates string

      The number of input documents with either a missing date field or a date that could not be parsed.

    • data.missing_fields string

      The number of input documents that are missing a field that the anomaly detection job is configured to analyze. Input documents with missing fields are still processed because it is possible that not all fields are missing. If you are using datafeeds or posting data to the job in JSON format, a high missing_field_count is often not an indication of data issues. It is not necessarily a cause for concern.

    • data.out_of_order_timestamps string

      The number of input documents that have a timestamp chronologically preceding the start of the current anomaly detection bucket offset by the latency window. This information is applicable only when you provide data to the anomaly detection job by using the post data API. These out of order documents are discarded, since jobs require time series data to be in ascending chronological order.

    • data.empty_buckets string

      The number of buckets which did not contain any data. If your data contains many empty buckets, consider increasing your bucket_span or using functions that are tolerant to gaps in data such as mean, non_null_sum or non_zero_count.

    • data.sparse_buckets string

      The number of buckets that contained few data points compared to the expected number of data points. If your data contains many sparse buckets, consider using a longer bucket_span.

    • data.buckets string

      The total number of buckets processed.

    • data.earliest_record string

      The timestamp of the earliest chronologically input document.

    • data.latest_record string

      The timestamp of the latest chronologically input document.

    • data.last string

      The timestamp at which data was last analyzed, according to server time.

    • data.last_empty_bucket string

      The timestamp of the last bucket that did not contain any data.

    • data.last_sparse_bucket string

      The timestamp of the last bucket that was considered sparse.

    • model.bytes number | string

      One of:
      ByteSize number ByteSize string
    • model.memory_status string

      Values are ok, soft_limit, or hard_limit.

    • model.bytes_exceeded number | string

      One of:
      ByteSize number ByteSize string
    • model.memory_limit string

      The upper limit for model memory usage, checked on increasing values.

    • model.by_fields string

      The number of by field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.over_fields string

      The number of over field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.partition_fields string

      The number of partition field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.bucket_allocation_failures string

      The number of buckets for which new entities in incoming data were not processed due to insufficient model memory. This situation is also signified by a hard_limit: memory_status property value.

    • model.categorization_status string

      Values are ok or warn.

    • model.categorized_doc_count string

      The number of documents that have had a field categorized.

    • model.total_category_count string

      The number of categories created by categorization.

    • model.frequent_category_count string

      The number of categories that match more than 1% of categorized documents.

    • model.rare_category_count string

      The number of categories that match just one categorized document.

    • model.dead_category_count string

      The number of categories created by categorization that will never be assigned again because another category’s definition makes it a superset of the dead category. Dead categories are a side effect of the way categorization has no prior training.

    • model.failed_category_count string

      The number of times that categorization wanted to create a new category but couldn’t because the job had hit its model_memory_limit. This count does not track which specific categories failed to be created. Therefore you cannot use this value to determine the number of unique categories that were missed.

    • model.log_time string

      The timestamp when the model stats were gathered, according to server time.

    • model.timestamp string

      The timestamp of the last record when the model stats were gathered.

    • forecasts.total string

      The number of individual forecasts currently available for the job. A value of one or more indicates that forecasts exist.

    • forecasts.memory.min string

      The minimum memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.max string

      The maximum memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.avg string

      The average memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.total string

      The total memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.records.min string

      The minimum number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.max string

      The maximum number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.avg string

      The average number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.total string

      The total number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.time.min string

      The minimum runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.max string

      The maximum runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.avg string

      The average runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.total string

      The total runtime in milliseconds for forecasts related to the anomaly detection job.

    • node.id string
    • node.name string

      The name of the assigned node.

    • node.ephemeral_id string
    • node.address string

      The network address of the assigned node.

    • buckets.count string

      The number of bucket results produced by the job.

    • buckets.time.total string

      The sum of all bucket processing times, in milliseconds.

    • buckets.time.min string

      The minimum of all bucket processing times, in milliseconds.

    • buckets.time.max string

      The maximum of all bucket processing times, in milliseconds.

    • buckets.time.exp_avg string

      The exponential moving average of all bucket processing times, in milliseconds.

    • buckets.time.exp_avg_hour string

      The exponential moving average of bucket processing times calculated in a one hour time window, in milliseconds.
GET /_cat/ml/anomaly_detectors 
curl \
 --request GET 'http://api.example.com/_cat/ml/anomaly_detectors' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/ml/anomaly_detectors?h=id,s,dpr,mb&v=true&format=json`. 
[
  {
    "id": "high_sum_total_sales",
    "s": "closed",
    "dpr": "14022",
    "mb": "1.5mb"
  },
  {
    "id": "low_request_rate",
    "s": "closed",
    "dpr": "1216",
    "mb": "40.5kb"
  },
  {
    "id": "response_code_rates",
    "s": "closed",
    "dpr": "28146",
    "mb": "132.7kb"
  },
  {
    "id": "url_scanning",
    "s": "closed",
    "dpr": "28146",
    "mb": "501.6kb"
  }
]

Get shard recovery information

GET /_cat/recovery/{index}
Api key auth Basic auth Bearer auth

Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • active_only boolean

    If true, the response only includes ongoing shard recoveries.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • detailed boolean

    If true, the response includes detailed information about shard recoveries.

  • index string | array[string]

    Comma-separated list or wildcard expression of index names to limit the returned information

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

GET /_cat/recovery/{index} 
curl \
 --request GET 'http://api.example.com/_cat/recovery/{index}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/recovery?v=true&format=json`. In this example, the source and target nodes are the same because the recovery type is `store`, meaning they were read from local storage on node start. 
[
  {
    "index": "my-index-000001 ",
    "shard": "0",
    "time": "13ms",
    "type": "store",
    "stage": "done",
    "source_host": "n/a",
    "source_node": "n/a",
    "target_host": "127.0.0.1",
    "target_node": "node-0",
    "repository": "n/a",
    "snapshot": "n/a",
    "files": "0",
    "files_recovered": "0",
    "files_percent": "100.0%",
    "files_total": "13",
    "bytes": "0b",
    "bytes_recovered": "0b",
    "bytes_percent": "100.0%",
    "bytes_total": "9928b",
    "translog_ops": "0",
    "translog_ops_recovered": "0",
    "translog_ops_percent": "100.0%"
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,shost,thost,f,fp,b,bp&format=json`. You can retrieve information about an ongoing recovery for example when you increase the replica count of an index and bring another node online to host the replicas. In this example, the recovery type is `peer`, meaning the shard recovered from another node. The `files` and `bytes` are real-time measurements. 
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1252ms",
    "ty": "peer",
    "st": "done",
    "shost": "192.168.1.1",
    "thost": "192.168.1.1",
    "f": "0",
    "fp": "100.0%",
    "b": "0b",
    "bp": "100.0%",
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,rep,snap,f,fp,b,bp&format=json`. You can restore backups of an index using the snapshot and restore API. You can use the cat recovery API to get information about a snapshot recovery. 
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1978ms",
    "ty": "snapshot",
    "st": "done",
    "rep": "my-repo",
    "snap": "snap-1",
    "f": "79",
    "fp": "8.0%",
    "b": "12086",
    "bp": "9.0%"
  }
]

Get the cluster state Added in 1.3.0

GET /_cluster/state/{metric}
Api key auth Basic auth Bearer auth

Get comprehensive information about the state of the cluster.

The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.

The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.

By default the API will route requests to the elected master node since this node is the authoritative source of cluster states. You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.

Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.

WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.

Path parameters

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

  • flat_settings boolean

    Return settings in flat format (default: false)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Specify timeout for connection to master

  • wait_for_metadata_version number

    Wait for the metadata version to be equal or greater than the specified metadata version

  • wait_for_timeout string

    The maximum time to wait for wait_for_metadata_version before timing out

Responses

GET /_cluster/state/{metric} 
curl \
 --request GET 'http://api.example.com/_cluster/state/{metric}' \
 --header "Authorization: $API_KEY"

Get the cluster state Added in 1.3.0

GET /_cluster/state/{metric}/{index}
Api key auth Basic auth Bearer auth

Get comprehensive information about the state of the cluster.

The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.

The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.

By default the API will route requests to the elected master node since this node is the authoritative source of cluster states. You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.

Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.

WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.

Path parameters

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

  • index string | array[string] Required

    A comma-separated list of index names; use _all or empty string to perform the operation on all indices

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

  • flat_settings boolean

    Return settings in flat format (default: false)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Specify timeout for connection to master

  • wait_for_metadata_version number

    Wait for the metadata version to be equal or greater than the specified metadata version

  • wait_for_timeout string

    The maximum time to wait for wait_for_metadata_version before timing out

Responses

GET /_cluster/state/{metric}/{index} 
curl \
 --request GET 'http://api.example.com/_cluster/state/{metric}/{index}' \
 --header "Authorization: $API_KEY"

Claim a connector sync job Technical preview

PUT /_connector/_sync_job/{connector_sync_job_id}/_claim
Api key auth Basic auth Bearer auth

This action updates the job status to in_progress and sets the last_seen and started_at timestamps to the current time. Additionally, it can set the sync_cursor property for the sync job.

This API is not intended for direct connector management by users. It supports the implementation of services that utilize the connector protocol to communicate with Elasticsearch.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

application/json

Body Required

  • sync_cursor object

    The cursor object from the last incremental sync job. This should reference the sync_cursor field in the connector state for which the job runs.

  • worker_hostname string Required

    The host name of the current system that will run the job.

Responses

PUT /_connector/_sync_job/{connector_sync_job_id}/_claim 
curl \
 --request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_claim' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"sync_cursor":{},"worker_hostname":"string"}'

Delete a connector sync job Beta

DELETE /_connector/_sync_job/{connector_sync_job_id}
Api key auth Basic auth Bearer auth

Remove a connector sync job and its associated data. This is a destructive action that is not recoverable.

Path parameters

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_connector/_sync_job/{connector_sync_job_id} 
curl \
 --request DELETE 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}

Get all connector sync jobs Beta

GET /_connector/_sync_job
Api key auth Basic auth Bearer auth

Get information about all stored connector sync jobs listed by their creation date in ascending order.

Query parameters

  • from number

    Starting offset (default: 0)

  • size number

    Specifies a max number of results to get

  • status string

    A sync job status to fetch connector sync jobs for

    Values are canceling, canceled, completed, error, in_progress, pending, or suspended.

  • connector_id string

    A connector id to fetch connector sync jobs for

  • job_type string | array[string]

    A comma-separated list of job types to fetch the sync jobs for

Responses

GET /_connector/_sync_job 
curl \
 --request GET 'http://api.example.com/_connector/_sync_job' \
 --header "Authorization: $API_KEY"

Create a connector sync job Beta

POST /_connector/_sync_job
Api key auth Basic auth Bearer auth

Create a connector sync job document in the internal index and initialize its counters and timestamps with default values.

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • id string Required
POST /_connector/_sync_job 
curl \
 --request POST 'http://api.example.com/_connector/_sync_job' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"id\": \"connector-id\",\n  \"job_type\": \"full\",\n  \"trigger_method\": \"on_demand\"\n}"'
Request example 
{
  "id": "connector-id",
  "job_type": "full",
  "trigger_method": "on_demand"
}

Update the connector configuration Beta

PUT /_connector/{connector_id}/_configuration
Api key auth Basic auth Bearer auth

Update the configuration field in the connector document.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_connector/{connector_id}/_configuration 
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_configuration' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"values\": {\n        \"tenant_id\": \"my-tenant-id\",\n        \"tenant_name\": \"my-sharepoint-site\",\n        \"client_id\": \"foo\",\n        \"secret_value\": \"bar\",\n        \"site_collections\": \"*\"\n    }\n}"'
Request examples 
{
    "values": {
        "tenant_id": "my-tenant-id",
        "tenant_name": "my-sharepoint-site",
        "client_id": "foo",
        "secret_value": "bar",
        "site_collections": "*"
    }
}
{
    "values": {
        "secret_value": "foo-bar"
    }
}
Response examples (200) 
{
  "result": "updated"
}

Update the connector pipeline Beta

PUT /_connector/{connector_id}/_pipeline
Api key auth Basic auth Bearer auth

When you create a new connector, the configuration of an ingest pipeline is populated with default settings.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_connector/{connector_id}/_pipeline 
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_pipeline' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"pipeline\": {\n        \"extract_binary_content\": true,\n        \"name\": \"my-connector-pipeline\",\n        \"reduce_whitespace\": true,\n        \"run_ml_inference\": true\n    }\n}"'
Request example 
{
    "pipeline": {
        "extract_binary_content": true,
        "name": "my-connector-pipeline",
        "reduce_whitespace": true,
        "run_ml_inference": true
    }
}
Response examples (200) 
{
  "result": "updated"
}

Update the connector status Technical preview

PUT /_connector/{connector_id}/_status
Api key auth Basic auth Bearer auth

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • status string Required

    Values are created, needs_configuration, configured, connected, or error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_connector/{connector_id}/_status 
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_status' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"status\": \"needs_configuration\"\n}"'
Request example 
{
    "status": "needs_configuration"
}
Response examples (200) 
{
  "result": "updated"
}

Get auto-follow patterns Added in 6.5.0

GET /_ccr/auto_follow/{name}
Api key auth Basic auth Bearer auth

Get cross-cluster replication auto-follow patterns.

External documentation

Path parameters

  • name string Required

    The auto-follow pattern collection that you want to retrieve. If you do not specify a name, the API returns information for all collections.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

Responses

GET /_ccr/auto_follow/{name} 
curl \
 --request GET 'http://api.example.com/_ccr/auto_follow/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ccr/auto_follow/my_auto_follow_pattern`, which gets auto-follow patterns. 
{
  "patterns": [
    {
      "name": "my_auto_follow_pattern",
      "pattern": {
        "active": true,
        "remote_cluster" : "remote_cluster",
        "leader_index_patterns" :
        [
          "leader_index*"
        ],
        "leader_index_exclusion_patterns":
        [
          "leader_index_001"
        ],
        "follow_index_pattern" : "{{leader_index}}-follower"
      }
    }
  ]
}

Get data streams Added in 7.9.0

GET /_data_stream/{name}
Api key auth Basic auth Bearer auth

Get information about one or more data streams.

Path parameters

  • name string | array[string] Required

    Comma-separated list of data stream names used to limit the request. Wildcard (*) expressions are supported. If omitted, all data streams are returned.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • verbose boolean

    Whether the maximum timestamp for each data stream should be calculated and returned.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • allow_custom_routing boolean

        If true, the data stream allows custom routing on write request.

      • failure_store object
        Hide failure_store attributes Show failure_store attributes object
      • generation number Required

        Current generation for the data stream. This number acts as a cumulative count of the stream’s rollovers, starting at 1.

      • hidden boolean Required

        If true, the data stream is hidden.

      • ilm_policy string
      • next_generation_managed_by string Required

        Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

      • prefer_ilm boolean Required

        Indicates if ILM should take precedence over DSL in case both are configured to managed this data stream.

      • indices array[object] Required

        Array of objects containing information about the data stream’s backing indices. The last item in this array contains information about the stream’s current write index.

        Hide indices attributes Show indices attributes object
      • lifecycle object
        Hide lifecycle attributes Show lifecycle attributes object
      • name string Required
      • replicated boolean

        If true, the data stream is created and managed by cross-cluster replication and the local cluster can not write into this data stream or change its mappings.

      • rollover_on_write boolean Required

        If true, the next write to this data stream will trigger a rollover first and the document will be indexed in the new backing index. If the rollover fails the indexing request will fail too.

      • status string Required

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • system boolean

        If true, the data stream is created and managed by an Elastic stack component and cannot be modified through normal user interaction.

      • template string Required
      • timestamp_field object Required
        Hide timestamp_field attribute Show timestamp_field attribute object
        • name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_mode string

        Values are standard, time_series, logsdb, or lookup.
GET /_data_stream/{name} 
curl \
 --request GET 'http://api.example.com/_data_stream/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving information about a data stream. 
{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2099.03.07-000001",
          "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        },
        {
          "index_name": ".ds-my-data-stream-2099.03.08-000002",
          "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 2,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "GREEN",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    },
    {
      "name": "my-data-stream-two",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-two-2099.03.08-000001",
          "index_uuid": "3liBu2SYS5axasRt6fUIpA",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "YELLOW",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}

Get data streams Added in 7.9.0

GET /_data_stream
Api key auth Basic auth Bearer auth

Get information about one or more data streams.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • verbose boolean

    Whether the maximum timestamp for each data stream should be calculated and returned.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • allow_custom_routing boolean

        If true, the data stream allows custom routing on write request.

      • failure_store object
        Hide failure_store attributes Show failure_store attributes object
      • generation number Required

        Current generation for the data stream. This number acts as a cumulative count of the stream’s rollovers, starting at 1.

      • hidden boolean Required

        If true, the data stream is hidden.

      • ilm_policy string
      • next_generation_managed_by string Required

        Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

      • prefer_ilm boolean Required

        Indicates if ILM should take precedence over DSL in case both are configured to managed this data stream.

      • indices array[object] Required

        Array of objects containing information about the data stream’s backing indices. The last item in this array contains information about the stream’s current write index.

        Hide indices attributes Show indices attributes object
      • lifecycle object
        Hide lifecycle attributes Show lifecycle attributes object
      • name string Required
      • replicated boolean

        If true, the data stream is created and managed by cross-cluster replication and the local cluster can not write into this data stream or change its mappings.

      • rollover_on_write boolean Required

        If true, the next write to this data stream will trigger a rollover first and the document will be indexed in the new backing index. If the rollover fails the indexing request will fail too.

      • status string Required

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • system boolean

        If true, the data stream is created and managed by an Elastic stack component and cannot be modified through normal user interaction.

      • template string Required
      • timestamp_field object Required
        Hide timestamp_field attribute Show timestamp_field attribute object
        • name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_mode string

        Values are standard, time_series, logsdb, or lookup.
GET /_data_stream 
curl \
 --request GET 'http://api.example.com/_data_stream' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving information about a data stream. 
{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2099.03.07-000001",
          "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        },
        {
          "index_name": ".ds-my-data-stream-2099.03.08-000002",
          "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 2,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "GREEN",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    },
    {
      "name": "my-data-stream-two",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-two-2099.03.08-000001",
          "index_uuid": "3liBu2SYS5axasRt6fUIpA",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "YELLOW",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}

Promote a data stream Added in 7.9.0

POST /_data_stream/_promote/{name}
Api key auth Basic auth Bearer auth

Promote a data stream from a replicated data stream managed by cross-cluster replication (CCR) to a regular data stream.

With CCR auto following, a data stream from a remote cluster can be replicated to the local cluster. These data streams can't be rolled over in the local cluster. These replicated data streams roll over only if the upstream data stream rolls over. In the event that the remote cluster is no longer available, the data stream in the local cluster can be promoted to a regular data stream, which allows these data streams to be rolled over in the local cluster.

NOTE: When promoting a data stream, ensure the local cluster has a data stream enabled index template that matches the data stream. If this is missing, the data stream will not be able to roll over until a matching index template is created. This will affect the lifecycle management of the data stream and interfere with the data stream size and retention.

Path parameters

  • name string Required

    The name of the data stream

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

POST /_data_stream/_promote/{name} 
curl \
 --request POST 'http://api.example.com/_data_stream/_promote/{name}' \
 --header "Authorization: $API_KEY"

Create or update a document in an index

PUT /{index}/_doc/{id}
Api key auth Basic auth Bearer auth

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
  • To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
  • To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
External documentation

Path parameters

  • index string Required

    The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.

  • id string Required

    A unique identifier for the document. To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • op_type string

    Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

    Values are index or create.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • timeout string

    The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.

  • version number

    An explicit version number for concurrency control. It must be a non-negative long number.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

object object

Responses

PUT /{index}/_doc/{id} 
curl \
 --request PUT 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'
Request examples
Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Response examples (200)
A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}
A successful response from `PUT my-index-000001/_doc/1`. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

Delete a document

DELETE /{index}/_doc/{id}
Api key auth Basic auth Bearer auth

Remove a JSON document from the specified index.

NOTE: You cannot send deletion requests directly to a data stream. To delete a document in a data stream, you must target the backing index containing the document.

Optimistic concurrency control

Delete operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Versioning

Each document indexed is versioned. When deleting a document, the version can be specified to make sure the relevant document you are trying to delete is actually being deleted and it has not changed in the meantime. Every write operation run on a document, deletes included, causes its version to be incremented. The version number of a deleted document remains available for a short time after deletion to allow for control of concurrent operations. The length of time for which a deleted document's version remains available is determined by the index.gc_deletes index setting.

Routing

If routing is used during indexing, the routing value also needs to be specified to delete a document.

If the _routing mapping is set to required and no routing value is specified, the delete API throws a RoutingMissingException and rejects the request.

For example:

DELETE /my-index-000001/_doc/1?routing=shard-1

This request deletes the document with ID 1, but it is routed based on the user. The document is not deleted if the correct routing is not specified.

Distributed

The delete operation gets hashed into a specific shard ID. It then gets redirected into the primary shard within that ID group and replicated (if needed) to shard replicas within that ID group.

Path parameters

  • index string Required

    The name of the target index.

  • id string Required

    A unique identifier for the document.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value used to route operations to a specific shard.

  • timeout string

    The period to wait for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the delete operation might not be available when the delete operation runs. Some reasons for this might be that the primary shard is currently recovering from a store or undergoing relocation. By default, the delete operation will wait on the primary shard to become available for up to 1 minute before failing and responding with an error.

  • version number

    An explicit version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The minimum number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

Responses

DELETE /{index}/_doc/{id} 
curl \
 --request DELETE 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /my-index-000001/_doc/1`, which deletes the JSON document 1 from the `my-index-000001` index. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 2,
  "_primary_term": 1,
  "_seq_no": 5,
  "result": "deleted"
}

Check for a document source Added in 5.4.0

HEAD /{index}/_source/{id}
Api key auth Basic auth Bearer auth

Check whether a document source exists in an index. For example:

HEAD my-index-000001/_source/1

A document's source is not available if it is disabled in the mapping.

External documentation

Path parameters

  • index string Required

    A comma-separated list of data streams, indices, and aliases. It supports wildcards (*).

  • id string Required

    A unique identifier for the document.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude in the response.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

Responses

HEAD /{index}/_source/{id} 
curl \
 --request HEAD 'http://api.example.com/{index}/_source/{id}' \
 --header "Authorization: $API_KEY"

Get multiple documents Added in 1.3.0

POST /_mget
Api key auth Basic auth Bearer auth

Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.

Filter source fields

By default, the _source field is returned for every document (if stored). Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document. You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.

Get stored fields

Use the stored_fields attribute to specify the set of stored fields you want to retrieve. Any requested fields that are not stored are ignored. You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.

Query parameters

  • force_synthetic_source boolean

    Should this request force synthetic _source? Use this to test if the mapping supports synthetic _source and to get a sense of the worst case performance. Fetches with this enabled will be slower the enabling synthetic source natively in the index.

  • preference string

    Specifies the node or shard the operation should be performed on. Random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes relevant shards before retrieving documents.

  • routing string

    Custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    True or false to return the _source field or not, or a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    If true, retrieves the document fields stored in the index rather than the document _source.

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required

      The response includes a docs array that contains the documents in the order specified in the request. The structure of the returned documents is similar to that returned by the get API. If there is a failure getting a particular document, the error is included in place of the document.

      One of:
      GetResult object MultiGetError object
      Hide attributes Show attributes
      • _index string Required
      • fields object

        If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

        Hide fields attribute Show fields attribute object
        • * object Additional properties
      • _ignored array[string]
      • found boolean Required

        Indicates whether the document exists.

      • _id string Required
      • _primary_term number

        The primary term assigned to the document for the indexing operation.

      • _routing string

        The explicit routing, if set.

      • _seq_no number
      • _source object

        If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

      • _version number
POST /_mget 
curl \
 --request POST 'http://api.example.com/_mget' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": \"1\"\n    },\n    {\n      \"_id\": \"2\"\n    }\n  ]\n}"'
Request examples
Run `GET /my-index-000001/_mget`. When you specify an index in the request URI, only the document IDs are required in the request body. 
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
Run `GET /_mget`. This request sets `_source` to `false` for document 1 to exclude the source entirely. It retrieves `field3` and `field4` from document 2. It retrieves the `user` field from document 3 but filters out the `user.location` field. 
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "_source": false
    },
    {
      "_index": "test",
      "_id": "2",
      "_source": [ "field3", "field4" ]
    },
    {
      "_index": "test",
      "_id": "3",
      "_source": {
        "include": [ "user" ],
        "exclude": [ "user.location" ]
      }
    }
  ]
}
Run `GET /_mget`. This request retrieves `field1` and `field2` from document 1 and `field3` and `field4` from document 2. 
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "stored_fields": [ "field1", "field2" ]
    },
    {
      "_index": "test",
      "_id": "2",
      "stored_fields": [ "field3", "field4" ]
    }
  ]
}
Run `GET /_mget?routing=key1`. If routing is used during indexing, you need to specify the routing value to retrieve documents. This request fetches `test/_doc/2` from the shard corresponding to routing key `key1`. It fetches `test/_doc/1` from the shard corresponding to routing key `key2`. 
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "routing": "key2"
    },
    {
      "_index": "test",
      "_id": "2"
    }
  ]
}

Get multiple term vectors

GET /_mtermvectors
Api key auth Basic auth Bearer auth

Get multiple term vectors with a single request. You can specify existing documents by index and ID or provide artificial documents in the body of the request. You can specify the index in the request body or request URI. The response contains a docs array with all the fetched termvectors. Each element has the structure provided by the termvectors API.

Artificial documents

You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified _index.

Query parameters

  • ids array[string]

    A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes term frequency and document frequency.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • docs array[object]

    An array of existing or artificial documents.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • doc object

      An artificial document (a document not present in the index) for which you want to retrieve term vectors.

    • fields string | array[string]
    • field_statistics boolean

      If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

    • filter object
      Hide filter attributes Show filter attributes object
      • max_doc_freq number

        Ignore words which occur in more than this many docs. Defaults to unbounded.

      • max_num_terms number

        The maximum number of terms that must be returned per field.

      • max_term_freq number

        Ignore words with more than this frequency in the source doc. It defaults to unbounded.

      • max_word_length number

        The maximum word length above which words will be ignored. Defaults to unbounded.

      • min_doc_freq number

        Ignore terms which do not occur in at least this many docs.

      • min_term_freq number

        Ignore words with less than this frequency in the source doc.

      • min_word_length number

        The minimum word length below which words will be ignored.

    • offsets boolean

      If true, the response includes term offsets.

    • payloads boolean

      If true, the response includes term payloads.

    • positions boolean

      If true, the response includes term positions.

    • routing string
    • term_statistics boolean

      If true, the response includes term frequency and document frequency.

    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

  • ids array[string]

    A simplified syntax to specify documents by their ID if they're in the same index.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /_mtermvectors 
curl \
 --request GET 'http://api.example.com/_mtermvectors' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n      {\n        \"_id\": \"2\",\n        \"fields\": [\n            \"message\"\n        ],\n        \"term_statistics\": true\n      },\n      {\n        \"_id\": \"1\"\n      }\n  ]\n}"'
Request examples
Run `POST /my-index-000001/_mtermvectors`. When you specify an index in the request URI, the index does not need to be specified for each documents in the request body. 
{
  "docs": [
      {
        "_id": "2",
        "fields": [
            "message"
        ],
        "term_statistics": true
      },
      {
        "_id": "1"
      }
  ]
}
Run `POST /my-index-000001/_mtermvectors`. If all requested documents are in same index and the parameters are the same, you can use a simplified syntax. 
{
  "ids": [ "1", "2" ],
  "fields": [
    "message"
  ],
  "term_statistics": true
}
Run `POST /_mtermvectors` to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified `_index`. 
{
  "docs": [
      {
        "_index": "my-index-000001",
        "doc" : {
            "message" : "test test test"
        }
      },
      {
        "_index": "my-index-000001",
        "doc" : {
          "message" : "Another test ..."
        }
      }
  ]
}

Get term vector information

POST /{index}/_termvectors/{id}
Api key auth Basic auth Bearer auth

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

  • term frequency in the field (always returned)
  • term positions (positions: true)
  • start and end offsets (offsets: true)
  • term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.


Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique identifier for the document.

Query parameters

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • doc object

    An artificial document (a document not present in the index) for which you want to retrieve term vectors.

  • filter object
    Hide filter attributes Show filter attributes object
    • max_doc_freq number

      Ignore words which occur in more than this many docs. Defaults to unbounded.

    • max_num_terms number

      The maximum number of terms that must be returned per field.

    • max_term_freq number

      Ignore words with more than this frequency in the source doc. It defaults to unbounded.

    • max_word_length number

      The maximum word length above which words will be ignored. Defaults to unbounded.

    • min_doc_freq number

      Ignore terms which do not occur in at least this many docs.

    • min_term_freq number

      Ignore words with less than this frequency in the source doc.

    • min_word_length number

      The minimum word length below which words will be ignored.

  • per_field_analyzer object

    Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.

    Hide per_field_analyzer attribute Show per_field_analyzer attribute object
    • * string Additional properties
  • fields string | array[string]
  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • routing string
  • version number
  • version_type string

    Values are internal, external, external_gte, or force.

Responses

POST /{index}/_termvectors/{id} 
curl \
 --request POST 'http://api.example.com/{index}/_termvectors/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'
Request examples
Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1. 
{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter. 
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}
Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low. 
{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}
Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated. 
{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created. 
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_termvectors/1`. 
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body. 
{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body. 
{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}

Update a document

POST /{index}/_update/{id}
Api key auth Basic auth Bearer auth

Update a document by running a script or passing a partial document.

If the Elasticsearch security features are enabled, you must have the index or write index privilege for the target index or index alias.

The script can update, delete, or skip modifying the document. The API also supports passing a partial document, which is merged into the existing document. To fully replace an existing document, use the index API. This operation:

  • Gets the document (collocated with the shard) from the index.
  • Runs the specified script.
  • Indexes the result.

The document must still be reindexed, but using this API removes some network roundtrips and reduces chances of version conflicts between the GET and the index operation.

The _source field must be enabled to use this API. In addition to _source, you can access the following variables through the ctx map: _index, _type, _id, _version, _routing, and _now (the current timestamp).

Path parameters

  • index string Required

    The name of the target index. By default, the index is created automatically if it doesn't exist.

  • id string Required

    A unique identifier for the document to be updated.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • lang string

    The script language.

  • refresh string

    If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', it does nothing with refreshes.

    Values are true, false, or wait_for.

  • require_alias boolean

    If true, the destination must be an index alias.

  • retry_on_conflict number

    The number of times the operation should be retried when a conflict occurs.

  • routing string

    A custom value used to route operations to a specific shard.

  • timeout string

    The period to wait for the following operations: dynamic mapping updates and waiting for active shards. Elasticsearch waits for at least the timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.

  • wait_for_active_shards number | string

    The number of copies of each shard that must be active before proceeding with the operation. Set to 'all' or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

  • _source boolean | string | array[string]

    If false, source retrieval is turned off. You can also specify a comma-separated list of the fields you want to retrieve.

  • _source_excludes string | array[string]

    The source fields you want to exclude.

  • _source_includes string | array[string]

    The source fields you want to retrieve.

application/json

Body Required

  • detect_noop boolean

    If true, the result in the response is set to noop (no operation) when there are no changes to the document.

  • doc object

    A partial update to an existing document. If both doc and script are specified, doc is ignored.

  • doc_as_upsert boolean

    If true, use the contents of 'doc' as the value of 'upsert'. NOTE: Using ingest pipelines with doc_as_upsert is not supported.

  • script object
    Hide script attributes Show script attributes object
  • scripted_upsert boolean

    If true, run the script whether or not the document exists.

  • _source boolean | object

    Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

    One of:
    SourceConfig boolean SourceFilter object
  • upsert object

    If the document does not already exist, the contents of 'upsert' are inserted as a new document. If the document exists, the 'script' is run.

Responses

POST /{index}/_update/{id} 
curl \
 --request POST 'http://api.example.com/{index}/_update/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"script\" : {\n    \"source\": \"ctx._source.counter += params.count\",\n    \"lang\": \"painless\",\n    \"params\" : {\n      \"count\" : 4\n    }\n  }\n}"'
Request examples
Run `POST test/_update/1` to increment a counter by using a script. 
{
  "script" : {
    "source": "ctx._source.counter += params.count",
    "lang": "painless",
    "params" : {
      "count" : 4
    }
  }
}
Run `POST test/_update/1` to perform a scripted upsert. When `scripted_upsert` is `true`, the script runs whether or not the document exists. 
{
  "scripted_upsert": true,
  "script": {
    "source": """
      if ( ctx.op == 'create' ) {
        ctx._source.counter = params.count
      } else {
        ctx._source.counter += params.count
      }
    """,
    "params": {
      "count": 4
    }
  },
  "upsert": {}
}
Run `POST test/_update/1` to perform a doc as upsert. Instead of sending a partial `doc` plus an `upsert` doc, you can set `doc_as_upsert` to `true` to use the contents of `doc` as the `upsert` value. 
{
  "doc": {
    "name": "new_name"
  },
  "doc_as_upsert": true
}
Run `POST test/_update/1` to use a script to add a tag to a list of tags. In this example, it is just a list, so the tag is added even it exists. 
{
  "script": {
    "source": "ctx._source.tags.add(params.tag)",
    "lang": "painless",
    "params": {
      "tag": "blue"
    }
  }
}
Run `POST test/_update/1` to use a script to remove a tag from a list of tags. The Painless function to remove a tag takes the array index of the element you want to remove. To avoid a possible runtime error, you first need to make sure the tag exists. If the list contains duplicates of the tag, this script just removes one occurrence. 
{
  "script": {
    "source": "if (ctx._source.tags.contains(params.tag)) { ctx._source.tags.remove(ctx._source.tags.indexOf(params.tag)) }",
    "lang": "painless",
    "params": {
      "tag": "blue"
    }
  }
}
Run `POST test/_update/1` to use a script to add a field `new_field` to the document. 
{
  "script" : "ctx._source.new_field = 'value_of_new_field'"
}
Run `POST test/_update/1` to use a script to remove a field `new_field` from the document. 
{
  "script" : "ctx._source.remove('new_field')"
}
Run `POST test/_update/1` to use a script to remove a subfield from an object field. 
{
  "script": "ctx._source['my-object'].remove('my-subfield')"
}
Run `POST test/_update/1` to change the operation that runs from within the script. For example, this request deletes the document if the `tags` field contains `green`, otherwise it does nothing (`noop`). 
{
  "script": {
    "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'noop' }",
    "lang": "painless",
    "params": {
      "tag": "green"
    }
  }
}
Run `POST test/_update/1` to do a partial update that adds a new field to the existing document. 
{
  "doc": {
    "name": "new_name"
  }
}
Run `POST test/_update/1` to perfom an upsert. If the document does not already exist, the contents of the upsert element are inserted as a new document. If the document exists, the script is run. 
{
  "script": {
    "source": "ctx._source.counter += params.count",
    "lang": "painless",
    "params": {
      "count": 4
    }
  },
  "upsert": {
    "counter": 1
  }
}
Response examples (200)
By default updates that don't change anything detect that they don't change anything and return `"result": "noop"`. 
{
   "_shards": {
        "total": 0,
        "successful": 0,
        "failed": 0
   },
   "_index": "test",
   "_id": "1",
   "_version": 2,
   "_primary_term": 1,
   "_seq_no": 1,
   "result": "noop"
}

Stop async ES|QL query Added in 8.18.0

POST /_query/async/{id}/stop
Api key auth Basic auth Bearer auth

This API interrupts the query execution and returns the results so far. If the Elasticsearch security features are enabled, only the user who first submitted the ES|QL query can stop it.

External documentation

Path parameters

  • id string Required

    The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Query parameters

  • drop_null_columns boolean

    Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • took number

      Time unit for milliseconds

    • is_partial boolean
    • all_columns array[object]
      Hide all_columns attributes Show all_columns attributes object
    • columns array[object] Required
      Hide columns attributes Show columns attributes object
    • values array[array] Required

      A field value.

      A field value.

    • _clusters object
      Hide _clusters attributes Show _clusters attributes object
    • profile object

      Profiling information. Present if profile was true in the request. The contents of this field are currently unstable.
POST /_query/async/{id}/stop 
curl \
 --request POST 'http://api.example.com/_query/async/{id}/stop' \
 --header "Authorization: $API_KEY"

Features

The feature APIs enable you to introspect and manage features provided by Elasticsearch and Elasticsearch plugins.

Index

Index APIs enable you to manage individual indices, index settings, aliases, mappings, and index templates.

Add an index block Added in 7.9.0

PUT /{index}/_block/{block}
Api key auth Basic auth Bearer auth

Add an index block to an index. Index blocks limit the operations allowed on an index by blocking specific operation types.

Path parameters

  • index string Required

    A comma-separated list or wildcard expression of index names used to limit the request. By default, you must explicitly name the indices you are adding blocks to. To allow the adding of blocks to indices with _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false. You can update this setting in the elasticsearch.yml file or by using the cluster update settings API.

  • block string Required

    The block type to add to the index.

    Values are metadata, read, read_only, or write.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. It can also be set to -1 to indicate that the request should never timeout.

Responses

PUT /{index}/_block/{block} 
curl \
 --request PUT 'http://api.example.com/{index}/_block/{block}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `PUT /my-index-000001/_block/write`, which adds an index block to an index.' 
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : [ {
    "name" : "my-index-000001",
    "blocked" : true
  } ]
}

Get tokens from text analysis

GET /_analyze
Api key auth Basic auth Bearer auth

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory. The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced. If more than this limit of tokens gets generated, an error occurs. The _analyze endpoint without a specified index will always use 10000 as its limit.

External documentation

Query parameters

  • index string

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

application/json

Body

Responses

GET /_analyze 
curl \
 --request GET 'http://api.example.com/_analyze' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"analyzer\": \"standard\",\n  \"text\": \"this is a test\"\n}"'
Request examples
You can apply any of the built-in analyzers to the text string without specifying an index. 
{
  "analyzer": "standard",
  "text": "this is a test"
}
If the text parameter is provided as array of strings, it is analyzed as a multi-value field. 
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}
You can test a custom transient analyzer built from tokenizers, token filters, and char filters. Token filters use the filter parameter. 
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}
Custom tokenizers, token filters, and character filters can be specified in the request body. 
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` to run an analysis on the text using the default index analyzer associated with the `analyze_sample` index. Alternatively, the analyzer can be derived based on a field mapping. 
{
  "field": "obj1.field1",
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` and supply a normalizer for a keyword field if there is a normalizer associated with the specified index. 
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}
If you want to get more advanced details, set `explain` to `true`. It will output all token attributes for each token. You can filter token attributes you want to output by setting the `attributes` option. NOTE: The format of the additional detail information is labelled as experimental in Lucene and it may change in the future. 
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}
Response examples (200)
A successful response for an analysis with `explain` set to `true`. 
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Get tokens from text analysis

GET /{index}/_analyze
Api key auth Basic auth Bearer auth

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory. The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced. If more than this limit of tokens gets generated, an error occurs. The _analyze endpoint without a specified index will always use 10000 as its limit.

External documentation

Path parameters

  • index string Required

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

Query parameters

  • index string

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

application/json

Body

Responses

GET /{index}/_analyze 
curl \
 --request GET 'http://api.example.com/{index}/_analyze' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"analyzer\": \"standard\",\n  \"text\": \"this is a test\"\n}"'
Request examples
You can apply any of the built-in analyzers to the text string without specifying an index. 
{
  "analyzer": "standard",
  "text": "this is a test"
}
If the text parameter is provided as array of strings, it is analyzed as a multi-value field. 
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}
You can test a custom transient analyzer built from tokenizers, token filters, and char filters. Token filters use the filter parameter. 
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}
Custom tokenizers, token filters, and character filters can be specified in the request body. 
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` to run an analysis on the text using the default index analyzer associated with the `analyze_sample` index. Alternatively, the analyzer can be derived based on a field mapping. 
{
  "field": "obj1.field1",
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` and supply a normalizer for a keyword field if there is a normalizer associated with the specified index. 
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}
If you want to get more advanced details, set `explain` to `true`. It will output all token attributes for each token. You can filter token attributes you want to output by setting the `attributes` option. NOTE: The format of the additional detail information is labelled as experimental in Lucene and it may change in the future. 
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}
Response examples (200)
A successful response for an analysis with `explain` set to `true`. 
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Clone an index Added in 7.4.0

POST /{index}/_clone/{target}
Api key auth Basic auth Bearer auth

Clone an existing index into a new index. Each original primary shard is cloned into a new primary shard in the new index.

IMPORTANT: Elasticsearch does not apply index templates to the resulting index. The API also does not copy index metadata from the original index. Index metadata includes aliases, index lifecycle management phase definitions, and cross-cluster replication (CCR) follower information. For example, if you clone a CCR follower index, the resulting clone will not be a follower index.

The clone API copies most index settings from the source index to the resulting index, with the exception of index.number_of_replicas and index.auto_expand_replicas. To set the number of replicas in the resulting index, configure these settings in the clone request.

Cloning works as follows:

  • First, it creates a new target index with the same definition as the source index.
  • Then it hard-links segments from the source index into the target index. If the file system does not support hard-linking, all segments are copied into the new index, which is a much more time consuming process.
  • Finally, it recovers the target index as though it were a closed index which had just been re-opened.

IMPORTANT: Indices can only be cloned if they meet the following requirements:

  • The index must be marked as read-only and have a cluster health status of green.
  • The target index must not exist.
  • The source index must have the same number of primary shards as the target index.
  • The node handling the clone process must have sufficient free disk space to accommodate a second copy of the existing index.

The current write index on a data stream cannot be cloned. In order to clone the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be cloned.

NOTE: Mappings cannot be specified in the _clone request. The mappings of the source index will be used for the target index.

Monitor the cloning process

The cloning process can be monitored with the cat recovery API or the cluster health API can be used to wait until all primary shards have been allocated by setting the wait_for_status parameter to yellow.

The _clone API returns as soon as the target index has been added to the cluster state, before any shards have been allocated. At this point, all shards are in the state unassigned. If, for any reason, the target index can't be allocated, its primary shard will remain unassigned until it can be allocated on that node.

Once the primary shard is allocated, it moves to state initializing, and the clone process begins. When the clone operation completes, the shard will become active. At that point, Elasticsearch will try to allocate any replicas and may decide to relocate the primary shard to another node.

Wait for active shards

Because the clone operation creates a new index to clone the shards to, the wait for active shards setting on index creation applies to the clone index action as well.

Path parameters

  • index string Required

    Name of the source index to clone.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

application/json

Body

  • aliases object

    Aliases for the resulting index.

    Hide aliases attribute Show aliases attribute object
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

POST /{index}/_clone/{target} 
curl \
 --request POST 'http://api.example.com/{index}/_clone/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.number_of_shards\": 5\n  },\n  \"aliases\": {\n    \"my_search_indices\": {}\n  }\n}"'
Request example
Clone `my_source_index` into a new index called `my_target_index` with `POST /my_source_index/_clone/my_target_index`. The API accepts `settings` and `aliases` parameters for the target index. 
{
  "settings": {
    "index.number_of_shards": 5
  },
  "aliases": {
    "my_search_indices": {}
  }
}

Create or update an alias

PUT /{index}/_alias/{name}
Api key auth Basic auth Bearer auth

Adds a data stream or index to an alias.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices to add. Supports wildcards (*). Wildcard patterns that match both data streams and indices return an error.

  • name string Required

    Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body

  • filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • index_routing string
  • is_write_index boolean

    If true, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests. If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.

  • routing string
  • search_routing string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /{index}/_alias/{name} 
curl \
 --request PUT 'http://api.example.com/{index}/_alias/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"actions\": [\n    {\n      \"add\": {\n        \"index\": \"my-data-stream\",\n        \"alias\": \"my-alias\"\n      }\n    }\n  ]\n}"'
Request example 
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}

Delete data stream lifecycles Added in 8.11.0

DELETE /_data_stream/{name}/_lifecycle
Api key auth Basic auth Bearer auth

Removes the data stream lifecycle from a data stream, rendering it not managed by the data stream lifecycle.

Path parameters

  • name string | array[string] Required

    A comma-separated list of data streams of which the data stream lifecycle will be deleted; use * to get all data streams

Query parameters

  • expand_wildcards string | array[string]

    Whether wildcard expressions should get expanded to open or closed indices (default: open)

  • master_timeout string

    Specify timeout for connection to master

  • timeout string

    Explicit timestamp for the document

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_data_stream/{name}/_lifecycle 
curl \
 --request DELETE 'http://api.example.com/_data_stream/{name}/_lifecycle' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for deleting a data stream lifecycle. 
{
  "acknowledged": true
}

Check aliases

HEAD /_alias/{name}
Api key auth Basic auth Bearer auth

Check if one or more data stream or index aliases exist.

Path parameters

  • name string | array[string] Required

    Comma-separated list of aliases to check. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, requests that include a missing data stream or index in the target indices or data streams return an error.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

HEAD /_alias/{name} 
curl \
 --request HEAD 'http://api.example.com/_alias/{name}' \
 --header "Authorization: $API_KEY"

Get field usage stats Technical preview

GET /{index}/_field_usage_stats
Api key auth Basic auth Bearer auth

Get field usage information for each shard and field of an index. Field usage statistics are automatically captured when queries are running on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.

The response body reports the per-shard usage count of the data structures that back the fields in the index. A given request will increment each count by a maximum value of 1, even if the request accesses the same field multiple times.

Path parameters

  • index string | array[string] Required

    Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

Responses

GET /{index}/_field_usage_stats 
curl \
 --request GET 'http://api.example.com/{index}/_field_usage_stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_field_usage_stats`. The `all_fields` object reports the sums of the usage counts for all fields in the index (on the listed shard). 
{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "my-index-000001": {
    "shards": [
      {
        "tracking_id": "MpOl0QlTQ4SYYhEe6KgJoQ",
        "tracking_started_at_millis": 1625558985010,
        "routing": {
          "state": "STARTED",
          "primary": true,
          "node": "gA6KeeVzQkGURFCUyV-e8Q",
          "relocating_node": null
        },
        "stats": {
          "all_fields": {
            "any": "6",
            "inverted_index": {
              "terms": 1,
              "postings": 1,
              "proximity": 1,
              "positions": 0,
              "term_frequencies": 1,
              "offsets": 0,
              "payloads": 0
            },
            "stored_fields": 2,
            "doc_values": 1,
            "points": 0,
            "norms": 1,
            "term_vectors": 0,
            "knn_vectors": 0
          },
          "fields": {
            "_id": {
              "any": 1,
              "inverted_index": {
                "terms": 1,
                "postings": 1,
                "proximity": 1,
                "positions": 0,
                "term_frequencies": 1,
                "offsets": 0,
                "payloads": 0
              },
              "stored_fields": 1,
              "doc_values": 0,
              "points": 0,
              "norms": 0,
              "term_vectors": 0,
              "knn_vectors": 0
            },
            "_source": {},
            "context": {},
            "message.keyword": {}
          }
        }
      }
    ]
  }
}

Get mapping definitions

GET /_mapping
Api key auth Basic auth Bearer auth

For data streams, the API retrieves mappings for the stream’s backing indices.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean Deprecated

    If true, the request retrieves information from the local node only.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /_mapping 
curl \
 --request GET 'http://api.example.com/_mapping' \
 --header "Authorization: $API_KEY"

Resolve the cluster Added in 8.13.0

GET /_resolve/cluster/{name}
Api key auth Basic auth Bearer auth

Resolve the specified index expressions to return information about each cluster, including the local "querying" cluster, if included. If no index expression is provided, the API will return information about all the remote clusters that are configured on the querying cluster.

This endpoint is useful before doing a cross-cluster search in order to determine which remote clusters should be included in a search.

You use the same index expression with this endpoint as you would for cross-cluster search. Index and cluster exclusions are also supported with this endpoint.

For each cluster in the index expression, information is returned about:

  • Whether the querying ("local") cluster is currently connected to each remote cluster specified in the index expression. Note that this endpoint actively attempts to contact the remote clusters, unlike the remote/info endpoint.
  • Whether each remote cluster is configured with skip_unavailable as true or false.
  • Whether there are any indices, aliases, or data streams on that cluster that match the index expression.
  • Whether the search is likely to have errors returned when you do the cross-cluster search (including any authorization errors if you do not have permission to query the index).
  • Cluster version information, including the Elasticsearch server version.

For example, GET /_resolve/cluster/my-index-*,cluster*:my-index-* returns information about the local cluster and all remotely configured clusters that start with the alias cluster*. Each cluster returns information about whether it has any indices, aliases or data streams that match my-index-*.

Note on backwards compatibility

The ability to query without an index expression was added in version 8.18, so when querying remote clusters older than that, the local cluster will send the index expression dummy* to those remote clusters. Thus, if an errors occur, you may see a reference to that index expression even though you didn't request it. If it causes a problem, you can instead include an index expression like *:* to bypass the issue.

You may want to exclude a cluster or index from a search when:

  • A remote cluster is not currently connected and is configured with skip_unavailable=false. Running a cross-cluster search under those conditions will cause the entire search to fail.
  • A cluster has no matching indices, aliases or data streams for the index expression (or your user does not have permissions to search them). For example, suppose your index expression is logs*,remote1:logs* and the remote1 cluster has no indices, aliases or data streams that match logs*. In that case, that cluster will return no results from that cluster if you include it in a cross-cluster search.
  • The index expression (combined with any query parameters you specify) will likely cause an exception to be thrown when you do the search. In these cases, the "error" field in the _resolve/cluster response will be present. (This is also where security/permission errors will be shown.)
  • A remote cluster is an older version that does not support the feature you want to use in your search.

Test availability of remote clusters

The remote/info endpoint is commonly used to test whether the "local" cluster (the cluster being queried) is connected to its remote clusters, but it does not necessarily reflect whether the remote cluster is available or not. The remote cluster may be available, while the local cluster is not currently connected to it.

You can use the _resolve/cluster API to attempt to reconnect to remote clusters. For example with GET _resolve/cluster or GET _resolve/cluster/*:*. The connected field in the response will indicate whether it was successful. If a connection was (re-)established, this will also cause the remote/info endpoint to now indicate a connected status.

Path parameters

  • name string | array[string] Required

    A comma-separated list of names or index patterns for the indices, aliases, and data streams to resolve. Resources on remote clusters can be specified using the <cluster>:<name> syntax. Index and cluster exclusions (e.g., -cluster1:*) are also supported. If no index expression is specified, information about all remote clusters configured on the local cluster is returned without doing any index matching

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded, or aliased indices are ignored when frozen. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • timeout string

    The maximum time to wait for remote clusters to respond. If a remote cluster does not respond within this timeout period, the API response will show the cluster as not connected and include an error message that the request timed out.

    The default timeout is unset and the query can take as long as the networking layer is configured to wait for remote clusters that are not responding (typically 30 seconds).

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
GET /_resolve/cluster/{name} 
curl \
 --request GET 'http://api.example.com/_resolve/cluster/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_resolve/cluster/my-index*,clust*:my-index*`. Each cluster has its own response section. The cluster you sent the request to is labelled as "(local)". 
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  }
}
A successful response from `GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailable=false&timeout=5s`. This type of request can be used to identify potential problems with your cross-cluster search. Note also that a `timeout` of 5 seconds is sent, which sets the maximum time the query will wait for remote clusters to respond. The local cluster has no index called `not_present`. Searching with `ignore_unavailable=false` would return a "no such index" error. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. The `cluster_two` remote cluster is not connected (the attempt to connect failed). Since this cluster is marked as `skip_unavailable=false`, you should probably exclude this cluster from the search by adding `-cluster_two:*` to the search index expression. For `cluster_three`, the error message indicates that this remote cluster did not respond within the 5-second timeout window specified, so it is also marked as not connected. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API, so you may want to exclude it from your cross-cluster search. 
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "error": "no such index [not_present]"
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": false,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": false,
    "skip_unavailable": false
  },
  "cluster_three": {
    "connected": false,
    "skip_unavailable": false,
    "error": "Request timed out before receiving a response from the remote cluster"
  },
  "oldcluster": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true
  }
}

Get index segments

GET /{index}/_segments
Api key auth Basic auth Bearer auth

Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the stream's backing indices.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
GET /{index}/_segments 
curl \
 --request GET 'http://api.example.com/{index}/_segments' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for creating a new index for a data stream. 
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "old_index": ".ds-my-data-stream-2099.05.06-000001",
  "new_index": ".ds-my-data-stream-2099.05.07-000002",
  "rolled_over": true,
  "dry_run": false,
  "lazy": false,
  "conditions": {
    "[max_age: 7d]": false,
    "[max_docs: 1000]": true,
    "[max_primary_shard_size: 50gb]": false,
    "[max_primary_shard_docs: 2000]": false
  }
}

Get index shard stores

GET /_shard_stores
Api key auth Basic auth Bearer auth

Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.

The index shard stores API returns the following information:

  • The node on which each replica shard exists.
  • The allocation ID for each replica shard.
  • A unique ID for each replica shard.
  • Any errors encountered while opening the shard index or from an earlier failure.

By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • status string | array[string]

    List of shard health statuses used to limit the request.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • shards object Required
          Hide shards attribute Show shards attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
GET /_shard_stores 
curl \
 --request GET 'http://api.example.com/_shard_stores' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /_shard_stores?status=green`. 
{
  "indices": {
    "my-index-000001": {
      "shards": {
        "0": {
          "stores": [
            {
              "sPa3OgxLSYGvQ4oPs-Tajw": {
                "name": "node_t0",
                "ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
                "transport_address": "local[1]",
                "external_id": "node_t0",
                "attributes": {},
                "roles": [],
                "version": "8.10.0",
                "min_index_version": 7000099,
                "max_index_version": 8100099
              },
              "allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
              "allocation": "primary",
              "store_exception": {}
            }
          ]
        }
      }
    }
  }
}

Split an index Added in 6.1.0

POST /{index}/_split/{target}
Api key auth Basic auth Bearer auth

Split an index into a new index with more primary shards.

  • Before you can split an index:

  • The index must be read-only.

  • The cluster health status must be green.

You can do make an index read-only with the following request using the add index block API:

PUT /my_source_index/_block/write

The current write index on a data stream cannot be split. In order to split the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be split.

The number of times the index can be split (and the number of shards that each original shard can be split into) is determined by the index.number_of_routing_shards setting. The number of routing shards specifies the hashing space that is used internally to distribute documents across shards with consistent hashing. For instance, a 5 shard index with number_of_routing_shards set to 30 (5 x 2 x 3) could be split by a factor of 2 or 3.

A split operation:

  • Creates a new target index with the same definition as the source index, but with a larger number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system doesn't support hard-linking, all segments are copied into the new index, which is a much more time consuming process.
  • Hashes all documents again, after low level files are created, to delete documents that belong to a different shard.
  • Recovers the target index as though it were a closed index which had just been re-opened.

IMPORTANT: Indices can only be split if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have fewer primary shards than the target index.
  • The number of primary shards in the target index must be a multiple of the number of primary shards in the source index.
  • The node handling the split process must have sufficient free disk space to accommodate a second copy of the existing index.

Path parameters

  • index string Required

    Name of the source index to split.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

application/json

Body

  • aliases object

    Aliases for the resulting index.

    Hide aliases attribute Show aliases attribute object
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

POST /{index}/_split/{target} 
curl \
 --request POST 'http://api.example.com/{index}/_split/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.number_of_shards\": 2\n  }\n}"'
Request example
Split an existing index into a new index with more primary shards. 
{
  "settings": {
    "index.number_of_shards": 2
  }
}

Create or update an alias Added in 1.3.0

POST /_aliases
Api key auth Basic auth Bearer auth

Adds a data stream or index to an alias.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_aliases 
curl \
 --request POST 'http://api.example.com/_aliases' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"actions":[{"add":{"alias":"string","aliases":"string","filter":{},"index":"string","indices":"string","index_routing":"string","is_hidden":true,"is_write_index":true,"routing":"string","search_routing":"string","must_exist":true},"remove":{"alias":"string","aliases":"string","index":"string","indices":"string","must_exist":true},"remove_index":{"index":"string","indices":"string","must_exist":true}}]}'

Index lifecycle management

Learn more about managing the index lifecycle

Get an inference endpoint Added in 8.11.0

GET /_inference/{task_type}/{inference_id}
Api key auth Basic auth Bearer auth

Path parameters

  • task_type string Required

    The task type

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The inference Id

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • endpoints array[object] Required
      Hide endpoints attributes Show endpoints attributes object
      • chunking_settings object
        Hide chunking_settings attributes Show chunking_settings attributes object
        • max_chunk_size number

          The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        • overlap number

          The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        • sentence_overlap number

          The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        • strategy string

          The chunking strategy: sentence or word.

      • service string Required

        The service type

      • service_settings object Required
      • task_settings object
      • inference_id string Required

        The inference Id

      • task_type string Required

        Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
GET /_inference/{task_type}/{inference_id} 
curl \
 --request GET 'http://api.example.com/_inference/{task_type}/{inference_id}' \
 --header "Authorization: $API_KEY"

Delete an inference endpoint Added in 8.11.0

DELETE /_inference/{task_type}/{inference_id}
Api key auth Basic auth Bearer auth

Path parameters

  • task_type string Required

    The task type

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The inference identifier.

Query parameters

  • dry_run boolean

    When true, the endpoint is not deleted and a list of ingest processors which reference this endpoint is returned.

  • force boolean

    When true, the inference endpoint is forcefully deleted even if it is still being used by ingest processors or semantic text fields.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • pipelines array[string] Required
DELETE /_inference/{task_type}/{inference_id} 
curl \
 --request DELETE 'http://api.example.com/_inference/{task_type}/{inference_id}' \
 --header "Authorization: $API_KEY"

Get an inference endpoint Added in 8.11.0

GET /_inference
Api key auth Basic auth Bearer auth

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • endpoints array[object] Required
      Hide endpoints attributes Show endpoints attributes object
      • chunking_settings object
        Hide chunking_settings attributes Show chunking_settings attributes object
        • max_chunk_size number

          The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        • overlap number

          The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        • sentence_overlap number

          The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        • strategy string

          The chunking strategy: sentence or word.

      • service string Required

        The service type

      • service_settings object Required
      • task_settings object
      • inference_id string Required

        The inference Id

      • task_type string Required

        Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
GET /_inference 
curl \
 --request GET 'http://api.example.com/_inference' \
 --header "Authorization: $API_KEY"

Create an Azure AI studio inference endpoint Added in 8.14.0

PUT /_inference/{task_type}/{azureaistudio_inference_id}
Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the azureaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are completion or text_embedding.

  • azureaistudio_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • chunking_settings object
    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is azureaistudio.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Azure AI Studio model deployment. This key can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • endpoint_type string Required

      The type of endpoint that is available for deployment through Azure AI Studio: token or realtime. The token endpoint type is for "pay as you go" endpoints that are billed per token. The realtime endpoint type is for "real-time" endpoints that are billed per hour of usage.

      External documentation
    • target string Required

      The target URL of your Azure AI Studio model deployment. This can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

    • provider string Required

      The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include:

      • cohere - available for text_embedding and completion task types
      • databricks - available for completion task type only
      • meta - available for completion task type only
      • microsoft_phi - available for completion task type only
      • mistral - available for completion task type only
      • openai - available for text_embedding and completion task types
    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object
  • task_settings object
    Hide task_settings attributes Show task_settings attributes object
    • do_sample number

      For a completion task, instruct the inference process to perform sampling. It has no effect unless temperature or top_p is specified.

    • max_new_tokens number

      For a completion task, provide a hint for the maximum number of output tokens to be generated.

    • temperature number

      For a completion task, control the apparent creativity of generated completions with a sampling temperature. It must be a number in the range of 0.0 to 2.0. It should not be used if top_p is specified.

    • top_p number

      For a completion task, make the model consider the results of the tokens with nucleus sampling probability. It is an alternative value to temperature and must be a number in the range of 0.0 to 2.0. It should not be used if temperature is specified.

    • user string

      For a text_embedding task, specify the user issuing the request. This information can be used for abuse detection.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{azureaistudio_inference_id} 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{azureaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"azureaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"Azure-AI-Studio-API-key\",\n        \"target\": \"Target-Uri\",\n        \"provider\": \"openai\",\n        \"endpoint_type\": \"token\"\n    }\n}"'
Request examples
Run `PUT _inference/text_embedding/azure_ai_studio_embeddings` to create an inference endpoint that performs a text_embedding task. Note that you do not specify a model here, as it is defined already in the Azure AI Studio deployment. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-Uri",
        "provider": "openai",
        "endpoint_type": "token"
    }
}
Run `PUT _inference/completion/azure_ai_studio_completion` to create an inference endpoint that performs a completion task. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-URI",
        "provider": "databricks",
        "endpoint_type": "realtime"
    }
}

Create an Google AI Studio inference endpoint Added in 8.15.0

PUT /_inference/{task_type}/{googleaistudio_inference_id}
Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the googleaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are completion or text_embedding.

  • googleaistudio_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • chunking_settings object
    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is googleaistudio.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Google Gemini account.

    • model_id string Required

      The name of the model to use for the inference task. Refer to the Google documentation for the list of supported models.

      External documentation
    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{googleaistudio_inference_id} 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{googleaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"googleaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"api-key\",\n        \"model_id\": \"model-id\"\n    }\n}"'
Request example
Run `PUT _inference/completion/google_ai_studio_completion` to create an inference endpoint to perform a `completion` task type. 
{
    "service": "googleaistudio",
    "service_settings": {
        "api_key": "api-key",
        "model_id": "model-id"
    }
}

Create a Watsonx inference endpoint Added in 8.16.0

PUT /_inference/{task_type}/{watsonx_inference_id}
Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the watsonxai service. You need an IBM Cloud Databases for Elasticsearch deployment to use the watsonxai inference service. You can provision one through the IBM catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The task type. The only valid task type for the model to perform is text_embedding.

    Value is text_embedding.

  • watsonx_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • service string Required

    Value is watsonxai.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Watsonx account. You can find your Watsonx API keys or you can create a new one on the API keys page.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • api_version string Required

      A version parameter that takes a version date in the format of YYYY-MM-DD. For the active version data parameters, refer to the Wastonx documentation.

      External documentation
    • model_id string Required

      The name of the model to use for the inference task. Refer to the IBM Embedding Models section in the Watsonx documentation for the list of available text embedding models.

      External documentation
    • project_id string Required

      The identifier of the IBM Cloud project to use for the inference task.

    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object
    • url string Required

      The URL of the inference endpoint that you created on Watsonx.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{watsonx_inference_id} 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{watsonx_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"service\": \"watsonxai\",\n  \"service_settings\": {\n      \"api_key\": \"Watsonx-API-Key\", \n      \"url\": \"Wastonx-URL\", \n      \"model_id\": \"ibm/slate-30m-english-rtrvr\",\n      \"project_id\": \"IBM-Cloud-ID\", \n      \"api_version\": \"2024-03-14\"\n  }\n}"'
Request example
Run `PUT _inference/text_embedding/watsonx-embeddings` to create an Watonsx inference endpoint that performs a text embedding task. 
{
  "service": "watsonxai",
  "service_settings": {
      "api_key": "Watsonx-API-Key", 
      "url": "Wastonx-URL", 
      "model_id": "ibm/slate-30m-english-rtrvr",
      "project_id": "IBM-Cloud-ID", 
      "api_version": "2024-03-14"
  }
}

Update an inference endpoint Added in 8.17.0

PUT /_inference/{task_type}/{inference_id}/_update
Api key auth Basic auth Bearer auth

Modify task_settings, secrets (within service_settings), or num_allocations for an inference endpoint, depending on the specific endpoint service and task_type.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Path parameters

  • task_type string Required

    The type of inference task that the model performs.

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body Required

  • chunking_settings object
    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    The service type

  • service_settings object Required
  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{inference_id}/_update 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{inference_id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"chunking_settings":{"max_chunk_size":42.0,"overlap":42.0,"sentence_overlap":42.0,"strategy":"string"},"service":"string","service_settings":{},"task_settings":{}}'

Get cluster info

GET /
Api key auth Basic auth Bearer auth

Get basic build, version, and cluster information.

Responses

GET / 
curl \
 --request GET 'http://api.example.com/' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /`s. 
{
  "name": "instance-0000000000",
  "cluster_name": "my_test_cluster",
  "cluster_uuid": "5QaxoN0pRZuOmWSxstBBwQ",
  "version": {
    "build_date": "2024-02-01T13:07:13.727175297Z",
    "minimum_wire_compatibility_version": "7.17.0",
    "build_hash": "6185ba65d27469afabc9bc951cded6c17c21e3f3",
    "number": "8.12.1",
    "lucene_version": "9.9.2",
    "minimum_index_compatibility_version": "7.0.0",
    "build_flavor": "default",
    "build_snapshot": false,
    "build_type": "docker"
  },
  "tagline": "You Know, for Search"
}

Get GeoIP database configurations Added in 8.15.0

GET /_ingest/geoip/database/{id}
Api key auth Basic auth Bearer auth

Get information about one or more IP geolocation database configurations.

Path parameters

  • id string | array[string] Required

    A comma-separated list of database configuration IDs to retrieve. Wildcard (*) expressions are supported. To get all database configurations, omit this parameter or use *.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • databases array[object] Required
      Hide databases attributes Show databases attributes object
      • id string Required
      • version number Required
      • modified_date_millis number

        Time unit for milliseconds

      • database object

        The configuration necessary to identify which IP geolocation provider to use to download a database, as well as any provider-specific configuration necessary for such downloading. At present, the only supported providers are maxmind and ipinfo, and the maxmind provider requires that an account_id (string) is configured. A provider (either maxmind or ipinfo) must be specified. The web and local providers can be returned as read only configurations.

        Hide database attributes Show database attributes object
GET /_ingest/geoip/database/{id} 
curl \
 --request GET 'http://api.example.com/_ingest/geoip/database/{id}' \
 --header "Authorization: $API_KEY"

Create or update a GeoIP database configuration Added in 8.15.0

PUT /_ingest/geoip/database/{id}
Api key auth Basic auth Bearer auth

Refer to the create or update IP geolocation database configuration API.

Path parameters

  • id string Required

    ID of the database configuration to create or update.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_ingest/geoip/database/{id} 
curl \
 --request PUT 'http://api.example.com/_ingest/geoip/database/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"name":"string","maxmind":{"account_id":"string"}}'

Delete pipelines Added in 5.0.0

DELETE /_ingest/pipeline/{id}
Api key auth Basic auth Bearer auth

Delete one or more ingest pipelines.

External documentation

Path parameters

  • id string Required

    Pipeline ID or wildcard expression of pipeline IDs used to limit the request. To delete all ingest pipelines in a cluster, use a value of *.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_ingest/pipeline/{id} 
curl \
 --request DELETE 'http://api.example.com/_ingest/pipeline/{id}' \
 --header "Authorization: $API_KEY"

Licensing

Licensing APIs enable you to manage your licenses.

For more information about the different types of licenses, refer to Elastic subscriptions

Update the license

PUT /_license
Api key auth Basic auth Bearer auth

You can update your license at runtime without shutting down your nodes. License updates take effect immediately. If the license you are installing does not support all of the features that were available with your previous license, however, you are notified in the response. You must then re-submit the API request with the acknowledge parameter set to true.

NOTE: If Elasticsearch security features are enabled and you are installing a gold or higher license, you must enable TLS on the transport networking layer before you install the license. If the operator privileges feature is enabled, only operator users can use this API.

Query parameters

  • acknowledge boolean

    Specifies whether you acknowledge the license changes.

  • master_timeout string

    The period to wait for a connection to the master node.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body

Responses

PUT /_license 
curl \
 --request PUT 'http://api.example.com/_license' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"licenses\": [\n    {\n      \"uid\":\"893361dc-9749-4997-93cb-802e3d7fa4xx\",\n      \"type\":\"basic\",\n      \"issue_date_in_millis\":1411948800000,\n      \"expiry_date_in_millis\":1914278399999,\n      \"max_nodes\":1,\n      \"issued_to\":\"issuedTo\",\n      \"issuer\":\"issuer\",\n      \"signature\":\"xx\"\n    }\n    ]\n}"'
Request example
Run `PUT _license` to update to a basic license. NOTE: These values are invalid; you must substitute the appropriate contents from your license file. 
{
  "licenses": [
    {
      "uid":"893361dc-9749-4997-93cb-802e3d7fa4xx",
      "type":"basic",
      "issue_date_in_millis":1411948800000,
      "expiry_date_in_millis":1914278399999,
      "max_nodes":1,
      "issued_to":"issuedTo",
      "issuer":"issuer",
      "signature":"xx"
    }
    ]
}
Response examples (200)
If you update to a basic license and you previously had a license with more features, you receive this type of response. You must re-submit the API request and set the `acknowledge` parameter to `true`. 
{
  "acknowledged": false,
  "license_status": "valid",
  "acknowledge": {
    "message": "\"\"\"This license update requires acknowledgement. To acknowledge the license, please read the following messages and update the license again, this time with the \"acknowledge=true\" parameter:\"\"\"",
    "watcher": [
      "Watcher will be disabled"
    ],
    "logstash": [
      "Logstash will no longer poll for centrally-managed pipelines"
    ],
    "security": [
      "The following X-Pack security functionality will be disabled ..."
    ]
  }
}

Update the license

POST /_license
Api key auth Basic auth Bearer auth

You can update your license at runtime without shutting down your nodes. License updates take effect immediately. If the license you are installing does not support all of the features that were available with your previous license, however, you are notified in the response. You must then re-submit the API request with the acknowledge parameter set to true.

NOTE: If Elasticsearch security features are enabled and you are installing a gold or higher license, you must enable TLS on the transport networking layer before you install the license. If the operator privileges feature is enabled, only operator users can use this API.

Query parameters

  • acknowledge boolean

    Specifies whether you acknowledge the license changes.

  • master_timeout string

    The period to wait for a connection to the master node.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body

Responses

POST /_license 
curl \
 --request POST 'http://api.example.com/_license' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"licenses\": [\n    {\n      \"uid\":\"893361dc-9749-4997-93cb-802e3d7fa4xx\",\n      \"type\":\"basic\",\n      \"issue_date_in_millis\":1411948800000,\n      \"expiry_date_in_millis\":1914278399999,\n      \"max_nodes\":1,\n      \"issued_to\":\"issuedTo\",\n      \"issuer\":\"issuer\",\n      \"signature\":\"xx\"\n    }\n    ]\n}"'
Request example
Run `PUT _license` to update to a basic license. NOTE: These values are invalid; you must substitute the appropriate contents from your license file. 
{
  "licenses": [
    {
      "uid":"893361dc-9749-4997-93cb-802e3d7fa4xx",
      "type":"basic",
      "issue_date_in_millis":1411948800000,
      "expiry_date_in_millis":1914278399999,
      "max_nodes":1,
      "issued_to":"issuedTo",
      "issuer":"issuer",
      "signature":"xx"
    }
    ]
}
Response examples (200)
If you update to a basic license and you previously had a license with more features, you receive this type of response. You must re-submit the API request and set the `acknowledge` parameter to `true`. 
{
  "acknowledged": false,
  "license_status": "valid",
  "acknowledge": {
    "message": "\"\"\"This license update requires acknowledgement. To acknowledge the license, please read the following messages and update the license again, this time with the \"acknowledge=true\" parameter:\"\"\"",
    "watcher": [
      "Watcher will be disabled"
    ],
    "logstash": [
      "Logstash will no longer poll for centrally-managed pipelines"
    ],
    "security": [
      "The following X-Pack security functionality will be disabled ..."
    ]
  }
}

Get Logstash pipelines Added in 7.12.0

GET /_logstash/pipeline/{id}
Api key auth Basic auth Bearer auth

Get pipelines that are used for Logstash Central Management.

External documentation

Path parameters

  • id string | array[string] Required

    A comma-separated list of pipeline identifiers.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string Required

        A description of the pipeline. This description is not used by Elasticsearch or Logstash.

      • last_modified string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
      • pipeline string Required

        The configuration for the pipeline.

        External documentation
      • pipeline_metadata object Required
        Hide pipeline_metadata attributes Show pipeline_metadata attributes object
      • pipeline_settings object Required
        Hide pipeline_settings attributes Show pipeline_settings attributes object
        • pipeline.workers number Required

          The number of workers that will, in parallel, execute the filter and output stages of the pipeline.

        • pipeline.batch.size number Required

          The maximum number of events an individual worker thread will collect from inputs before attempting to execute its filters and outputs.

        • pipeline.batch.delay number Required

          When creating pipeline event batches, how long in milliseconds to wait for each event before dispatching an undersized batch to pipeline workers.

        • queue.type string Required

          The internal queuing model to use for event buffering.

        • queue.max_bytes string Required

          The total capacity of the queue (queue.type: persisted) in number of bytes.

        • queue.checkpoint.writes number Required

          The maximum number of written events before forcing a checkpoint when persistent queues are enabled (queue.type: persisted).

      • username string Required

        The user who last updated the pipeline.
GET /_logstash/pipeline/{id} 
curl \
 --request GET 'http://api.example.com/_logstash/pipeline/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _logstash/pipeline/my_pipeline`. 
{
  "my_pipeline": {
    "description": "Sample pipeline for illustration purposes",
    "last_modified": "2021-01-02T02:50:51.250Z",
    "pipeline_metadata": {
      "type": "logstash_pipeline",
      "version": "1"
    },
    "username": "elastic",
    "pipeline": "input {}\\n filter { grok {} }\\n output {}",
    "pipeline_settings": {
      "pipeline.workers": 1,
      "pipeline.batch.size": 125,
      "pipeline.batch.delay": 50,
      "queue.type": "memory",
      "queue.max_bytes": "1gb",
      "queue.checkpoint.writes": 1024
    }
  }
}

Get machine learning memory usage info Added in 8.2.0

GET /_ml/memory/_stats
Api key auth Basic auth Bearer auth

Get information about how machine learning jobs and trained models are using memory, on each node, both within the JVM heap, and natively, outside of the JVM.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /_ml/memory/_stats 
curl \
 --request GET 'http://api.example.com/_ml/memory/_stats' \
 --header "Authorization: $API_KEY"

Get datafeeds configuration info Added in 5.5.0

GET /_ml/datafeeds/{datafeed_id}
Api key auth Basic auth Bearer auth

You can get information for multiple datafeeds in a single API request by using a comma-separated list of datafeeds or a wildcard expression. You can get information for all datafeeds by using _all, by specifying * as the <feed_id>, or by omitting the <feed_id>. This API returns a maximum of 10,000 datafeeds.

Path parameters

  • datafeed_id string | array[string] Required

    Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no datafeeds that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • exclude_generated boolean

    Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • datafeeds array[object] Required
      Hide datafeeds attributes Show datafeeds attributes object
      • aggregations object
      • authorization object
        Hide authorization attributes Show authorization attributes object
        • api_key object
          Hide api_key attributes Show api_key attributes object
          • id string Required

            The identifier for the API key.

          • name string Required

            The name of the API key.

        • roles array[string]

          If a user ID was used for the most recent update to the datafeed, its roles at the time of the update are listed in the response.

        • service_account string

          If a service account was used for the most recent update to the datafeed, the account name is listed in the response.

      • chunking_config object
        Hide chunking_config attributes Show chunking_config attributes object
        • mode string Required

          Values are auto, manual, or off.

        • time_span string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • datafeed_id string Required
      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indices array[string] Required
      • indexes array[string]
      • job_id string Required
      • max_empty_searches number
      • query_delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • script_fields object
        Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
            Hide script attributes Show script attributes object
            • source
            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • ignore_failure boolean
      • scroll_size number
      • delayed_data_check_config object Required
        Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
        • check_window string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • enabled boolean Required

          Specifies whether the datafeed periodically checks for delayed data.

      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
          • fetch_fields array[object]

            For type lookup

          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source
            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • indices_options object
        Hide indices_options attributes Show indices_options attributes object
        • allow_no_indices boolean

          If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

        • expand_wildcards string | array[string]
        • ignore_unavailable boolean

          If true, missing or closed indices are not included in the response.

        • ignore_throttled boolean

          If true, concrete, expanded or aliased indices are ignored when frozen.

      • query object Required

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {"boost": 1}}.

        Query DSL
GET /_ml/datafeeds/{datafeed_id} 
curl \
 --request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}' \
 --header "Authorization: $API_KEY"

Delete an anomaly detection job Added in 5.4.0

DELETE /_ml/anomaly_detectors/{job_id}
Api key auth Basic auth Bearer auth

All job configuration, model state and results are deleted. It is not currently possible to delete multiple jobs using wildcards or a comma separated list. If you delete a job that has a datafeed, the request first tries to delete the datafeed. This behavior is equivalent to calling the delete datafeed API with the same timeout and force parameters as the delete job request.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • force boolean

    Use to forcefully delete an opened job; this method is quicker than closing and deleting the job.

  • delete_user_annotations boolean

    Specifies whether annotations that have been added by the user should be deleted along with any auto-generated annotations when the job is reset.

  • wait_for_completion boolean

    Specifies whether the request should return immediately or wait until the job deletion completes.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_ml/anomaly_detectors/{job_id} 
curl \
 --request DELETE 'http://api.example.com/_ml/anomaly_detectors/{job_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when deleting an anomaly detection job. 
{
  "acknowledged": true
}
A successful response when deleting an anomaly detection job asynchronously. When the `wait_for_completion` query parameter is set to `false`, the response contains an identifier for the job deletion task. 
{
  "task": "oTUltX4IQMOUUVeiohTt8A:39"
}

Get anomaly detection job results for buckets Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/results/buckets
Api key auth Basic auth Bearer auth

The API presents a chronological view of the records, grouped by bucket.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • anomaly_score number

    Returns buckets with anomaly scores greater or equal than this value.

  • desc boolean

    If true, the buckets are sorted in descending order.

  • end string | number

    Returns buckets with timestamps earlier than this time. -1 means it is unset and results are not limited to specific timestamps.

  • exclude_interim boolean

    If true, the output excludes interim results.

  • expand boolean

    If true, the output includes anomaly records.

  • from number

    Skips the specified number of buckets.

  • size number

    Specifies the maximum number of buckets to obtain.

  • sort string

    Specifies the sort field for the requested buckets.

  • start string | number

    Returns buckets with timestamps after this time. -1 means it is unset and results are not limited to specific timestamps.

application/json

Body

  • anomaly_score number

    Refer to the description for the anomaly_score query parameter.

  • desc boolean

    Refer to the description for the desc query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number
  • exclude_interim boolean

    Refer to the description for the exclude_interim query parameter.

  • expand boolean

    Refer to the description for the expand query parameter.

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

  • sort string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • buckets array[object] Required
      Hide buckets attributes Show buckets attributes object
      • anomaly_score number Required

        The maximum anomaly score, between 0-100, for any of the bucket influencers. This is an overall, rate-limited score for the job. All the anomaly records in the bucket contribute to this score. This value might be updated as new data is analyzed.

      • bucket_influencers array[object] Required
        Hide bucket_influencers attributes Show bucket_influencers attributes object
        • anomaly_score number Required

          A normalized score between 0-100, which is calculated for each bucket influencer. This score might be updated as newer data is analyzed.

        • bucket_span number

          Time unit for seconds

        • influencer_field_name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • initial_anomaly_score number Required

          The score between 0-100 for each bucket influencer. This score is the initial value that was calculated at the time the bucket was processed.

        • is_interim boolean Required

          If true, this is an interim result. In other words, the results are calculated based on partial input data.

        • job_id string Required
        • probability number Required

          The probability that the bucket has this behavior, in the range 0 to 1. This value can be held to a high precision of over 300 decimal places, so the anomaly_score is provided as a human-readable and friendly interpretation of this.

        • raw_anomaly_score number Required

          Internal.

        • result_type string Required

          Internal. This value is always set to bucket_influencer.

        • timestamp number

          Time unit for milliseconds

        • timestamp_string string
      • bucket_span number

        Time unit for seconds

      • event_count number Required

        The number of input data records processed in this bucket.

      • initial_anomaly_score number Required

        The maximum anomaly score for any of the bucket influencers. This is the initial value that was calculated at the time the bucket was processed.

      • is_interim boolean Required

        If true, this is an interim result. In other words, the results are calculated based on partial input data.

      • job_id string Required
      • processing_time_ms number

        Time unit for milliseconds

      • result_type string Required

        Internal. This value is always set to bucket.

      • timestamp number

        Time unit for milliseconds

      • timestamp_string string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
    • count number Required
POST /_ml/anomaly_detectors/{job_id}/results/buckets 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/buckets' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"anomaly_score":42.0,"desc":true,"":"string","exclude_interim":true,"expand":true,"page":{"from":42.0,"size":42.0},"sort":"string"}'

Get datafeeds configuration info Added in 5.5.0

GET /_ml/datafeeds
Api key auth Basic auth Bearer auth

You can get information for multiple datafeeds in a single API request by using a comma-separated list of datafeeds or a wildcard expression. You can get information for all datafeeds by using _all, by specifying * as the <feed_id>, or by omitting the <feed_id>. This API returns a maximum of 10,000 datafeeds.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no datafeeds that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • exclude_generated boolean

    Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • datafeeds array[object] Required
      Hide datafeeds attributes Show datafeeds attributes object
      • aggregations object
      • authorization object
        Hide authorization attributes Show authorization attributes object
        • api_key object
          Hide api_key attributes Show api_key attributes object
          • id string Required

            The identifier for the API key.

          • name string Required

            The name of the API key.

        • roles array[string]

          If a user ID was used for the most recent update to the datafeed, its roles at the time of the update are listed in the response.

        • service_account string

          If a service account was used for the most recent update to the datafeed, the account name is listed in the response.

      • chunking_config object
        Hide chunking_config attributes Show chunking_config attributes object
        • mode string Required

          Values are auto, manual, or off.

        • time_span string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • datafeed_id string Required
      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indices array[string] Required
      • indexes array[string]
      • job_id string Required
      • max_empty_searches number
      • query_delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • script_fields object
        Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
            Hide script attributes Show script attributes object
            • source
            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • ignore_failure boolean
      • scroll_size number
      • delayed_data_check_config object Required
        Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
        • check_window string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • enabled boolean Required

          Specifies whether the datafeed periodically checks for delayed data.

      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
          • fetch_fields array[object]

            For type lookup

          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source
            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • indices_options object
        Hide indices_options attributes Show indices_options attributes object
        • allow_no_indices boolean

          If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

        • expand_wildcards string | array[string]
        • ignore_unavailable boolean

          If true, missing or closed indices are not included in the response.

        • ignore_throttled boolean

          If true, concrete, expanded or aliased indices are ignored when frozen.

      • query object Required

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {"boost": 1}}.

        Query DSL
GET /_ml/datafeeds 
curl \
 --request GET 'http://api.example.com/_ml/datafeeds' \
 --header "Authorization: $API_KEY"

Get anomaly detection jobs usage info Added in 5.5.0

GET /_ml/anomaly_detectors/{job_id}/_stats
Api key auth Basic auth Bearer auth

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

Responses

GET /_ml/anomaly_detectors/{job_id}/_stats 
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_stats' \
 --header "Authorization: $API_KEY"

Get anomaly records for an anomaly detection job Added in 5.4.0

GET /_ml/anomaly_detectors/{job_id}/results/records
Api key auth Basic auth Bearer auth

Records contain the detailed analytical results. They describe the anomalous activity that has been identified in the input data based on the detector configuration. There can be many anomaly records depending on the characteristics and size of the input data. In practice, there are often too many to be able to manually process them. The machine learning features therefore perform a sophisticated aggregation of the anomaly records into buckets. The number of record results depends on the number of anomalies found in each bucket, which relates to the number of time series being modeled and the number of detectors.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • desc boolean

    If true, the results are sorted in descending order.

  • end string | number

    Returns records with timestamps earlier than this time. The default value means results are not limited to specific timestamps.

  • exclude_interim boolean

    If true, the output excludes interim results.

  • from number

    Skips the specified number of records.

  • record_score number

    Returns records with anomaly scores greater or equal than this value.

  • size number

    Specifies the maximum number of records to obtain.

  • sort string

    Specifies the sort field for the requested records.

  • start string | number

    Returns records with timestamps after this time. The default value means results are not limited to specific timestamps.

application/json

Body

  • desc boolean

    Refer to the description for the desc query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number
  • exclude_interim boolean

    Refer to the description for the exclude_interim query parameter.

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

  • record_score number

    Refer to the description for the record_score query parameter.

  • sort string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • records array[object] Required
      Hide records attributes Show records attributes object
      • actual array[number]

        The actual value for the bucket.

      • anomaly_score_explanation object
        Hide anomaly_score_explanation attributes Show anomaly_score_explanation attributes object
      • bucket_span number

        Time unit for seconds

      • by_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

      • by_field_value string

        The value of by_field_name.

      • causes array[object]

        For population analysis, an over field must be specified in the detector. This property contains an array of anomaly records that are the causes for the anomaly that has been identified for the over field. This sub-resource contains the most anomalous records for the over_field_name. For scalability reasons, a maximum of the 10 most significant causes of the anomaly are returned. As part of the core analytical modeling, these low-level anomaly records are aggregated for their parent over field record. The causes resource contains similar elements to the record resource, namely actual, typical, geo_results.actual_point, geo_results.typical_point, *_field_name and *_field_value. Probability and scores are not applicable to causes.

        Hide causes attributes Show causes attributes object
      • detector_index number Required

        A unique identifier for the detector.

      • field_name string

        Certain functions require a field to operate on, for example, sum(). For those functions, this value is the name of the field to be analyzed.

      • function string

        The function in which the anomaly occurs, as specified in the detector configuration. For example, max.

      • function_description string

        The description of the function in which the anomaly occurs, as specified in the detector configuration.

      • geo_results object
        Hide geo_results attributes Show geo_results attributes object
        • actual_point string

          The actual value for the bucket formatted as a geo_point.

        • typical_point string

          The typical value for the bucket formatted as a geo_point.

      • influencers array[object]

        If influencers were specified in the detector configuration, this array contains influencers that contributed to or were to blame for an anomaly.

        Hide influencers attributes Show influencers attributes object
      • initial_record_score number Required

        A normalized score between 0-100, which is based on the probability of the anomalousness of this record. This is the initial value that was calculated at the time the bucket was processed.

      • is_interim boolean Required

        If true, this is an interim result. In other words, the results are calculated based on partial input data.

      • job_id string Required

        Identifier for the anomaly detection job.

      • over_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

      • over_field_value string

        The value of over_field_name.

      • partition_field_name string

        The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

      • partition_field_value string

        The value of partition_field_name.

      • probability number Required

        The probability of the individual anomaly occurring, in the range 0 to 1. For example, 0.0000772031. This value can be held to a high precision of over 300 decimal places, so the record_score is provided as a human-readable and friendly interpretation of this.

      • record_score number Required

        A normalized score between 0-100, which is based on the probability of the anomalousness of this record. Unlike initial_record_score, this value will be updated by a re-normalization process as new data is analyzed.

      • result_type string Required

        Internal. This is always set to record.

      • timestamp number

        Time unit for milliseconds

      • typical array[number]

        The typical value for the bucket, according to analytical modeling.
GET /_ml/anomaly_detectors/{job_id}/results/records 
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/records' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"desc":true,"":"string","exclude_interim":true,"page":{"from":42.0,"size":42.0},"record_score":42.0,"sort":"string"}'

Get anomaly records for an anomaly detection job Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/results/records
Api key auth Basic auth Bearer auth

Records contain the detailed analytical results. They describe the anomalous activity that has been identified in the input data based on the detector configuration. There can be many anomaly records depending on the characteristics and size of the input data. In practice, there are often too many to be able to manually process them. The machine learning features therefore perform a sophisticated aggregation of the anomaly records into buckets. The number of record results depends on the number of anomalies found in each bucket, which relates to the number of time series being modeled and the number of detectors.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • desc boolean

    If true, the results are sorted in descending order.

  • end string | number

    Returns records with timestamps earlier than this time. The default value means results are not limited to specific timestamps.

  • exclude_interim boolean

    If true, the output excludes interim results.

  • from number

    Skips the specified number of records.

  • record_score number

    Returns records with anomaly scores greater or equal than this value.

  • size number

    Specifies the maximum number of records to obtain.

  • sort string

    Specifies the sort field for the requested records.

  • start string | number

    Returns records with timestamps after this time. The default value means results are not limited to specific timestamps.

application/json

Body

  • desc boolean

    Refer to the description for the desc query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number
  • exclude_interim boolean

    Refer to the description for the exclude_interim query parameter.

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

  • record_score number

    Refer to the description for the record_score query parameter.

  • sort string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
    DateTime string UnitMillis number

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • records array[object] Required
      Hide records attributes Show records attributes object
      • actual array[number]

        The actual value for the bucket.

      • anomaly_score_explanation object
        Hide anomaly_score_explanation attributes Show anomaly_score_explanation attributes object
      • bucket_span number

        Time unit for seconds

      • by_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

      • by_field_value string

        The value of by_field_name.

      • causes array[object]

        For population analysis, an over field must be specified in the detector. This property contains an array of anomaly records that are the causes for the anomaly that has been identified for the over field. This sub-resource contains the most anomalous records for the over_field_name. For scalability reasons, a maximum of the 10 most significant causes of the anomaly are returned. As part of the core analytical modeling, these low-level anomaly records are aggregated for their parent over field record. The causes resource contains similar elements to the record resource, namely actual, typical, geo_results.actual_point, geo_results.typical_point, *_field_name and *_field_value. Probability and scores are not applicable to causes.

        Hide causes attributes Show causes attributes object
      • detector_index number Required

        A unique identifier for the detector.

      • field_name string

        Certain functions require a field to operate on, for example, sum(). For those functions, this value is the name of the field to be analyzed.

      • function string

        The function in which the anomaly occurs, as specified in the detector configuration. For example, max.

      • function_description string

        The description of the function in which the anomaly occurs, as specified in the detector configuration.

      • geo_results object
        Hide geo_results attributes Show geo_results attributes object
        • actual_point string

          The actual value for the bucket formatted as a geo_point.

        • typical_point string

          The typical value for the bucket formatted as a geo_point.

      • influencers array[object]

        If influencers were specified in the detector configuration, this array contains influencers that contributed to or were to blame for an anomaly.

        Hide influencers attributes Show influencers attributes object
      • initial_record_score number Required

        A normalized score between 0-100, which is based on the probability of the anomalousness of this record. This is the initial value that was calculated at the time the bucket was processed.

      • is_interim boolean Required

        If true, this is an interim result. In other words, the results are calculated based on partial input data.

      • job_id string Required

        Identifier for the anomaly detection job.

      • over_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

      • over_field_value string

        The value of over_field_name.

      • partition_field_name string

        The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

      • partition_field_value string

        The value of partition_field_name.

      • probability number Required

        The probability of the individual anomaly occurring, in the range 0 to 1. For example, 0.0000772031. This value can be held to a high precision of over 300 decimal places, so the record_score is provided as a human-readable and friendly interpretation of this.

      • record_score number Required

        A normalized score between 0-100, which is based on the probability of the anomalousness of this record. Unlike initial_record_score, this value will be updated by a re-normalization process as new data is analyzed.

      • result_type string Required

        Internal. This is always set to record.

      • timestamp number

        Time unit for milliseconds

      • typical array[number]

        The typical value for the bucket, according to analytical modeling.
POST /_ml/anomaly_detectors/{job_id}/results/records 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/records' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"desc":true,"":"string","exclude_interim":true,"page":{"from":42.0,"size":42.0},"record_score":42.0,"sort":"string"}'

Open anomaly detection jobs Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_open
Api key auth Basic auth Bearer auth

An anomaly detection job must be opened to be ready to receive and analyze data. It can be opened and closed multiple times throughout its lifecycle. When you open a new job, it starts with an empty model. When you open an existing job, the most recent model state is automatically loaded. The job is ready to resume its analysis from where it left off, once new data is received.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • timeout string

    Controls the time to wait until a job has opened.

application/json

Body

  • timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_ml/anomaly_detectors/{job_id}/_open 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_open' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"timeout\": \"35m\"\n}"'
Request example
A request to open anomaly detection jobs. The timeout specifies to wait 35 minutes for the job to open. 
{
  "timeout": "35m"
}
Response examples (200)
A successful response when opening an anomaly detection job. 
{
  "opened": true,
  "node": "node-1"
}

Send data to an anomaly detection job for analysis Deprecated Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_data
Api key auth Basic auth Bearer auth

IMPORTANT: For each job, data can be accepted from only a single connection at a time. It is not currently possible to post data to multiple jobs using wildcards or a comma-separated list.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. The job must have a state of open to receive and process the data.

Query parameters

  • reset_end string | number

    Specifies the end of the bucket resetting range.

  • reset_start string | number

    Specifies the start of the bucket resetting range.

application/json

Body Required

object object

Responses

POST /_ml/anomaly_detectors/{job_id}/_data 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_data' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{}]'

Stop datafeeds Added in 5.4.0

POST /_ml/datafeeds/{datafeed_id}/_stop
Api key auth Basic auth Bearer auth

A datafeed that is stopped ceases to retrieve data from Elasticsearch. A datafeed can be started and stopped multiple times throughout its lifecycle.

Path parameters

  • datafeed_id string Required

    Identifier for the datafeed. You can stop multiple datafeeds in a single API request by using a comma-separated list of datafeeds or a wildcard expression. You can close all datafeeds by using _all or by specifying * as the identifier.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no datafeeds that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    If true, the datafeed is stopped forcefully.

  • timeout string

    Specifies the amount of time to wait until a datafeed stops.

application/json

Body

  • allow_no_match boolean

    Refer to the description for the allow_no_match query parameter.

  • force boolean

    Refer to the description for the force query parameter.

  • timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_ml/datafeeds/{datafeed_id}/_stop 
curl \
 --request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_stop' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"allow_no_match":true,"force":true,"timeout":"string"}'

Validate an anomaly detection job Added in 5.4.0

POST /_ml/anomaly_detectors/_validate/detector
Api key auth Basic auth Bearer auth
application/json

Body Required

  • by_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • custom_rules array[object]

    Custom rules enable you to customize the way detectors operate. For example, a rule may dictate conditions under which results should be skipped. Kibana refers to custom rules as job rules.

    Hide custom_rules attributes Show custom_rules attributes object
    • actions array[string]

      The set of actions to be triggered when the rule applies. If more than one action is specified the effects of all actions are combined.

      Values are skip_result or skip_model_update.

    • conditions array[object]

      An array of numeric conditions when the rule applies. A rule must either have a non-empty scope or at least one condition. Multiple conditions are combined together with a logical AND.

      Hide conditions attributes Show conditions attributes object
      • applies_to string Required

        Values are actual, typical, diff_from_typical, or time.

      • operator string Required

        Values are gt, gte, lt, or lte.

      • value number Required

        The value that is compared against the applies_to field using the operator.

    • scope object

      A scope of series where the rule applies. A rule must either have a non-empty scope or at least one condition. By default, the scope includes all series. Scoping is allowed for any of the fields that are also specified in by_field_name, over_field_name, or partition_field_name.

      Hide scope attribute Show scope attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
  • detector_description string

    A description of the detector.

  • detector_index number

    A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero. If you specify a value for this property, it is ignored.

  • exclude_frequent string

    Values are all, none, by, or over.

  • field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • function string

    The analysis function that is used. For example, count, rare, mean, min, max, or sum.

  • over_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • partition_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • use_null boolean

    Defines whether a new series is used as the null series when there is no value for the by or partition fields.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_ml/anomaly_detectors/_validate/detector 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/_validate/detector' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{"applies_to":"actual","operator":"gt","value":42.0}],"scope":{"additionalProperty1":{"filter_id":"string","filter_type":"include"},"additionalProperty2":{"filter_id":"string","filter_type":"include"}}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}'

Stop data frame analytics jobs Added in 7.3.0

POST /_ml/data_frame/analytics/{id}/_stop
Api key auth Basic auth Bearer auth

A data frame analytics job can be started and stopped multiple times throughout its lifecycle.

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    If true, the data frame analytics job is stopped forcefully.

  • timeout string

    Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_ml/data_frame/analytics/{id}/_stop 
curl \
 --request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_stop' \
 --header "Authorization: $API_KEY"

Update a data frame analytics job Added in 7.3.0

POST /_ml/data_frame/analytics/{id}/_update
Api key auth Basic auth Bearer auth

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

application/json

Body Required

  • description string

    A description of the job.

  • model_memory_limit string

    The approximate maximum amount of memory resources that are permitted for analytical processing. If your elasticsearch.yml file contains an xpack.ml.max_model_memory_limit setting, an error occurs when you try to create data frame analytics jobs that have model_memory_limit values greater than that setting.

  • max_num_threads number

    The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.

  • allow_lazy_start boolean

    Specifies whether this job can start when there is insufficient machine learning node capacity for it to be immediately assigned to a node.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • authorization object
      Hide authorization attributes Show authorization attributes object
      • api_key object
        Hide api_key attributes Show api_key attributes object
        • id string Required

          The identifier for the API key.

        • name string Required

          The name of the API key.

      • roles array[string]

        If a user ID was used for the most recent update to the job, its roles at the time of the update are listed in the response.

      • service_account string

        If a service account was used for the most recent update to the job, the account name is listed in the response.

    • allow_lazy_start boolean Required
    • analysis object Required
      Hide analysis attributes Show analysis attributes object
      • classification object
        Hide classification attributes Show classification attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • class_assignment_objective string
        • num_top_classes number

          Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.

      • outlier_detection object
        Hide outlier_detection attributes Show outlier_detection attributes object
        • compute_feature_influence boolean

          Specifies whether the feature influence calculation is enabled.

        • feature_influence_threshold number

          The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.

        • method string

          The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

        • n_neighbors number

          Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

        • outlier_fraction number

          The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

        • standardization_enabled boolean

          If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

      • regression object
        Hide regression attributes Show regression attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • loss_function string

          The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).

        • loss_function_parameter number

          A positive number that is used as a parameter to the loss_function.

    • analyzed_fields object
      Hide analyzed_fields attributes Show analyzed_fields attributes object
      • includes array[string]

        An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

      • excludes array[string]

        An array of strings that defines the fields that will be included in the analysis.

    • create_time number Required
    • description string
    • dest object Required
      Hide dest attributes Show dest attributes object
      • index string Required
      • results_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max_num_threads number Required
    • model_memory_limit string Required
    • source object Required
      Hide source attributes Show source attributes object
      • index string | array[string] Required
      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • _source object
        Hide _source attributes Show _source attributes object
        • includes array[string]

          An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

        • excludes array[string]

          An array of strings that defines the fields that will be included in the analysis.

      • query object

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.

        Query DSL
    • version string Required
POST /_ml/data_frame/analytics/{id}/_update 
curl \
 --request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"description":"string","model_memory_limit":"string","max_num_threads":42.0,"allow_lazy_start":true}'

Get trained models usage info Added in 7.10.0

GET /_ml/trained_models/_stats
Api key auth Basic auth Bearer auth

You can get usage information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no models that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.

  • from number

    Skips the specified number of models.

  • size number

    Specifies the maximum number of models to obtain.

Responses

GET /_ml/trained_models/_stats 
curl \
 --request GET 'http://api.example.com/_ml/trained_models/_stats' \
 --header "Authorization: $API_KEY"

Evaluate a trained model Added in 8.3.0

POST /_ml/trained_models/{model_id}/_infer
Api key auth Basic auth Bearer auth

Path parameters

  • model_id string Required

    The unique identifier of the trained model.

Query parameters

  • timeout string

    Controls the amount of time to wait for inference results.

application/json

Body Required

  • docs array[object] Required

    An array of objects to pass to the model for inference. The objects should contain a fields matching your configured trained model input. Typically, for NLP models, the field name is text_field. Currently, for NLP models, only a single value is allowed.

    Hide docs attribute Show docs attribute object
    • * object Additional properties
  • inference_config object
    Hide inference_config attributes Show inference_config attributes object
    • regression object
      Hide regression attributes Show regression attributes object
    • classification object
      Hide classification attributes Show classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • num_top_feature_importance_values number

        Specifies the maximum number of feature importance values per document.

      • prediction_field_type string

        Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • top_classes_results_field string

        Specifies the field to which the top classes are written. Defaults to top_classes.

    • text_classification object
      Hide text_classification attributes Show text_classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • classification_labels array[string]

        Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

    • zero_shot_classification object
      Hide zero_shot_classification attributes Show zero_shot_classification attributes object
      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • multi_label boolean

        Update the configured multi label option. Indicates if more than one true label exists. Defaults to the configured value.

      • labels array[string] Required

        The labels to predict.

    • fill_mask object
      Hide fill_mask attributes Show fill_mask attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

    • ner object
      Hide ner attributes Show ner attributes object
      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

    • pass_through object
      Hide pass_through attributes Show pass_through attributes object
      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

    • text_embedding object
      Hide text_embedding attributes Show text_embedding attributes object
      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

    • text_expansion object
      Hide text_expansion attributes Show text_expansion attributes object
      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

    • question_answering object
      Hide question_answering attributes Show question_answering attributes object
      • question string Required

        The question to answer given the inference context

      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object
        Hide tokenization attributes Show tokenization attributes object
        • truncate string

          Values are first, second, or none.

        • span number

          Span options to apply

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • max_answer_length number

        The maximum answer length to consider for extraction

Responses

POST /_ml/trained_models/{model_id}/_infer 
curl \
 --request POST 'http://api.example.com/_ml/trained_models/{model_id}/_infer' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"docs":[{"additionalProperty1":{},"additionalProperty2":{}}],"inference_config":{"regression":{"results_field":"string","num_top_feature_importance_values":42.0},"classification":{"num_top_classes":42.0,"num_top_feature_importance_values":42.0,"prediction_field_type":"string","results_field":"string","top_classes_results_field":"string"},"text_classification":{"num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string","classification_labels":["string"]},"zero_shot_classification":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string","multi_label":true,"labels":["string"]},"fill_mask":{"num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"ner":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"pass_through":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"text_embedding":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"text_expansion":{"tokenization":{"truncate":"first","span":42.0},"results_field":"string"},"question_answering":{"question":"string","num_top_classes":42.0,"tokenization":{"truncate":"first","span":42.0},"results_field":"string","max_answer_length":42.0}}}'

Start the feature migration Added in 7.16.0

POST /_migration/system_features
Api key auth Basic auth Bearer auth

Version upgrades sometimes require changes to how features store configuration information and data in system indices. This API starts the automatic migration process.

Some functionality might be temporarily unavailable during the migration process.

TIP: The API is designed for indirect use by the Upgrade Assistant. We strongly recommend you use the Upgrade Assistant.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_migration/system_features 
curl \
 --request POST 'http://api.example.com/_migration/system_features' \
 --header "Authorization: $API_KEY"
Response examples (200)
When you run `POST /_migration/system_features` to start the migration process, the response lists the features that will be migrated. 
{
  "accepted" : true,
  "features" : [
    {
      "feature_name" : "security"
    }
  ]
}

Monitoring

Node lifecycle

Query rules

Query rules enable you to configure per-query rules that are applied at query time to queries that match the specific rule. Query rules are organized into rulesets, collections of query rules that are matched against incoming queries. Query rules are applied using the rule query. If a query matches one or more rules in the ruleset, the query is re-written to apply the rules before searching. This allows pinning documents for only queries that match a specific term.

Learn more about the rule query

Create or update a query rule Added in 8.15.0

PUT /_query_rules/{ruleset_id}/_rule/{rule_id}
Api key auth Basic auth Bearer auth

Create or update a query rule within a query ruleset.

IMPORTANT: Due to limitations within pinned queries, you can only pin documents using ids or docs, but cannot use both in single rule. It is advised to use one or the other in query rulesets, to avoid errors. Additionally, pinned queries have a maximum limit of 100 pinned hits. If multiple matching rules pin more than 100 documents, only the first 100 documents are pinned in the order they are specified in the ruleset.

Path parameters

  • ruleset_id string Required

    The unique identifier of the query ruleset containing the rule to be created or updated.

  • rule_id string Required

    The unique identifier of the query rule within the specified ruleset to be created or updated.

application/json

Body Required

  • type string Required

    Values are pinned or exclude.

  • criteria object | array[object] Required

    The criteria that must be met for the rule to be applied. If multiple criteria are specified for a rule, all criteria must be met for the rule to be applied.

    One of:
    QueryRuleCriteria object array-2 array[object]
    Hide attributes Show attributes
    • type string Required

      Values are global, exact, exact_fuzzy, fuzzy, prefix, suffix, contains, lt, lte, gt, gte, or always.

    • metadata string

      The metadata field to match against. This metadata will be used to match against match_criteria sent in the rule. It is required for all criteria types except always.

    • values array[object]

      The values to match against the metadata field. Only one value must match for the criteria to be met. It is required for all criteria types except always.

  • actions object Required
    Hide actions attributes Show actions attributes object
    • ids array[string]

      The unique document IDs of the documents to apply the rule to. Only one of ids or docs may be specified and at least one must be specified.

    • docs array[object]

      The documents to apply the rule to. Only one of ids or docs may be specified and at least one must be specified. There is a maximum value of 100 documents in a rule. You can specify the following attributes for each document:

      • _index: The index of the document to pin.
      • _id: The unique document ID.
      Hide docs attributes Show docs attributes object
  • priority number

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_query_rules/{ruleset_id}/_rule/{rule_id} 
curl \
 --request PUT 'http://api.example.com/_query_rules/{ruleset_id}/_rule/{rule_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"match_criteria\": {\n    \"query_string\": \"puggles\"\n  }\n}"'
Request example
Run `POST _query_rules/my-ruleset/_test` to test a ruleset. Provide the match criteria that you want to test against. 
{
  "match_criteria": {
    "query_string": "puggles"
  }
}

Get rollup job information Deprecated Technical preview

GET /_rollup/job/{id}
Api key auth Basic auth Bearer auth

Get the configuration, stats, and status of rollup jobs.

NOTE: This API returns only active (both STARTED and STOPPED) jobs. If a job was created, ran for a while, then was deleted, the API does not return any details about it. For details about a historical rollup job, the rollup capabilities API may be more useful.

Path parameters

  • id string Required

    Identifier for the rollup job. If it is _all or omitted, the API returns all rollup jobs.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • config object Required
        Hide config attributes Show config attributes object
        • cron string Required
        • groups object Required
          Hide groups attributes Show groups attributes object
          • date_histogram object
            Hide date_histogram attributes Show date_histogram attributes object
            • delay string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
            • interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • calendar_interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • fixed_interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_zone string
          • histogram object
            Hide histogram attributes Show histogram attributes object
            • fields string | array[string] Required
            • interval number Required

              The interval of histogram buckets to be generated when rolling up. For example, a value of 5 creates buckets that are five units wide (0-5, 5-10, etc). Note that only one interval can be specified in the histogram group, meaning that all fields being grouped via the histogram must share the same interval.

          • terms object
            Hide terms attribute Show terms attribute object
            • fields string | array[string] Required
        • id string Required
        • index_pattern string Required
        • metrics array[object] Required
          Hide metrics attributes Show metrics attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • metrics array[string] Required

            An array of metrics to collect for the field. At least one metric must be configured.

            Values are min, max, sum, avg, or value_count.

        • page_size number Required
        • rollup_index string Required
        • timeout string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • stats object Required
        Hide stats attributes Show stats attributes object
      • status object Required
        Hide status attributes Show status attributes object
GET /_rollup/job/{id} 
curl \
 --request GET 'http://api.example.com/_rollup/job/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _rollup/job/sensor`. 
{
  "jobs": [
    {
      "config": {
        "id": "sensor",
        "index_pattern": "sensor-*",
        "rollup_index": "sensor_rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "fixed_interval": "1h",
            "delay": "7d",
            "field": "timestamp",
            "time_zone": "UTC"
          },
          "terms": {
            "fields": [
              "node"
            ]
          }
        },
        "metrics": [
          {
            "field": "temperature",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          },
          {
            "field": "voltage",
            "metrics": [
              "avg"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "stopped"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 0,
        "index_failures": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "search_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0
      }
    }
  ]
}

Search rolled-up data Deprecated Technical preview

POST /{index}/_rollup_search
Api key auth Basic auth Bearer auth

The rollup search endpoint is needed because, internally, rolled-up documents utilize a different document structure than the original data. It rewrites standard Query DSL into a format that matches the rollup documents then takes the response and rewrites it back to what a client would expect given the original query.

The request body supports a subset of features from the regular search API. The following functionality is not available:

size: Because rollups work on pre-aggregated data, no search hits can be returned and so size must be set to zero or omitted entirely. highlighter, suggestors, post_filter, profile, explain: These are similarly disallowed.

Searching both historical rollup and non-rollup data

The rollup search API has the capability to search across both "live" non-rollup data and the aggregated rollup data. This is done by simply adding the live indices to the URI. For example:

GET sensor-1,sensor_rollup/_rollup_search
{
  "size": 0,
  "aggregations": {
     "max_temperature": {
      "max": {
        "field": "temperature"
      }
    }
  }
}

The rollup search endpoint does two things when the search runs:

  • The original request is sent to the non-rollup index unaltered.
  • A rewritten version of the original request is sent to the rollup index.

When the two responses are received, the endpoint rewrites the rollup response and merges the two together. During the merging process, if there is any overlap in buckets between the two responses, the buckets from the non-rollup index are used.

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams and indices used to limit the request. This parameter has the following rules:

    • At least one data stream, index, or wildcard expression must be specified. This target can include a rollup or non-rollup index. For data streams, the stream's backing indices can only serve as non-rollup indices. Omitting the parameter or using _all are not permitted.
    • Multiple non-rollup indices may be specified.
    • Only one rollup index may be specified. If more than one are supplied, an exception occurs.
    • Wildcard expressions (*) may be used. If they match more than one rollup index, an exception occurs. However, you can use an expression to match multiple non-rollup indices or data streams.

Query parameters

  • rest_total_hits_as_int boolean

    Indicates whether hits.total should be rendered as an integer or an object in the rest search response

  • typed_keys boolean

    Specify whether aggregation and suggester names should be prefixed by their respective types in the response

application/json

Body Required

Responses

POST /{index}/_rollup_search 
curl \
 --request POST 'http://api.example.com/{index}/_rollup_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"size\": 0,\n  \"aggregations\": {\n    \"max_temperature\": {\n      \"max\": {\n        \"field\": \"temperature\"\n      }\n    }\n  }\n}"'
Request example
Search rolled up data stored in `sensor_rollup` with `GET /sensor_rollup/_rollup_search` 
{
  "size": 0,
  "aggregations": {
    "max_temperature": {
      "max": {
        "field": "temperature"
      }
    }
  }
}
Response examples (200)
An abbreviated response from `GET /sensor_rollup/_rollup_search` with a `max` aggregation on a `temperature` field. The response provides some metadata about the request (`took`, `_shards`), the search hits (which is always empty for rollup searches), and the aggregation response. 
{
  "took" : 102,
  "timed_out" : false,
  "terminated_early" : false,
  "_shards" : {} ,
  "hits" : {
    "total" : {
        "value": 0,
        "relation": "eq"
    },
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_temperature" : {
      "value" : 202.0
    }
  }
}

Delete a script or search template

DELETE /_scripts/{id}
Api key auth Basic auth Bearer auth

Deletes a stored script or search template.

Path parameters

  • id string Required

    The identifier for the stored script or search template.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_scripts/{id} 
curl \
 --request DELETE 'http://api.example.com/_scripts/{id}' \
 --header "Authorization: $API_KEY"

Run a script Technical preview

GET /_scripts/painless/_execute
Api key auth Basic auth Bearer auth

Runs a script and returns a result. Use this API to build and test scripts, such as when defining a script for a runtime field. This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.

The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.

Each context requires a script, but additional parameters depend on the context you're using for that script.

application/json

Body

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /_scripts/painless/_execute 
curl \
 --request GET 'http://api.example.com/_scripts/painless/_execute' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"script\": {\n    \"source\": \"params.count / params.total\",\n    \"params\": {\n      \"count\": 100.0,\n      \"total\": 1000.0\n    }\n  }\n}"'
Request examples
Run `POST /_scripts/painless/_execute`. The `painless_test` context is the default context. It runs scripts without additional parameters. The only variable that is available is `params`, which can be used to access user defined values. The result of the script is always converted to a string. 
{
  "script": {
    "source": "params.count / params.total",
    "params": {
      "count": 100.0,
      "total": 1000.0
    }
  }
}
Run `POST /_scripts/painless/_execute` with a `filter` context. It treats scripts as if they were run inside a script query. For testing purposes, a document must be provided so that it will be temporarily indexed in-memory and is accessible from the script. More precisely, the `_source`, stored fields, and doc values of such a document are available to the script being tested. 
{
  "script": {
    "source": "doc['field'].value.length() <= params.max_length",
    "params": {
      "max_length": 4
    }
  },
  "context": "filter",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "field": "four"
    }
  }
}
Run `POST /_scripts/painless/_execute` with a `score` context. It treats scripts as if they were run inside a `script_score` function in a `function_score` query. 
{
  "script": {
    "source": "doc['rank'].value / params.max_rank",
    "params": {
      "max_rank": 5.0
    }
  },
  "context": "score",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "rank": 4
    }
  }
}
Response examples (200)
A successful response from `POST /_scripts/painless/_execute` with a `painless_test` context. 
{
  "result": "0.1"
}
A successful response from `POST /_scripts/painless/_execute` with a `filter` context. 
{
  "result": true
}
A successful response from `POST /_scripts/painless/_execute` with a `score` context. 
{
  "result": 0.8
}

Get the async search status Added in 7.11.0

GET /_async_search/status/{id}
Api key auth Basic auth Bearer auth

Get the status of a previously submitted async search request given its identifier, without retrieving search results. If the Elasticsearch security features are enabled, the access to the status of a specific async search is restricted to:

  • The user or API key that submitted the original async search request.
  • Users that have the monitor cluster privilege or greater privileges.

Path parameters

  • id string Required

    A unique identifier for the async search.

Query parameters

  • keep_alive string

    The length of time that the async search needs to be available. Ongoing async searches and any saved search results are deleted after this period.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • is_partial boolean Required

      When the query is no longer running, this property indicates whether the search failed or was successfully completed on all shards. While the query is running, is_partial is always set to true.

    • is_running boolean Required

      Indicates whether the search is still running or has completed.


      If the search failed after some shards returned their results or the node that is coordinating the async search dies, results may be partial even though is_running is false.

    • expiration_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number

    • Time unit for milliseconds

    • start_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number

    • Time unit for milliseconds

    • completion_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number

    • Time unit for milliseconds

    • _shards object Required
      Hide _shards attributes Show _shards attributes object
    • _clusters object
      Hide _clusters attributes Show _clusters attributes object
    • completion_status number

      If the async search completed, this field shows the status code of the search. For example, 200 indicates that the async search was successfully completed. 503 indicates that the async search was completed with an error.
GET /_async_search/status/{id} 
curl \
 --request GET 'http://api.example.com/_async_search/status/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A succesful response from `GET /_async_search/status/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=`, which retrieves the status of a previously submitted async search without the results. 
{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
  "is_running" : true,
  "is_partial" : true,
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "_shards" : {
      "total" : 562,
      "successful" : 188, 
      "skipped" : 0,
      "failed" : 0
  }
}
A succesful response from `GET /_async_search/status/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=` for an async search that has completed. The status response has an additional `completion_status` field that shows the status code of the completed async search. 
{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
  "is_running" : false,
  "is_partial" : false,
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "_shards" : {
      "total" : 562,
      "successful" : 562,
      "skipped" : 0,
      "failed" : 0
  },
"completion_status" : 200 
}
A response from `GET /_async_search/status/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=` for an async search that has completed with an error. The status response has an additional `completion_status` field that shows the status code of the completed async search. 
{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
  "is_running" : false,
  "is_partial" : true,
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "_shards" : {
      "total" : 562,
      "successful" : 450,
      "skipped" : 0,
      "failed" : 112
  },
"completion_status" : 503 
}

Run a scrolling search

POST /_search/scroll/{scroll_id}
Api key auth Basic auth Bearer auth

IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).

The scroll API gets large sets of results from a single scrolling search request. To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter. The scroll parameter indicates how long Elasticsearch should retain the search context for the request. The search response returns a scroll ID in the _scroll_id response body parameter. You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request. If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.

You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.

IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.

External documentation

Path parameters

Query parameters

  • scroll string

    The period to retain the search context for scrolling.

  • scroll_id string Deprecated

    The scroll ID for scrolled search

  • rest_total_hits_as_int boolean

    If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.

application/json

Body

  • scroll string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • scroll_id string Required

Responses

POST /_search/scroll/{scroll_id} 
curl \
 --request POST 'http://api.example.com/_search/scroll/{scroll_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'
Request example
Run `GET /_search/scroll` to get the next batch of results for a scrolling search. 
{
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}

Count search results

GET /_count
Api key auth Basic auth Bearer auth

Get the number of documents matching a query.

The query can be provided either by using a simple query string as a parameter, or by defining Query DSL within the request body. The query is optional. When no query is provided, the API uses match_all to count all the documents.

The count API supports multi-target syntax. You can run a single count API search across multiple data streams and indices.

The operation is broadcast across all shards. For each shard ID group, a replica is chosen and the search is run against it. This means that replicas increase the scalability of the count.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • analyzer string

    The analyzer to use for the query string. This parameter can be used only when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed. This parameter can be used only when the q query string parameter is specified.

  • default_operator string

    The default operator for query string query: AND or OR. This parameter can be used only when the q query string parameter is specified.

    Values are and, AND, or, or OR.

  • df string

    The field to use as a default when no field prefix is given in the query string. This parameter can be used only when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded, or aliased indices are ignored when frozen.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when the q query string parameter is specified.

  • min_score number

    The minimum _score value that documents must have to be included in the result.

  • preference string

    The node or shard the operation should be performed on. By default, it is random.

  • routing string

    A custom value used to route operations to a specific shard.

  • terminate_after number

    The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.

  • q string

    The query in Lucene query string syntax. This parameter cannot be used with a request body.

application/json

Body

Responses

GET /_count 
curl \
 --request GET 'http://api.example.com/_count' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\" : {\n    \"term\" : { \"user.id\" : \"kimchy\" }\n  }\n}"'
Request example
Run `GET /my-index-000001/_count?q=user:kimchy`. Alternatively, run `GET /my-index-000001/_count` with the same query in the request body. Both requests count the number of documents in `my-index-000001` with a `user.id` of `kimchy`. 
{
  "query" : {
    "term" : { "user.id" : "kimchy" }
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_count?q=user:kimchy`. 
{
  "count": 1,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  }
}

Run multiple templated searches Added in 5.0.0

GET /_msearch/template
Api key auth Basic auth Bearer auth

Run multiple templated searches with a single request. If you are providing a text file or text input to curl, use the --data-binary flag instead of -d to preserve newlines. For example:

$ cat requests
{ "index": "my-index" }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ "index": "my-other-index" }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}

$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
External documentation

Query parameters

  • ccs_minimize_roundtrips boolean

    If true, network round-trips are minimized for cross-cluster search requests.

  • max_concurrent_searches number

    The maximum number of concurrent searches the API can run.

  • search_type string

    The type of the search operation.

    Values are query_then_fetch or dfs_query_then_fetch.

  • rest_total_hits_as_int boolean

    If true, the response returns hits.total as an integer. If false, it returns hits.total as an object.

  • typed_keys boolean

    If true, the response prefixes aggregation and suggester names with their respective types.

application/json

Body object Required

One of:
MultisearchHeader object TemplateConfig object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
GET /_msearch/template 
curl \
 --request GET 'http://api.example.com/_msearch/template' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ }\n{ \"id\": \"my-search-template\", \"params\": { \"query_string\": \"hello world\", \"from\": 0, \"size\": 10 }}\n{ }\n{ \"id\": \"my-other-search-template\", \"params\": { \"query_type\": \"match_all\" }}"'
Request example
Run `GET my-index/_msearch/template` to run multiple templated searches. 
{ }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}

Run multiple templated searches Added in 5.0.0

POST /{index}/_msearch/template
Api key auth Basic auth Bearer auth

Run multiple templated searches with a single request. If you are providing a text file or text input to curl, use the --data-binary flag instead of -d to preserve newlines. For example:

$ cat requests
{ "index": "my-index" }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ "index": "my-other-index" }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}

$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams and indices, omit this parameter or use *.

Query parameters

  • ccs_minimize_roundtrips boolean

    If true, network round-trips are minimized for cross-cluster search requests.

  • max_concurrent_searches number

    The maximum number of concurrent searches the API can run.

  • search_type string

    The type of the search operation.

    Values are query_then_fetch or dfs_query_then_fetch.

  • rest_total_hits_as_int boolean

    If true, the response returns hits.total as an integer. If false, it returns hits.total as an object.

  • typed_keys boolean

    If true, the response prefixes aggregation and suggester names with their respective types.

application/json

Body object Required

One of:
MultisearchHeader object TemplateConfig object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /{index}/_msearch/template 
curl \
 --request POST 'http://api.example.com/{index}/_msearch/template' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ }\n{ \"id\": \"my-search-template\", \"params\": { \"query_string\": \"hello world\", \"from\": 0, \"size\": 10 }}\n{ }\n{ \"id\": \"my-other-search-template\", \"params\": { \"query_type\": \"match_all\" }}"'
Request example
Run `GET my-index/_msearch/template` to run multiple templated searches. 
{ }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}

Evaluate ranked search results Added in 6.2.0

POST /_rank_eval
Api key auth Basic auth Bearer auth

Evaluate the quality of ranked search results over a set of typical search queries.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • search_type string

    Search operation type

application/json

Body Required

  • requests array[object] Required

    A set of typical search requests, together with their provided ratings.

    Hide requests attributes Show requests attributes object
    • id string Required
    • request object
      Hide request attributes Show request attributes object
    • ratings array[object] Required

      List of document ratings

      Hide ratings attributes Show ratings attributes object
      • _id string Required
      • _index string Required
      • rating number Required

        The document’s relevance with regard to this search request.

    • template_id string
    • params object

      The search template parameters.

      Hide params attribute Show params attribute object
      • * object Additional properties
  • metric object
    Hide metric attributes Show metric attributes object
    • precision object
      Hide precision attributes Show precision attributes object
      • k number

        Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.

      • relevant_rating_threshold number

        Sets the rating threshold above which documents are considered to be "relevant".

      • ignore_unlabeled boolean

        Controls how unlabeled documents in the search results are counted. If set to true, unlabeled documents are ignored and neither count as relevant or irrelevant. Set to false (the default), they are treated as irrelevant.

    • recall object
      Hide recall attributes Show recall attributes object
      • k number

        Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.

      • relevant_rating_threshold number

        Sets the rating threshold above which documents are considered to be "relevant".

    • mean_reciprocal_rank object
      Hide mean_reciprocal_rank attributes Show mean_reciprocal_rank attributes object
      • k number

        Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.

      • relevant_rating_threshold number

        Sets the rating threshold above which documents are considered to be "relevant".

    • dcg object
      Hide dcg attributes Show dcg attributes object
      • k number

        Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.

      • normalize boolean

        If set to true, this metric will calculate the Normalized DCG.

    • expected_reciprocal_rank object
      Hide expected_reciprocal_rank attributes Show expected_reciprocal_rank attributes object
      • k number

        Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.

      • maximum_relevance number Required

        The highest relevance grade used in the user-supplied relevance judgments.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • metric_score number Required

      The overall evaluation quality calculated by the defined metric

    • details object Required

      The details section contains one entry for every query in the original requests section, keyed by the search request id

      Hide details attribute Show details attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • metric_score number Required

          The metric_score in the details section shows the contribution of this query to the global quality metric score

        • unrated_docs array[object] Required

          The unrated_docs section contains an _index and _id entry for each document in the search result for this query that didn’t have a ratings value. This can be used to ask the user to supply ratings for these documents

          Hide unrated_docs attributes Show unrated_docs attributes object
        • hits array[object] Required

          The hits section shows a grouping of the search results with their supplied ratings

          Hide hits attributes Show hits attributes object
        • metric_details object Required

          The metric_details give additional information about the calculated quality metric (e.g. how many of the retrieved documents were relevant). The content varies for each metric but allows for better interpretation of the results

          Hide metric_details attribute Show metric_details attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • * object Additional properties
    • failures object Required
      Hide failures attribute Show failures attribute object
      • * object Additional properties
POST /_rank_eval 
curl \
 --request POST 'http://api.example.com/_rank_eval' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"requests":[{"id":"string","request":{"query":{},"size":42.0},"ratings":[{"_id":"string","_index":"string","rating":42.0}],"template_id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}}}],"metric":{"precision":{"k":42.0,"relevant_rating_threshold":42.0,"ignore_unlabeled":true},"recall":{"k":42.0,"relevant_rating_threshold":42.0},"mean_reciprocal_rank":{"k":42.0,"relevant_rating_threshold":42.0},"dcg":{"k":42.0,"normalize":true},"expected_reciprocal_rank":{"k":42.0,"maximum_relevance":42.0}}}'

Get the search shards

GET /_search_shards
Api key auth Basic auth Bearer auth

Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. When filtered aliases are used, the filter is returned as part of the indices section.

If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. IT can also be set to -1 to indicate that the request should never timeout.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • routing string

    A custom value used to route operations to a specific shard.

Responses

GET /_search_shards 
curl \
 --request GET 'http://api.example.com/_search_shards' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_search_shards`. 
{
  "nodes": {},
  "indices": {
      "my-index-000001": { }
  },
  "shards": [
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 0,
          "state": "STARTED",
          "allocation_id": {"id":"0TvkCyF7TAmM1wHP4a42-A"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 1,
          "state": "STARTED",
          "allocation_id": {"id":"fMju3hd1QHWmWrIgFnI4Ww"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 2,
          "state": "STARTED",
          "allocation_id": {"id":"Nwl0wbMBTHCWjEEbGYGapg"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 3,
          "state": "STARTED",
          "allocation_id": {"id":"bU_KLGJISbW0RejwnwDPKw"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 4,
          "state": "STARTED",
          "allocation_id": {"id":"DMs7_giNSwmdqVukF7UydA"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ]
    ]
  }

Get the search shards

POST /_search_shards
Api key auth Basic auth Bearer auth

Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. When filtered aliases are used, the filter is returned as part of the indices section.

If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. IT can also be set to -1 to indicate that the request should never timeout.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • routing string

    A custom value used to route operations to a specific shard.

Responses

POST /_search_shards 
curl \
 --request POST 'http://api.example.com/_search_shards' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_search_shards`. 
{
  "nodes": {},
  "indices": {
      "my-index-000001": { }
  },
  "shards": [
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 0,
          "state": "STARTED",
          "allocation_id": {"id":"0TvkCyF7TAmM1wHP4a42-A"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 1,
          "state": "STARTED",
          "allocation_id": {"id":"fMju3hd1QHWmWrIgFnI4Ww"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 2,
          "state": "STARTED",
          "allocation_id": {"id":"Nwl0wbMBTHCWjEEbGYGapg"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 3,
          "state": "STARTED",
          "allocation_id": {"id":"bU_KLGJISbW0RejwnwDPKw"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 4,
          "state": "STARTED",
          "allocation_id": {"id":"DMs7_giNSwmdqVukF7UydA"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ]
    ]
  }

Get the search shards

GET /{index}/_search_shards
Api key auth Basic auth Bearer auth

Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. When filtered aliases are used, the filter is returned as part of the indices section.

If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. IT can also be set to -1 to indicate that the request should never timeout.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • routing string

    A custom value used to route operations to a specific shard.

Responses

GET /{index}/_search_shards 
curl \
 --request GET 'http://api.example.com/{index}/_search_shards' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_search_shards`. 
{
  "nodes": {},
  "indices": {
      "my-index-000001": { }
  },
  "shards": [
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 0,
          "state": "STARTED",
          "allocation_id": {"id":"0TvkCyF7TAmM1wHP4a42-A"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 1,
          "state": "STARTED",
          "allocation_id": {"id":"fMju3hd1QHWmWrIgFnI4Ww"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 2,
          "state": "STARTED",
          "allocation_id": {"id":"Nwl0wbMBTHCWjEEbGYGapg"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 3,
          "state": "STARTED",
          "allocation_id": {"id":"bU_KLGJISbW0RejwnwDPKw"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 4,
          "state": "STARTED",
          "allocation_id": {"id":"DMs7_giNSwmdqVukF7UydA"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ]
    ]
  }

Authenticate a user Added in 5.5.0

GET /_security/_authenticate
Api key auth Basic auth Bearer auth

Authenticates a user and returns information about the authenticated user. Include the user information in a basic auth header. A successful call returns a JSON structure that shows user information such as their username, the roles that are assigned to the user, any assigned metadata, and information about the realms that authenticated and authorized the user. If the user cannot be authenticated, this API returns a 401 status code.

Responses

GET /_security/_authenticate 
curl \
 --request GET 'http://api.example.com/_security/_authenticate' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/_authenticate`. 
{
  "username": "rdeniro",
  "roles": [
    "admin"
  ],
  "full_name": null,
  "email":  null,
  "metadata": { },
  "enabled": true,
  "authentication_realm": {
    "name" : "file",
    "type" : "file"
  },
  "lookup_realm": {
    "name" : "file",
    "type" : "file"
  },
  "authentication_type": "realm"
}

Change passwords

PUT /_security/user/{username}/_password
Api key auth Basic auth Bearer auth

Change the passwords of users in the native realm and built-in users.

Path parameters

  • username string Required

    The user whose password you want to change. If you do not specify this parameter, the password is changed for the current user.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • password string
  • password_hash string

    A hash of the new password value. This must be produced using the same hashing algorithm as has been configured for password storage. For more details, see the explanation of the xpack.security.authc.password_hashing.algorithm setting.

Responses

PUT /_security/user/{username}/_password 
curl \
 --request PUT 'http://api.example.com/_security/user/{username}/_password' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"password\" : \"new-test-password\"\n}"'
Request example
Run `POST /_security/user/jacknich/_password` to update the password for the `jacknich` user. 
{
  "password" : "new-test-password"
}

Create a cross-cluster API key

POST /_security/cross_cluster/api_key
Api key auth Basic auth Bearer auth

Create an API key of the cross_cluster type for the API key based remote cluster access. A cross_cluster API key cannot be used to authenticate through the REST interface.

IMPORTANT: To authenticate this request you must use a credential that is not an API key. Even if you use an API key that has the required privilege, the API returns an error.

Cross-cluster API keys are created by the Elasticsearch API key service, which is automatically enabled.

NOTE: Unlike REST API keys, a cross-cluster API key does not capture permissions of the authenticated user. The API key’s effective permission is exactly as specified with the access property.

A successful request returns a JSON structure that contains the API key, its unique ID, and its name. If applicable, it also returns expiration information for the API key in milliseconds.

By default, API keys never expire. You can specify expiration information when you create the API keys.

Cross-cluster API keys can only be updated with the update cross-cluster API key API. Attempting to update them with the update REST API key API or the bulk update REST API keys API will result in an error.

External documentation
application/json

Body Required

  • access object Required
    Hide access attributes Show access attributes object
    • replication array[object]

      A list of indices permission entries for cross-cluster replication.

      Hide replication attributes Show replication attributes object
    • search array[object]

      A list of indices permission entries for cross-cluster search.

      Hide search attributes Show search attributes object
  • expiration string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • name string Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • api_key string Required

      Generated API key.

    • expiration number

      Time unit for milliseconds

    • id string Required
    • name string Required
    • encoded string Required

      API key credentials which is the base64-encoding of the UTF-8 representation of id and api_key joined by a colon (:).
POST /_security/cross_cluster/api_key 
curl \
 --request POST 'http://api.example.com/_security/cross_cluster/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"name\": \"my-cross-cluster-api-key\",\n  \"expiration\": \"1d\",   \n  \"access\": {\n    \"search\": [  \n      {\n        \"names\": [\"logs*\"]\n      }\n    ],\n    \"replication\": [  \n      {\n        \"names\": [\"archive*\"]\n      }\n    ]\n  },\n  \"metadata\": {\n    \"description\": \"phase one\",\n    \"environment\": {\n      \"level\": 1,\n      \"trusted\": true,\n      \"tags\": [\"dev\", \"staging\"]\n    }\n  }\n}"'
Request example
Run `POST /_security/cross_cluster/api_key` to create a cross-cluster API key. 
{
  "name": "my-cross-cluster-api-key",
  "expiration": "1d",   
  "access": {
    "search": [  
      {
        "names": ["logs*"]
      }
    ],
    "replication": [  
      {
        "names": ["archive*"]
      }
    ]
  },
  "metadata": {
    "description": "phase one",
    "environment": {
      "level": 1,
      "trusted": true,
      "tags": ["dev", "staging"]
    }
  }
}
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token`. 
{
  "created": true,
  "token": {
    "name": "Jk5J1HgBuyBK5TpDrdo4",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ"
  }
}

Create a service account token

POST /_security/service/{namespace}/{service}/credential/token/{name}
Api key auth Basic auth Bearer auth

Create a service accounts token for access without requiring basic authentication.

NOTE: Service account tokens never expire. You must actively delete them if they are no longer needed.

External documentation

Path parameters

  • namespace string Required

    The name of the namespace, which is a top-level grouping of service accounts.

  • service string Required

    The name of the service.

  • name string Required

    The name for the service account token. If omitted, a random name will be generated.

    Token names must be at least one and no more than 256 characters. They can contain alphanumeric characters (a-z, A-Z, 0-9), dashes (-), and underscores (_), but cannot begin with an underscore.

    NOTE: Token names must be unique in the context of the associated service account. They must also be globally unique with their fully qualified names, which are comprised of the service account principal and token name, such as <namespace>/<service>/<token-name>.

Query parameters

  • refresh string

    If true then refresh the affected shards to make this operation visible to search, if wait_for (the default) then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_security/service/{namespace}/{service}/credential/token/{name} 
curl \
 --request POST 'http://api.example.com/_security/service/{namespace}/{service}/credential/token/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token/token1`. The response includes the service account token, its name, and its secret value as a bearer token. 
{
  "created": true,
  "token": {
    "name": "token1",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  }
}

Get application privileges Added in 6.4.0

GET /_security/privilege/{application}/{name}
Api key auth Basic auth Bearer auth

To use this API, you must have one of the following privileges:

  • The read_security cluster privilege (or a greater privilege such as manage_security or all).
  • The "Manage Application Privileges" global privilege for the application being referenced in the request.
External documentation

Path parameters

  • application string Required

    The name of the application. Application privileges are always associated with exactly one application. If you do not specify this parameter, the API returns information about all privileges for all applications.

  • name string | array[string] Required

    The name of the privilege. If you do not specify this parameter, the API returns information about all privileges for the requested application.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
GET /_security/privilege/{application}/{name} 
curl \
 --request GET 'http://api.example.com/_security/privilege/{application}/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/privilege/myapp/read`. The response contains information about the `read` privilege for the `app01` application. 
{
  "myapp": {
    "read": {
      "application": "myapp",
      "name": "read",
      "actions": [
        "data:read/*",
        "action:login"
      ],
      "metadata": {
        "description": "Read access to myapp"
      }
    }
  }
}

Create or update roles

POST /_security/role/{name}
Api key auth Basic auth Bearer auth

The role management APIs are generally the preferred way to manage roles in the native realm, rather than using file-based role management. The create or update roles API cannot update roles that are defined in roles files. File-based role management is not available in Elastic Serverless.

External documentation

Path parameters

  • name string Required

    The name of the role that is being created or updated. On Elasticsearch Serverless, the role name must begin with a letter or digit and can only contain letters, digits and the characters '_', '-', and '.'. Each role must have a unique name, as this will serve as the identifier for that role.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • applications array[object]

    A list of application privilege entries.

    Hide applications attributes Show applications attributes object
    • application string Required

      The name of the application to which this entry applies.

    • privileges array[string] Required

      A list of strings, where each element is the name of an application privilege or action.

    • resources array[string] Required

      A list resources to which the privileges are applied.

  • cluster array[string]

    A list of cluster privileges. These privileges define the cluster-level actions for users with this role.

  • global object

    An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

    Hide global attribute Show global attribute object
    • * object Additional properties
  • indices array[object]

    A list of indices permissions entries.

    Hide indices attributes Show indices attributes object
    • field_security object
      Hide field_security attributes Show field_security attributes object

    • names string | array[string]

      A list of indices (or index name patterns) to which the permissions in this entry apply.

      One of:
      IndexName string array-2 array[string]
    • privileges array[string] Required

      The index level privileges that owners of the role have on the specified indices.

    • query string | object

      While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

      Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

      One of:
      IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
    • allow_restricted_indices boolean

      Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

  • remote_indices array[object]

    A list of remote indices permissions entries.

    NOTE: Remote indices are effective for remote clusters configured with the API key based model. They have no effect for remote clusters configured with the certificate based model.

    Hide remote_indices attributes Show remote_indices attributes object
  • remote_cluster array[object]

    A list of remote cluster permissions entries.

    Hide remote_cluster attributes Show remote_cluster attributes object
    • clusters string | array[string] Required
    • privileges array[string] Required

      The cluster level privileges that owners of the role have on the remote cluster.

      Values are monitor_enrich or monitor_stats.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • run_as array[string]

    A list of users that the owners of this role can impersonate. Note: in Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

  • description string

    Optional description of the role descriptor

  • transient_metadata object

    Indicates roles that might be incompatible with the current cluster license, specifically roles with document and field level security. When the cluster license doesn’t allow certain features for a given role, this parameter is updated dynamically to list the incompatible features. If enabled is false, the role is ignored, but is still listed in the response from the authenticate API.

    Hide transient_metadata attribute Show transient_metadata attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • role object Required
      Hide role attribute Show role attribute object
POST /_security/role/{name} 
curl \
 --request POST 'http://api.example.com/_security/role/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"description\": \"Grants full access to all management features within the cluster.\",\n  \"cluster\": [\"all\"],\n  \"indices\": [\n    {\n      \"names\": [ \"index1\", \"index2\" ],\n      \"privileges\": [\"all\"],\n      \"field_security\" : { // optional\n        \"grant\" : [ \"title\", \"body\" ]\n      },\n      \"query\": \"{\\\"match\\\": {\\\"title\\\": \\\"foo\\\"}}\" // optional\n    }\n  ],\n  \"applications\": [\n    {\n      \"application\": \"myapp\",\n      \"privileges\": [ \"admin\", \"read\" ],\n      \"resources\": [ \"*\" ]\n    }\n  ],\n  \"run_as\": [ \"other_user\" ], // optional\n  \"metadata\" : { // optional\n    \"version\" : 1\n  }\n}"'
Request examples
Run `POST /_security/role/my_admin_role` to create a role. 
{
  "description": "Grants full access to all management features within the cluster.",
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "index1", "index2" ],
      "privileges": ["all"],
      "field_security" : { // optional
        "grant" : [ "title", "body" ]
      },
      "query": "{\"match\": {\"title\": \"foo\"}}" // optional
    }
  ],
  "applications": [
    {
      "application": "myapp",
      "privileges": [ "admin", "read" ],
      "resources": [ "*" ]
    }
  ],
  "run_as": [ "other_user" ], // optional
  "metadata" : { // optional
    "version" : 1
  }
}
Run `POST /_security/role/cli_or_drivers_minimal` to configure a role that can run SQL in JDBC. 
{
  "cluster": ["cluster:monitor/main"],
  "indices": [
    {
      "names": ["test"],
      "privileges": ["read", "indices:admin/get"]
    }
  ]
}
Run `POST /_security/role/only_remote_access_role` to configure a role with remote indices and remote cluster privileges for a remote cluster. 
{
  "remote_indices": [
    {
      "clusters": ["my_remote"], 
      "names": ["logs*"], 
      "privileges": ["read", "read_cross_cluster", "view_index_metadata"] 
    }
  ],
  "remote_cluster": [
    {
      "clusters": ["my_remote"], 
      "privileges": ["monitor_stats"]  
    }
  ]
}
Response examples (200)
A successful response from `POST /_security/role/my_admin_role`. 
{
  "role": {
    "created": true 
  }
}

Create or update role mappings Added in 5.5.0

PUT /_security/role_mapping/{name}
Api key auth Basic auth Bearer auth

Role mappings define which roles are assigned to each user. Each mapping has rules that identify users and a list of roles that are granted to those users. The role mapping APIs are generally the preferred way to manage role mappings rather than using role mapping files. The create or update role mappings API cannot update role mappings that are defined in role mapping files.

NOTE: This API does not create roles. Rather, it maps users to existing roles. Roles can be created by using the create or update roles API or roles files.

Role templates

The most common use for role mappings is to create a mapping from a known value on the user to a fixed role name. For example, all users in the cn=admin,dc=example,dc=com LDAP group should be given the superuser role in Elasticsearch. The roles field is used for this purpose.

For more complex needs, it is possible to use Mustache templates to dynamically determine the names of the roles that should be granted to the user. The role_templates field is used for this purpose.

NOTE: To use role templates successfully, the relevant scripting feature must be enabled. Otherwise, all attempts to create a role mapping with role templates fail.

All of the user fields that are available in the role mapping rules are also available in the role templates. Thus it is possible to assign a user to a role that reflects their username, their groups, or the name of the realm to which they authenticated.

By default a template is evaluated to produce a single string that is the name of the role which should be assigned to the user. If the format of the template is set to "json" then the template is expected to produce a JSON string or an array of JSON strings for the role names.

External documentation

Path parameters

  • name string Required

    The distinct name that identifies the role mapping. The name is used solely as an identifier to facilitate interaction via the API; it does not affect the behavior of the mapping in any way.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • enabled boolean

    Mappings that have enabled set to false are ignored when role mapping is performed.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • roles array[string]

    A list of role names that are granted to the users that match the role mapping rules. Exactly one of roles or role_templates must be specified.

  • role_templates array[object]

    A list of Mustache templates that will be evaluated to determine the roles names that should granted to the users that match the role mapping rules. Exactly one of roles or role_templates must be specified.

    Hide role_templates attributes Show role_templates attributes object
  • rules object
    Hide rules attributes Show rules attributes object
  • run_as array[string]

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
PUT /_security/role_mapping/{name} 
curl \
 --request PUT 'http://api.example.com/_security/role_mapping/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"roles\": [ \"user\"],\n  \"enabled\": true, \n  \"rules\": {\n    \"field\" : { \"username\" : \"*\" }\n  },\n  \"metadata\" : { \n    \"version\" : 1\n  }\n}"'
Request examples
Run `POST /_security/role_mapping/mapping1` to assign the `user` role to all users. 
{
  "roles": [ "user"],
  "enabled": true, 
  "rules": {
    "field" : { "username" : "*" }
  },
  "metadata" : { 
    "version" : 1
  }
}
Run `POST /_security/role_mapping/mapping2` to assign the "user" and "admin" roles to specific users. 
{
  "roles": [ "user", "admin" ],
  "enabled": true,
  "rules": {
    "field" : { "username" : [ "esadmin01", "esadmin02" ] }
  }
}
Run `POST /_security/role_mapping/mapping3` to match users who authenticated against a specific realm. 
{
  "roles": [ "ldap-user" ],
  "enabled": true,
  "rules": {
    "field" : { "realm.name" : "ldap1" }
  }
}
Run `POST /_security/role_mapping/mapping4` to match any user where either the username is `esadmin` or the user is in the `cn=admin,dc=example,dc=com group`. This example is useful when the group names in your identity management system (such as Active Directory, or a SAML Identity Provider) do not have a one-to-one correspondence with the names of roles in Elasticsearch. The role mapping is the means by which you link a group name with a role name. 
{
  "roles": [ "superuser" ],
  "enabled": true,
  "rules": {
    "any": [
      {
        "field": {
          "username": "esadmin"
        }
      },
      {
        "field": {
          "groups": "cn=admins,dc=example,dc=com"
        }
      }
    ]
  }
}
Run `POST /_security/role_mapping/mapping5` to use an array syntax for the groups field when there are multiple groups. This pattern matches any of the groups (rather than all of the groups). 
{
  "role_templates": [
    {
      "template": { "source": "{{#tojson}}groups{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": {
    "field" : { "realm.name" : "saml1" }
  },
  "enabled": true
}
Run `POST /_security/role_mapping/mapping6` for rare cases when the names of your groups may be an exact match for the names of your Elasticsearch roles. This can be the case when your SAML Identity Provider includes its own "group mapping" feature and can be configured to release Elasticsearch role names in the user's SAML attributes. In these cases it is possible to use a template that treats the group names as role names. NOTE: This should only be done if you intend to define roles for all of the provided groups. Mapping a user to a large number of unnecessary or undefined roles is inefficient and can have a negative effect on system performance. If you only need to map a subset of the groups, you should do it by using explicit mappings. The `tojson` mustache function is used to convert the list of group names into a valid JSON array. Because the template produces a JSON array, the `format` must be set to `json`. 
{
  "role_templates": [
    {
      "template": { "source": "{{#tojson}}groups{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": {
    "field" : { "realm.name" : "saml1" }
  },
  "enabled": true
}
Run `POST /_security/role_mapping/mapping7` to match users within a particular LDAP sub-tree in a specific realm. 
{
  "roles": [ "ldap-example-user" ],
  "enabled": true,
  "rules": {
    "all": [
      { "field" : { "dn" : "*,ou=subtree,dc=example,dc=com" } },
      { "field" : { "realm.name" : "ldap1" } }
    ]
  }
}
Run `POST /_security/role_mapping/mapping8` to assign rules that are complex and include wildcard matching. For example, this mapping matches any user where all of these conditions are met: the Distinguished Name matches the pattern `*,ou=admin,dc=example,dc=com`, or the `username` is `es-admin`, or the `username` is `es-system`; the user is in the `cn=people,dc=example,dc=com` group; the user does not have a `terminated_date`. 
{
  "roles": [ "superuser" ],
  "enabled": true,
  "rules": {
    "all": [
      {
        "any": [
          {
            "field": {
              "dn": "*,ou=admin,dc=example,dc=com"
            }
          },
          {
            "field": {
              "username": [ "es-admin", "es-system" ]
            }
          }
        ]
      },
      {
        "field": {
          "groups": "cn=people,dc=example,dc=com"
        }
      },
      {
        "except": {
          "field": {
            "metadata.terminated_date": null
          }
        }
      }
    ]
  }
}
Run `POST /_security/role_mapping/mapping9` to use templated roles to automatically map every user to their own custom role. In this example every user who authenticates using the `cloud-saml` realm will be automatically mapped to two roles: the `saml_user` role and a role that is their username prefixed with `_user_`. For example, the user `nwong` would be assigned the `saml_user` and `_user_nwong` roles. 
{
  "rules": { "field": { "realm.name": "cloud-saml" } },
  "role_templates": [
    { "template": { "source" : "saml_user" } }, 
    { "template": { "source" : "_user_{{username}}" } }
  ],
  "enabled": true
}
Response examples (200)
A successful response from `POST /_security/role_mapping/mapping1`. 
{
  "role_mapping" : {
    "created" : true 
  }
}

Delete role mappings Added in 5.5.0

DELETE /_security/role_mapping/{name}
Api key auth Basic auth Bearer auth

Role mappings define which roles are assigned to each user. The role mapping APIs are generally the preferred way to manage role mappings rather than using role mapping files. The delete role mappings API cannot remove role mappings that are defined in role mapping files.

External documentation

Path parameters

  • name string Required

    The distinct name that identifies the role mapping. The name is used solely as an identifier to facilitate interaction via the API; it does not affect the behavior of the mapping in any way.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • found boolean Required

      If the mapping is successfully deleted, found is true. Otherwise, found is false.
DELETE /_security/role_mapping/{name} 
curl \
 --request DELETE 'http://api.example.com/_security/role_mapping/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_security/role_mapping/mapping1`. If the mapping is successfully deleted, `found` is `true`. 
{
  "found" : true
}

Get users

GET /_security/user/{username}
Api key auth Basic auth Bearer auth

Get information about users in the native realm and built-in users.

Path parameters

  • username string | array[string] Required

    An identifier for the user. You can specify multiple usernames as a comma-separated list. If you omit this parameter, the API retrieves information about all users.

Query parameters

  • with_profile_uid boolean

    Determines whether to retrieve the user profile UID, if it exists, for the users.

Responses

GET /_security/user/{username} 
curl \
 --request GET 'http://api.example.com/_security/user/{username}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/user/jacknich?with_profile_uid=true`. It includes the user `profile_uid` as part of the response. 
{
  "jacknich": {
    "username": "jacknich",
    "roles": [
      "admin", "other_role1"
    ],
    "full_name": "Jack Nicholson",
    "email": "jacknich@example.com",
    "metadata": { "intelligence" : 7 },
    "enabled": true,
    "profile_uid": "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
  }
}

Delete users

DELETE /_security/user/{username}
Api key auth Basic auth Bearer auth

Delete users from the native realm.

Path parameters

  • username string Required

    An identifier for the user.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • found boolean Required

      If the user is successfully deleted, the request returns {"found": true}. Otherwise, found is set to false.
DELETE /_security/user/{username} 
curl \
 --request DELETE 'http://api.example.com/_security/user/{username}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_security/user/jacknich`. 
{
  "found" : true
}

Enable a user profile Added in 8.2.0

PUT /_security/profile/{uid}/_enable
Api key auth Basic auth Bearer auth

Enable user profiles to make them visible in user profile searches.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

When you activate a user profile, it's automatically enabled and visible in user profile searches. If you later disable the user profile, you can use the enable user profile API to make the profile visible in these searches again.

Path parameters

  • uid string Required

    A unique identifier for the user profile.

Query parameters

  • refresh string

    If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', nothing is done with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_security/profile/{uid}/_enable 
curl \
 --request PUT 'http://api.example.com/_security/profile/{uid}/_enable' \
 --header "Authorization: $API_KEY"

Enroll Kibana Added in 8.0.0

GET /_security/enroll/kibana
Api key auth Basic auth Bearer auth

Enable a Kibana instance to configure itself for communication with a secured Elasticsearch cluster.

NOTE: This API is currently intended for internal use only by Kibana. Kibana uses this API internally to configure itself for communications with an Elasticsearch cluster that already has security features enabled.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • token object Required
      Hide token attributes Show token attributes object
      • name string Required

        The name of the bearer token for the elastic/kibana service account.

      • value string Required

        The value of the bearer token for the elastic/kibana service account. Use this value to authenticate the service account with Elasticsearch.

    • http_ca string Required

      The CA certificate used to sign the node certificates that Elasticsearch uses for TLS on the HTTP layer. The certificate is returned as a Base64 encoded string of the ASN.1 DER encoding of the certificate.
GET /_security/enroll/kibana 
curl \
 --request GET 'http://api.example.com/_security/enroll/kibana' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/enroll/kibana`. 
{
  "token" : {
    "name" : "enroll-process-token-1629123923000", 
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  },
  "http_ca" : "MIIJlAIBAzVoGCSqGSIb3...vsDfsA3UZBAjEPfhubpQysAICAA=", 
}

Get application privileges Added in 6.4.0

GET /_security/privilege/{application}
Api key auth Basic auth Bearer auth

To use this API, you must have one of the following privileges:

  • The read_security cluster privilege (or a greater privilege such as manage_security or all).
  • The "Manage Application Privileges" global privilege for the application being referenced in the request.
External documentation

Path parameters

  • application string Required

    The name of the application. Application privileges are always associated with exactly one application. If you do not specify this parameter, the API returns information about all privileges for all applications.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
GET /_security/privilege/{application} 
curl \
 --request GET 'http://api.example.com/_security/privilege/{application}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/privilege/myapp/read`. The response contains information about the `read` privilege for the `app01` application. 
{
  "myapp": {
    "read": {
      "application": "myapp",
      "name": "read",
      "actions": [
        "data:read/*",
        "action:login"
      ],
      "metadata": {
        "description": "Read access to myapp"
      }
    }
  }
}

Get service accounts Added in 7.13.0

GET /_security/service/{namespace}/{service}
Api key auth Basic auth Bearer auth

Get a list of service accounts that match the provided path parameters.

NOTE: Currently, only the elastic/fleet-server service account is available.

External documentation

Path parameters

  • namespace string Required

    The name of the namespace. Omit this parameter to retrieve information about all service accounts. If you omit this parameter, you must also omit the service parameter.

  • service string Required

    The service name. Omit this parameter to retrieve information about all service accounts that belong to the specified namespace.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • role_descriptor object Required
        Hide role_descriptor attributes Show role_descriptor attributes object
        • cluster array[string] Required

          A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

        • indices array[object] Required

          A list of indices permissions entries.

          Hide indices attributes Show indices attributes object
          • field_security object
            Hide field_security attributes Show field_security attributes object

          • names string | array[string]

            A list of indices (or index name patterns) to which the permissions in this entry apply.

            One of:
            IndexName string array-2 array[string]
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query string | object

            While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

            Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

            One of:
            IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_indices array[object]

          A list of indices permissions for remote clusters.

          Hide remote_indices attributes Show remote_indices attributes object
          • clusters string | array[string] Required
          • field_security object
            Hide field_security attributes Show field_security attributes object

          • names string | array[string]

            A list of indices (or index name patterns) to which the permissions in this entry apply.

            One of:
            IndexName string array-2 array[string]
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query string | object

            While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

            Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

            One of:
            IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_cluster array[object]

          A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          Hide remote_cluster attributes Show remote_cluster attributes object
          • clusters string | array[string] Required
          • privileges array[string] Required

            The cluster level privileges that owners of the role have on the remote cluster.

            Values are monitor_enrich or monitor_stats.

        • global array[object] | object

          An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

          One of:
          array-1 array[object] GlobalPrivilege object
          Hide attribute Show attribute object
        • applications array[object]

          A list of application privilege entries

          Hide applications attributes Show applications attributes object
          • application string Required

            The name of the application to which this entry applies.

          • privileges array[string] Required

            A list of strings, where each element is the name of an application privilege or action.

          • resources array[string] Required

            A list resources to which the privileges are applied.

        • metadata object
          Hide metadata attribute Show metadata attribute object
          • * object Additional properties
        • run_as array[string]

          A list of users that the API keys can impersonate.

        • description string

          An optional description of the role descriptor.

        • restriction object
          Hide restriction attribute Show restriction attribute object
          • workflows array[string] Required

            A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

        • transient_metadata object
          Hide transient_metadata attribute Show transient_metadata attribute object
          • * object Additional properties
GET /_security/service/{namespace}/{service} 
curl \
 --request GET 'http://api.example.com/_security/service/{namespace}/{service}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/service/elastic/fleet-server`. The response contains information about the `elastic/fleet-server` service account. 
{
  "elastic/fleet-server": {
    "role_descriptor": {
      "cluster": [
        "monitor",
        "manage_own_api_key",
        "read_fleet_secrets"
      ],
      "indices": [
        {
          "names": [
            "logs-*",
            "metrics-*",
            "traces-*",
            ".logs-endpoint.diagnostic.collection-*",
            ".logs-endpoint.action.responses-*",
            ".logs-endpoint.heartbeat-*"
          ],
          "privileges": [
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "profiling-*"
          ],
          "privileges": [
            "read",
            "write"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "traces-apm.sampled-*"
          ],
          "privileges": [
            "read",
            "monitor",
            "maintenance"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            ".fleet-secrets*"
          ],
          "privileges": [
            "read"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-actions*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-agents*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-artifacts*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-enrollment-api-keys*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies-leader*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-servers*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-fileds*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            "synthetics-*"
          ],
          "privileges": [
            "read",
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        }
      ],
      "applications": [
        {
          "application": "kibana-*",
          "privileges": [
            "reserved_fleet-setup"
          ],
          "resources": [
            "*"
          ]
        }
      ],
      "run_as": [],
      "metadata": {},
      "transient_metadata": {
        "enabled": true
      }
    }
  }
}

Get service accounts Added in 7.13.0

GET /_security/service
Api key auth Basic auth Bearer auth

Get a list of service accounts that match the provided path parameters.

NOTE: Currently, only the elastic/fleet-server service account is available.

External documentation

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • role_descriptor object Required
        Hide role_descriptor attributes Show role_descriptor attributes object
        • cluster array[string] Required

          A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

        • indices array[object] Required

          A list of indices permissions entries.

          Hide indices attributes Show indices attributes object
          • field_security object
            Hide field_security attributes Show field_security attributes object

          • names string | array[string]

            A list of indices (or index name patterns) to which the permissions in this entry apply.

            One of:
            IndexName string array-2 array[string]
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query string | object

            While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

            Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

            One of:
            IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_indices array[object]

          A list of indices permissions for remote clusters.

          Hide remote_indices attributes Show remote_indices attributes object
          • clusters string | array[string] Required
          • field_security object
            Hide field_security attributes Show field_security attributes object

          • names string | array[string]

            A list of indices (or index name patterns) to which the permissions in this entry apply.

            One of:
            IndexName string array-2 array[string]
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query string | object

            While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

            Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

            One of:
            IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_cluster array[object]

          A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          Hide remote_cluster attributes Show remote_cluster attributes object
          • clusters string | array[string] Required
          • privileges array[string] Required

            The cluster level privileges that owners of the role have on the remote cluster.

            Values are monitor_enrich or monitor_stats.

        • global array[object] | object

          An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

          One of:
          array-1 array[object] GlobalPrivilege object
          Hide attribute Show attribute object
        • applications array[object]

          A list of application privilege entries

          Hide applications attributes Show applications attributes object
          • application string Required

            The name of the application to which this entry applies.

          • privileges array[string] Required

            A list of strings, where each element is the name of an application privilege or action.

          • resources array[string] Required

            A list resources to which the privileges are applied.

        • metadata object
          Hide metadata attribute Show metadata attribute object
          • * object Additional properties
        • run_as array[string]

          A list of users that the API keys can impersonate.

        • description string

          An optional description of the role descriptor.

        • restriction object
          Hide restriction attribute Show restriction attribute object
          • workflows array[string] Required

            A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

        • transient_metadata object
          Hide transient_metadata attribute Show transient_metadata attribute object
          • * object Additional properties
GET /_security/service 
curl \
 --request GET 'http://api.example.com/_security/service' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/service/elastic/fleet-server`. The response contains information about the `elastic/fleet-server` service account. 
{
  "elastic/fleet-server": {
    "role_descriptor": {
      "cluster": [
        "monitor",
        "manage_own_api_key",
        "read_fleet_secrets"
      ],
      "indices": [
        {
          "names": [
            "logs-*",
            "metrics-*",
            "traces-*",
            ".logs-endpoint.diagnostic.collection-*",
            ".logs-endpoint.action.responses-*",
            ".logs-endpoint.heartbeat-*"
          ],
          "privileges": [
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "profiling-*"
          ],
          "privileges": [
            "read",
            "write"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "traces-apm.sampled-*"
          ],
          "privileges": [
            "read",
            "monitor",
            "maintenance"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            ".fleet-secrets*"
          ],
          "privileges": [
            "read"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-actions*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-agents*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-artifacts*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-enrollment-api-keys*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies-leader*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-servers*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-fileds*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            "synthetics-*"
          ],
          "privileges": [
            "read",
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        }
      ],
      "applications": [
        {
          "application": "kibana-*",
          "privileges": [
            "reserved_fleet-setup"
          ],
          "resources": [
            "*"
          ]
        }
      ],
      "run_as": [],
      "metadata": {},
      "transient_metadata": {
        "enabled": true
      }
    }
  }
}

Grant an API key Added in 7.9.0

POST /_security/api_key/grant
Api key auth Basic auth Bearer auth

Create an API key on behalf of another user. This API is similar to the create API keys API, however it creates the API key for a user that is different than the user that runs the API. The caller must have authentication credentials for the user on whose behalf the API key will be created. It is not possible to use this API to create an API key without that user's credentials. The supported user authentication credential types are:

  • username and password
  • Elasticsearch access tokens
  • JWTs

The user, for whom the authentication credentials is provided, can optionally "run as" (impersonate) another user. In this case, the API key will be created on behalf of the impersonated user.

This API is intended be used by applications that need to create and manage API keys for end users, but cannot guarantee that those users have permission to create API keys on their own behalf. The API keys are created by the Elasticsearch API key service, which is automatically enabled.

A successful grant API key API call returns a JSON structure that contains the API key, its unique id, and its name. If applicable, it also returns expiration information for the API key in milliseconds.

By default, API keys never expire. You can specify expiration information when you create the API keys.

application/json

Body Required

  • api_key object Required
    Hide api_key attributes Show api_key attributes object
    • name string Required
    • expiration string

      A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

    • role_descriptors object | array[object]

      The role descriptors for this API key. When it is not specified or is an empty array, the API key has a point in time snapshot of permissions of the specified user or access token. If you supply role descriptors, the resultant permissions are an intersection of API keys permissions and the permissions of the user or access token.

      One of:
      object-1 object array-2 array[object]
      Hide attribute Show attribute
      • * object Additional properties
        Hide * attributes Show * attributes object
        • cluster array[string]

          A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

        • indices array[object]

          A list of indices permissions entries.

          Hide indices attributes Show indices attributes object
          • field_security object
          • names
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_indices array[object]

          A list of indices permissions for remote clusters.

          Hide remote_indices attributes Show remote_indices attributes object
          • clusters string | array[string] Required
          • field_security object
          • names
          • privileges array[string] Required

            The index level privileges that owners of the role have on the specified indices.

          • query
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

        • remote_cluster array[object]

          A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          Hide remote_cluster attributes Show remote_cluster attributes object
          • clusters string | array[string] Required
          • privileges array[string] Required

            The cluster level privileges that owners of the role have on the remote cluster.

            Values are monitor_enrich or monitor_stats.

        • global array[object] | object

          An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

          One of:
          array-1 array[object] GlobalPrivilege object
        • applications array[object]

          A list of application privilege entries

          Hide applications attributes Show applications attributes object
          • application string Required

            The name of the application to which this entry applies.

          • privileges array[string] Required

            A list of strings, where each element is the name of an application privilege or action.

          • resources array[string] Required

            A list resources to which the privileges are applied.

        • metadata object
          Hide metadata attribute Show metadata attribute object
          • * object Additional properties
        • run_as array[string]

          A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

        • description string

          Optional description of the role descriptor

        • restriction object
          Hide restriction attribute Show restriction attribute object
          • workflows array[string] Required

            A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

        • transient_metadata object
          Hide transient_metadata attribute Show transient_metadata attribute object
          • * object Additional properties
    • metadata object
      Hide metadata attribute Show metadata attribute object
      • * object Additional properties
  • grant_type string Required

    Values are access_token or password.

  • access_token string

    The user's access token. If you specify the access_token grant type, this parameter is required. It is not valid with other grant types.

  • username string
  • password string
  • run_as string

Responses

POST /_security/api_key/grant 
curl \
 --request POST 'http://api.example.com/_security/api_key/grant' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"grant_type\": \"password\",\n  \"username\" : \"test_admin\",\n  \"password\" : \"x-pack-test-password\",\n  \"api_key\" : {\n    \"name\": \"my-api-key\",\n    \"expiration\": \"1d\",\n    \"role_descriptors\": {\n      \"role-a\": {\n        \"cluster\": [\"all\"],\n        \"indices\": [\n          {\n          \"names\": [\"index-a*\"],\n          \"privileges\": [\"read\"]\n          }\n        ]\n      },\n      \"role-b\": {\n        \"cluster\": [\"all\"],\n        \"indices\": [\n          {\n          \"names\": [\"index-b*\"],\n          \"privileges\": [\"all\"]\n          }\n        ]\n      }\n    },\n    \"metadata\": {\n      \"application\": \"my-application\",\n      \"environment\": {\n        \"level\": 1,\n        \"trusted\": true,\n        \"tags\": [\"dev\", \"staging\"]\n      }\n    }\n  }\n}"'
Request examples
Run `POST /_security/api_key/grant` to create an API key on behalf of the `test_admin` user. 
{
  "grant_type": "password",
  "username" : "test_admin",
  "password" : "x-pack-test-password",
  "api_key" : {
    "name": "my-api-key",
    "expiration": "1d",
    "role_descriptors": {
      "role-a": {
        "cluster": ["all"],
        "indices": [
          {
          "names": ["index-a*"],
          "privileges": ["read"]
          }
        ]
      },
      "role-b": {
        "cluster": ["all"],
        "indices": [
          {
          "names": ["index-b*"],
          "privileges": ["all"]
          }
        ]
      }
    },
    "metadata": {
      "application": "my-application",
      "environment": {
        "level": 1,
        "trusted": true,
        "tags": ["dev", "staging"]
      }
    }
  }
}
Run `POST /_security/api_key/grant`. The user (`test_admin`) whose credentials are provided can "run as" another user (`test_user`). The API key will be granted to the impersonated user (`test_user`). 
{
  "grant_type": "password",
  "username" : "test_admin",  
  "password" : "x-pack-test-password",  
  "run_as": "test_user",  
  "api_key" : {
    "name": "another-api-key"
  }
}

Check user privileges Added in 6.4.0

POST /_security/user/_has_privileges
Api key auth Basic auth Bearer auth

Determine whether the specified user has a specified list of privileges. All users can use this API, but only to determine their own privileges. To check the privileges of other users, you must use the run as feature.

External documentation
application/json

Body Required

  • application array[object]
    Hide application attributes Show application attributes object
    • application string Required

      The name of the application.

    • privileges array[string] Required

      A list of the privileges that you want to check for the specified resources. It may be either application privilege names or the names of actions that are granted by those privileges

    • resources array[string] Required

      A list of resource names against which the privileges should be checked.

  • cluster array[string]

    A list of the cluster privileges that you want to check.

  • index array[object]
    Hide index attributes Show index attributes object
    • names string | array[string] Required
    • privileges array[string] Required

      A list of the privileges that you want to check for the specified indices.

    • allow_restricted_indices boolean

      This needs to be set to true (default is false) if using wildcards or regexps for patterns that cover restricted indices. Implicitly, restricted indices do not match index patterns because restricted indices usually have limited privileges and including them in pattern tests would render most such tests false. If restricted indices are explicitly included in the names list, privileges will be checked against them regardless of the value of allow_restricted_indices.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • application object Required
      Hide application attribute Show application attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • * boolean Additional properties
    • cluster object Required
      Hide cluster attribute Show cluster attribute object
      • * boolean Additional properties
    • has_all_requested boolean Required
    • index object Required
      Hide index attribute Show index attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • * boolean Additional properties
    • username string Required
POST /_security/user/_has_privileges 
curl \
 --request POST 'http://api.example.com/_security/user/_has_privileges' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"cluster\": [ \"monitor\", \"manage\" ],\n  \"index\" : [\n    {\n      \"names\": [ \"suppliers\", \"products\" ],\n      \"privileges\": [ \"read\" ]\n    },\n    {\n      \"names\": [ \"inventory\" ],\n      \"privileges\" : [ \"read\", \"write\" ]\n    }\n  ],\n  \"application\": [\n    {\n      \"application\": \"inventory_manager\",\n      \"privileges\" : [ \"read\", \"data:write/inventory\" ],\n      \"resources\" : [ \"product/1852563\" ]\n    }\n  ]\n}"'
Request example
Run `GET /_security/user/_has_privileges` to check whether the current user has a specific set of cluster, index, and application privileges. 
{
  "cluster": [ "monitor", "manage" ],
  "index" : [
    {
      "names": [ "suppliers", "products" ],
      "privileges": [ "read" ]
    },
    {
      "names": [ "inventory" ],
      "privileges" : [ "read", "write" ]
    }
  ],
  "application": [
    {
      "application": "inventory_manager",
      "privileges" : [ "read", "data:write/inventory" ],
      "resources" : [ "product/1852563" ]
    }
  ]
}
Response examples (200)
A successful response from `GET /_security/user/_has_privileges`, which lists the privileges for the `rdeniro` user. 
{
  "username": "rdeniro",
  "has_all_requested" : false,
  "cluster" : {
    "monitor" : true,
    "manage" : false
  },
  "index" : {
    "suppliers" : {
      "read" : true
    },
    "products" : {
      "read" : true
    },
    "inventory" : {
      "read" : true,
      "write" : false
    }
  },
  "application" : {
    "inventory_manager" : {
      "product/1852563" : {
        "read": false,
        "data:write/inventory": false
      }
    }
  }
}

Find roles with a query Added in 8.15.0

GET /_security/_query/role
Api key auth Basic auth Bearer auth

Get roles in a paginated manner. The role management APIs are generally the preferred way to manage roles, rather than using file-based role management. The query roles API does not retrieve roles that are defined in roles files, nor built-in ones. You can optionally filter the results with a query. Also, the results can be paginated and sorted.

application/json

Body

  • query object
    Hide query attributes Show query attributes object
    • bool
    • exists
    • ids
    • match object

      Returns roles that match a provided text, number, date or boolean value. The provided text is analyzed before matching.

    • match_all
    • prefix object

      Returns roles that contain a specific prefix in a provided field.

    • range object

      Returns roles that contain terms within a provided range.

    • simple_query_string
    • term object

      Returns roles that contain an exact term in a provided field. To return a document, the query term must exactly match the queried field's value, including whitespace and capitalization.

    • terms
    • wildcard object

      Returns roles that contain terms matching a wildcard pattern.

  • from number

    The starting document offset. It must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • sort string | object | array[string | object]

    One of:
    Field string SortOptions object Sort array[string | object]

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • size number

    The number of hits to return. It must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • search_after array[number | string | boolean | null]

    A field value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • total number Required

      The total number of roles found.

    • count number Required

      The number of roles returned in the response.

    • roles array[object] Required

      A list of roles that match the query. The returned role format is an extension of the role definition format. It adds the transient_metadata.enabled and the _sort fields. transient_metadata.enabled is set to false in case the role is automatically disabled, for example when the role grants privileges that are not allowed by the installed license. _sort is present when the search query sorts on some field. It contains the array of values that have been used for sorting.

      Hide roles attributes Show roles attributes object
      • cluster array[string]

        A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

      • indices array[object]

        A list of indices permissions entries.

        Hide indices attributes Show indices attributes object
        • field_security object
        • names
        • privileges array[string] Required

          The index level privileges that owners of the role have on the specified indices.

        • query
        • allow_restricted_indices boolean

          Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • remote_indices array[object]

        A list of indices permissions for remote clusters.

        Hide remote_indices attributes Show remote_indices attributes object
        • clusters string | array[string] Required
        • field_security object
        • names
        • privileges array[string] Required

          The index level privileges that owners of the role have on the specified indices.

        • query
        • allow_restricted_indices boolean

          Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • remote_cluster array[object]

        A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

        Hide remote_cluster attributes Show remote_cluster attributes object
        • clusters string | array[string] Required
        • privileges array[string] Required

          The cluster level privileges that owners of the role have on the remote cluster.

          Values are monitor_enrich or monitor_stats.

      • global array[object] | object

        An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

        One of:
        array-1 array[object] GlobalPrivilege object
      • applications array[object]

        A list of application privilege entries

        Hide applications attributes Show applications attributes object
        • application string Required

          The name of the application to which this entry applies.

        • privileges array[string] Required

          A list of strings, where each element is the name of an application privilege or action.

        • resources array[string] Required

          A list resources to which the privileges are applied.

      • metadata object
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • run_as array[string]

        A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

      • description string

        Optional description of the role descriptor

      • restriction object
        Hide restriction attribute Show restriction attribute object
        • workflows array[string] Required

          A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

      • transient_metadata object
        Hide transient_metadata attribute Show transient_metadata attribute object
        • * object Additional properties
      • _sort array[number | string | boolean | null]

        A field value.

      • name string Required

        Name of the role.
GET /_security/_query/role 
curl \
 --request GET 'http://api.example.com/_security/_query/role' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"sort\": [\"name\"]\n}"'
Request examples
Run `POST /_security/_query/role` to lists all roles, sorted by the role name. 
{
    "sort": ["name"]
}
Run `POST /_security/_query/role` to query only the user access role, given its description. It returns only the best matching role because `size` is set to `1`. 
{
  "query": {
    "match": {
      "description": {
        "query": "user access"
      }
    }
  },
  "size": 1 
}
Response examples (200)
A successful response from `POST /_security/_query/role`. It returns a JSON structure that contains the information retrieved for one or more roles. 
{
    "total": 2,
    "count": 2,
    "roles": [ 
        {
          "name" : "my_admin_role",
          "cluster" : [
            "all"
          ],
          "indices" : [
            {
              "names" : [
                "index1",
                "index2"
              ],
              "privileges" : [
                "all"
              ],
              "field_security" : {
                "grant" : [
                  "title",
                  "body"
                ]
              },
              "allow_restricted_indices" : false
            }
          ],
          "applications" : [ ],
          "run_as" : [
            "other_user"
          ],
          "metadata" : {
            "version" : 1
          },
          "transient_metadata" : {
            "enabled" : true
          },
          "description" : "Grants full access to all management features within the cluster.",
          "_sort" : [
            "my_admin_role"
          ]
        },
        {
          "name" : "my_user_role",
          "cluster" : [ ],
          "indices" : [
            {
              "names" : [
                "index1",
                "index2"
              ],
              "privileges" : [
                "all"
              ],
              "field_security" : {
                "grant" : [
                  "title",
                  "body"
                ]
              },
              "allow_restricted_indices" : false
            }
          ],
          "applications" : [ ],
          "run_as" : [ ],
          "metadata" : {
            "version" : 1
          },
          "transient_metadata" : {
            "enabled" : true
          },
          "description" : "Grants user access to some indicies.",
          "_sort" : [
            "my_user_role"
          ]
        }
    ]
}
A successful response from `POST /_security/_query/role`. 
{
    "total": 2,
    "count": 1,
    "roles": [
        {
          "name" : "my_user_role",
          "cluster" : [ ],
          "indices" : [
            {
              "names" : [
                "index1",
                "index2"
              ],
              "privileges" : [
                "all"
              ],
              "field_security" : {
                "grant" : [
                  "title",
                  "body"
                ]
              },
              "allow_restricted_indices" : false
            }
          ],
          "applications" : [ ],
          "run_as" : [ ],
          "metadata" : {
            "version" : 1
          },
          "transient_metadata" : {
            "enabled" : true
          },
          "description" : "Grants user access to some indicies."
        }
    ]
}

Authenticate SAML Added in 7.5.0

POST /_security/saml/authenticate
Api key auth Basic auth Bearer auth

Submit a SAML response message to Elasticsearch for consumption.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

The SAML message that is submitted can be:

  • A response to a SAML authentication request that was previously created using the SAML prepare authentication API.
  • An unsolicited SAML message in the case of an IdP-initiated single sign-on (SSO) flow.

In either case, the SAML message needs to be a base64 encoded XML document with a root element of <Response>.

After successful validation, Elasticsearch responds with an Elasticsearch internal access token and refresh token that can be subsequently used for authentication. This API endpoint essentially exchanges SAML responses that indicate successful authentication in the IdP for Elasticsearch access and refresh tokens, which can be used for authentication against Elasticsearch.

External documentation
application/json

Body Required

  • content string Required

    The SAML response as it was sent by the user's browser, usually a Base64 encoded XML document.

  • ids string | array[string] Required

    One of:
    Id string Ids array[string]
  • realm string

    The name of the realm that should authenticate the SAML response. Useful in cases where many SAML realms are defined.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • access_token string Required

      The access token that was generated by Elasticsearch.

    • username string Required

      The authenticated user's name.

    • expires_in number Required

      The amount of time (in seconds) left until the token expires.

    • refresh_token string Required

      The refresh token that was generated by Elasticsearch.

    • realm string Required

      The name of the realm where the user was authenticated.
POST /_security/saml/authenticate 
curl \
 --request POST 'http://api.example.com/_security/saml/authenticate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"content\" : \"PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2wiIHhtbG5zOnNhbWw9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMD.....\",\n  \"ids\" : [\"4fee3b046395c4e751011e97f8900b5273d56685\"]\n}"'
Request example
Run `POST /_security/saml/authenticate` to exchange a SAML Response indicating a successful authentication at the SAML IdP for an Elasticsearch access token and refresh token to be used in subsequent requests. 
{
  "content" : "PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2wiIHhtbG5zOnNhbWw9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMD.....",
  "ids" : ["4fee3b046395c4e751011e97f8900b5273d56685"]
}
Response examples (200)
A successful response from `POST /_security/saml/authenticate`. 
{
  "access_token" : "46ToAxZVaXVVZTVKOVF5YU04ZFJVUDVSZlV3",
  "username" : "Bearer",
  "expires_in" : 1200,
  "refresh_token": "mJdXLtmvTUSpoLwMvdBt_w",
  "realm": "saml1"
}

Logout of SAML Added in 7.5.0

POST /_security/saml/logout
Api key auth Basic auth Bearer auth

Submits a request to invalidate an access token and refresh token.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

This API invalidates the tokens that were generated for a user by the SAML authenticate API. If the SAML realm in Elasticsearch is configured accordingly and the SAML IdP supports this, the Elasticsearch response contains a URL to redirect the user to the IdP that contains a SAML logout request (starting an SP-initiated SAML Single Logout).

External documentation
application/json

Body Required

  • token string Required

    The access token that was returned as a response to calling the SAML authenticate API. Alternatively, the most recent token that was received after refreshing the original one by using a refresh_token.

  • refresh_token string

    The refresh token that was returned as a response to calling the SAML authenticate API. Alternatively, the most recent refresh token that was received after refreshing the original access token.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • redirect string Required

      A URL that contains a SAML logout request as a parameter. You can use this URL to be redirected back to the SAML IdP and to initiate Single Logout.
POST /_security/saml/logout 
curl \
 --request POST 'http://api.example.com/_security/saml/logout' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"token\" : \"46ToAxZVaXVVZTVKOVF5YU04ZFJVUDVSZlV3\",\n  \"refresh_token\" : \"mJdXLtmvTUSpoLwMvdBt_w\"\n}"'
Request example
Run `POST /_security/saml/logout` to invalidate the pair of tokens that were generated by calling the SAML authenticate API with a successful SAML response. 
{
  "token" : "46ToAxZVaXVVZTVKOVF5YU04ZFJVUDVSZlV3",
  "refresh_token" : "mJdXLtmvTUSpoLwMvdBt_w"
}
Response examples (200)
A successful response from `POST /_security/saml/logout`. 
{
  "redirect" : "https://my-idp.org/logout/SAMLRequest=...."
}

Create a snapshot Added in 0.0.0

PUT /_snapshot/{repository}/{snapshot}
Api key auth Basic auth Bearer auth

Take a snapshot of a cluster or of data streams and indices.

External documentation

Path parameters

  • repository string Required

    The name of the repository for the snapshot.

  • snapshot string Required

    The name of the snapshot. It supportes date math. It must be unique in the repository.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_completion boolean

    If true, the request returns a response when the snapshot is complete. If false, the request returns a response when the snapshot initializes.

application/json

Body

  • expand_wildcards string | array[string]
  • feature_states array[string]

    The feature states to include in the snapshot. Each feature state includes one or more system indices containing related data. You can view a list of eligible features using the get features API.

    If include_global_state is true, all current feature states are included by default. If include_global_state is false, no feature states are included by default.

    Note that specifying an empty array will result in the default behavior. To exclude all feature states, regardless of the include_global_state value, specify an array with only the value none (["none"]).

  • ignore_unavailable boolean

    If true, the request ignores data streams and indices in indices that are missing or closed. If false, the request returns an error for any data stream or index that is missing or closed.

  • include_global_state boolean

    If true, the current cluster state is included in the snapshot. The cluster state includes persistent cluster settings, composable index templates, legacy index templates, ingest pipelines, and ILM policies. It also includes data stored in system indices, such as Watches and task records (configurable via feature_states).

  • indices string | array[string]
  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • partial boolean

    If true, it enables you to restore a partial snapshot of indices with unavailable shards. Only shards that were successfully included in the snapshot will be restored. All missing shards will be recreated as empty.

    If false, the entire restore operation will fail if one or more indices included in the snapshot do not have all primary shards available.

Responses

PUT /_snapshot/{repository}/{snapshot} 
curl \
 --request PUT 'http://api.example.com/_snapshot/{repository}/{snapshot}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"indices\": \"index_1,index_2\",\n  \"ignore_unavailable\": true,\n  \"include_global_state\": false,\n  \"metadata\": {\n    \"taken_by\": \"user123\",\n    \"taken_because\": \"backup before upgrading\"\n  }\n}"'
Request example
Run `PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true` to take a snapshot of `index_1` and `index_2`. 
{
  "indices": "index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": false,
  "metadata": {
    "taken_by": "user123",
    "taken_because": "backup before upgrading"
  }
}
Response examples (200)
A successful response from `PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true`. 
{
  "snapshot": {
    "snapshot": "snapshot_2",
    "uuid": "vdRctLCxSketdKb54xw67g",
    "repository": "my_repository",
    "version_id": <version_id>,
    "version": <version>,
    "indices": [],
    "data_streams": [],
    "feature_states": [],
    "include_global_state": false,
    "metadata": {
      "taken_by": "user123",
      "taken_because": "backup before upgrading"
    },
    "state": "SUCCESS",
    "start_time": "2020-06-25T14:00:28.850Z",
    "start_time_in_millis": 1593093628850,
    "end_time": "2020-06-25T14:00:28.850Z",
    "end_time_in_millis": 1593094752018,
    "duration_in_millis": 0,
    "failures": [],
    "shards": {
      "total": 0,
      "failed": 0,
      "successful": 0
    }
  }
}

Delete snapshot repositories Added in 0.0.0

DELETE /_snapshot/{repository}
Api key auth Basic auth Bearer auth

When a repository is unregistered, Elasticsearch removes only the reference to the location where the repository is storing the snapshots. The snapshots themselves are left untouched and in place.

Path parameters

  • repository string | array[string] Required

    The ame of the snapshot repositories to unregister. Wildcard (*) patterns are supported.

Query parameters

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. To indicate that the request should never timeout, set it to -1.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_snapshot/{repository} 
curl \
 --request DELETE 'http://api.example.com/_snapshot/{repository}' \
 --header "Authorization: $API_KEY"

Analyze a snapshot repository Added in 7.12.0

POST /_snapshot/{repository}/_analyze
Api key auth Basic auth Bearer auth

Analyze the performance characteristics and any incorrect behaviour found in a repository.

The response exposes implementation details of the analysis which may change from version to version. The response body format is therefore not considered stable and may be different in newer versions.

There are a large number of third-party storage systems available, not all of which are suitable for use as a snapshot repository by Elasticsearch. Some storage systems behave incorrectly, or perform poorly, especially when accessed concurrently by multiple clients as the nodes of an Elasticsearch cluster do. This API performs a collection of read and write operations on your repository which are designed to detect incorrect behaviour and to measure the performance characteristics of your storage system.

The default values for the parameters are deliberately low to reduce the impact of running an analysis inadvertently and to provide a sensible starting point for your investigations. Run your first analysis with the default parameter values to check for simple problems. If successful, run a sequence of increasingly large analyses until you encounter a failure or you reach a blob_count of at least 2000, a max_blob_size of at least 2gb, a max_total_data_size of at least 1tb, and a register_operation_count of at least 100. Always specify a generous timeout, possibly 1h or longer, to allow time for each analysis to run to completion. Perform the analyses using a multi-node cluster of a similar size to your production cluster so that it can detect any problems that only arise when the repository is accessed by many nodes at once.

If the analysis fails, Elasticsearch detected that your repository behaved unexpectedly. This usually means you are using a third-party storage system with an incorrect or incompatible implementation of the API it claims to support. If so, this storage system is not suitable for use as a snapshot repository. You will need to work with the supplier of your storage system to address the incompatibilities that Elasticsearch detects.

If the analysis is successful, the API returns details of the testing process, optionally including how long each operation took. You can use this information to determine the performance of your storage system. If any operation fails or returns an incorrect result, the API returns an error. If the API returns an error, it may not have removed all the data it wrote to the repository. The error will indicate the location of any leftover data and this path is also recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the connection from your client to Elasticsearch is closed while the client is waiting for the result of the analysis, the test is cancelled. Some clients are configured to close their connection if no response is received within a certain timeout. An analysis takes a long time to complete so you might need to relax any such client-side timeouts. On cancellation the analysis attempts to clean up the data it was writing, but it may not be able to remove it all. The path to the leftover data is recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the analysis is successful then it detected no incorrect behaviour, but this does not mean that correct behaviour is guaranteed. The analysis attempts to detect common bugs but it does not offer 100% coverage. Additionally, it does not test the following:

  • Your repository must perform durable writes. Once a blob has been written it must remain in place until it is deleted, even after a power loss or similar disaster.
  • Your repository must not suffer from silent data corruption. Once a blob has been written, its contents must remain unchanged until it is deliberately modified or deleted.
  • Your repository must behave correctly even if connectivity from the cluster is disrupted. Reads and writes may fail in this case, but they must not return incorrect results.

IMPORTANT: An analysis writes a substantial amount of data to your repository and then reads it back again. This consumes bandwidth on the network between the cluster and the repository, and storage space and I/O bandwidth on the repository itself. You must ensure this load does not affect other users of these systems. Analyses respect the repository settings max_snapshot_bytes_per_sec and max_restore_bytes_per_sec if available and the cluster setting indices.recovery.max_bytes_per_sec which you can use to limit the bandwidth they consume.

NOTE: This API is intended for exploratory use by humans. You should expect the request parameters and the response format to vary in future versions.

NOTE: Different versions of Elasticsearch may perform different checks for repository compatibility, with newer versions typically being stricter than older ones. A storage system that passes repository analysis with one version of Elasticsearch may fail with a different version. This indicates it behaves incorrectly in ways that the former version did not detect. You must work with the supplier of your storage system to address the incompatibilities detected by the repository analysis API in any version of Elasticsearch.

NOTE: This API may not work correctly in a mixed-version cluster.

Implementation details

NOTE: This section of documentation describes how the repository analysis API works in this version of Elasticsearch, but you should expect the implementation to vary between versions. The request parameters and response format depend on details of the implementation so may also be different in newer versions.

The analysis comprises a number of blob-level tasks, as set by the blob_count parameter and a number of compare-and-exchange operations on linearizable registers, as set by the register_operation_count parameter. These tasks are distributed over the data and master-eligible nodes in the cluster for execution.

For most blob-level tasks, the executing node first writes a blob to the repository and then instructs some of the other nodes in the cluster to attempt to read the data it just wrote. The size of the blob is chosen randomly, according to the max_blob_size and max_total_data_size parameters. If any of these reads fails then the repository does not implement the necessary read-after-write semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will instruct some of its peers to attempt to read the data before the writing process completes. These reads are permitted to fail, but must not return partial data. If any read returns partial data then the repository does not implement the necessary atomicity semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will overwrite the blob while its peers are reading it. In this case the data read may come from either the original or the overwritten blob, but the read operation must not return partial data or a mix of data from the two blobs. If any of these reads returns partial data or a mix of the two blobs then the repository does not implement the necessary atomicity semantics that Elasticsearch requires for overwrites.

The executing node will use a variety of different methods to write the blob. For instance, where applicable, it will use both single-part and multi-part uploads. Similarly, the reading nodes will use a variety of different methods to read the data back again. For instance they may read the entire blob from start to end or may read only a subset of the data.

For some blob-level tasks, the executing node will cancel the write before it is complete. In this case, it still instructs some of the other nodes in the cluster to attempt to read the blob but all of these reads must fail to find the blob.

Linearizable registers are special blobs that Elasticsearch manipulates using an atomic compare-and-exchange operation. This operation ensures correct and strongly-consistent behavior even when the blob is accessed by multiple nodes at the same time. The detailed implementation of the compare-and-exchange operation on linearizable registers varies by repository type. Repository analysis verifies that that uncontended compare-and-exchange operations on a linearizable register blob always succeed. Repository analysis also verifies that contended operations either succeed or report the contention but do not return incorrect results. If an operation fails due to contention, Elasticsearch retries the operation until it succeeds. Most of the compare-and-exchange operations performed by repository analysis atomically increment a counter which is represented as an 8-byte blob. Some operations also verify the behavior on small blobs with sizes other than 8 bytes.

Path parameters

Query parameters

  • blob_count number

    The total number of blobs to write to the repository during the test. For realistic experiments, you should set it to at least 2000.

  • concurrency number

    The number of operations to run concurrently during the test.

  • detailed boolean

    Indicates whether to return detailed results, including timing information for every operation performed during the analysis. If false, it returns only a summary of the analysis.

  • early_read_node_count number

    The number of nodes on which to perform an early read operation while writing each blob. Early read operations are only rarely performed.

  • max_blob_size number | string

    The maximum size of a blob to be written during the test. For realistic experiments, you should set it to at least 2gb.

  • max_total_data_size number | string

    An upper limit on the total size of all the blobs written during the test. For realistic experiments, you should set it to at least 1tb.

  • rare_action_probability number

    The probability of performing a rare action such as an early read, an overwrite, or an aborted write on each blob.

  • rarely_abort_writes boolean

    Indicates whether to rarely cancel writes before they complete.

  • read_node_count number

    The number of nodes on which to read a blob after writing.

  • register_operation_count number

    The minimum number of linearizable register operations to perform in total. For realistic experiments, you should set it to at least 100.

  • seed number

    The seed for the pseudo-random number generator used to generate the list of operations performed during the test. To repeat the same set of operations in multiple experiments, use the same seed in each experiment. Note that the operations are performed concurrently so might not always happen in the same order on each run.

  • timeout string

    The period of time to wait for the test to complete. If no response is received before the timeout expires, the test is cancelled and returns an error.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • blob_count number Required

      The number of blobs written to the repository during the test.

    • blob_path string Required

      The path in the repository under which all the blobs were written during the test.

    • concurrency number Required

      The number of write operations performed concurrently during the test.

    • coordinating_node object Required
      Hide coordinating_node attributes Show coordinating_node attributes object
    • delete_elapsed string Required

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • delete_elapsed_nanos number

      Time unit for nanoseconds

    • details object Required
      Hide details attributes Show details attributes object
      • blob object Required
        Hide blob attributes Show blob attributes object
        • name string Required

          The name of the blob.

        • overwritten boolean Required

          Indicates whether the blob was overwritten while the read operations were ongoing. /**

        • read_early boolean Required
        • read_end number Required

          The position, in bytes, at which read operations completed.

        • read_start number Required

          The position, in bytes, at which read operations started.

        • reads object Required
          Hide reads attributes Show reads attributes object
          • before_write_complete boolean

            Indicates whether the read operation may have started before the write operation was complete.

          • elapsed string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • elapsed_nanos number

            Time unit for nanoseconds

          • first_byte_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • first_byte_time_nanos number

            Time unit for nanoseconds

          • found boolean Required

            Indicates whether the blob was found by the read operation. If the read was started before the write completed or the write was ended before completion, it might be false.

          • node object Required
            Hide node attributes Show node attributes object
          • throttled string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttled_nanos number

            Time unit for nanoseconds

        • size number | string Required

          One of:
          ByteSize number ByteSize string
        • size_bytes number Required

          The size of the blob in bytes.

      • overwrite_elapsed string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • overwrite_elapsed_nanos number

        Time unit for nanoseconds

      • write_elapsed string Required

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • write_elapsed_nanos number

        Time unit for nanoseconds

      • write_throttled string Required

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • write_throttled_nanos number

        Time unit for nanoseconds

      • writer_node object Required
        Hide writer_node attributes Show writer_node attributes object
    • early_read_node_count number Required

      The limit on the number of nodes on which early read operations were performed after writing each blob.

    • issues_detected array[string] Required

      A list of correctness issues detected, which is empty if the API succeeded. It is included to emphasize that a successful response does not guarantee correct behaviour in future.

    • listing_elapsed string Required

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • listing_elapsed_nanos number

      Time unit for nanoseconds

    • max_blob_size number | string Required

      One of:
      ByteSize number ByteSize string
    • max_blob_size_bytes number Required

      The limit, in bytes, on the size of a blob written during the test.

    • max_total_data_size number | string Required

      One of:
      ByteSize number ByteSize string
    • max_total_data_size_bytes number Required

      The limit, in bytes, on the total size of all blob written during the test.

    • rare_action_probability number Required

      The probability of performing rare actions during the test.

    • read_node_count number Required

      The limit on the number of nodes on which read operations were performed after writing each blob.

    • repository string Required

      The name of the repository that was the subject of the analysis.

    • seed number Required

      The seed for the pseudo-random number generator used to generate the operations used during the test.

    • summary object Required
      Hide summary attributes Show summary attributes object
      • read object Required
        Hide read attributes Show read attributes object
        • count number Required

          The number of read operations performed in the test.

        • max_wait string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • max_wait_nanos number

          Time unit for nanoseconds

        • total_elapsed string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_elapsed_nanos number

          Time unit for nanoseconds

        • total_size number | string Required

          One of:
          ByteSize number ByteSize string
        • total_size_bytes number Required

          The total size of all the blobs or partial blobs read in the test, in bytes.

        • total_throttled string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_throttled_nanos number

          Time unit for nanoseconds

        • total_wait string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_wait_nanos number

          Time unit for nanoseconds

      • write object Required
        Hide write attributes Show write attributes object
        • count number Required

          The number of write operations performed in the test.

        • total_elapsed string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_elapsed_nanos number

          Time unit for nanoseconds

        • total_size number | string Required

          One of:
          ByteSize number ByteSize string
        • total_size_bytes number Required

          The total size of all the blobs written in the test, in bytes.

        • total_throttled string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_throttled_nanos number Required

          The total time spent waiting due to the max_snapshot_bytes_per_sec throttle, in nanoseconds.
POST /_snapshot/{repository}/_analyze 
curl \
 --request POST 'http://api.example.com/_snapshot/{repository}/_analyze' \
 --header "Authorization: $API_KEY"

Get the snapshot status Added in 7.8.0

GET /_snapshot/{repository}/_status
Api key auth Basic auth Bearer auth

Get a detailed description of the current state for each shard participating in the snapshot.

Note that this API should be used only to obtain detailed shard-level information for ongoing snapshots. If this detail is not needed or you want to obtain information about one or more existing snapshots, use the get snapshot API.

If you omit the <snapshot> request path parameter, the request retrieves information only for currently running snapshots. This usage is preferred. If needed, you can specify <repository> and <snapshot> to retrieve information for specific snapshots, even if they're not currently running.

WARNING: Using the API to return the status of any snapshots other than currently running snapshots can be expensive. The API requires a read from the repository for each shard in each snapshot. For example, if you have 100 snapshots with 1,000 shards each, an API request that includes all snapshots will require 100,000 reads (100 snapshots x 1,000 shards).

Depending on the latency of your storage, such requests can take an extremely long time to return results. These requests can also tax machine resources and, when using cloud storage, incur high processing costs.

Path parameters

  • repository string Required

    The snapshot repository name used to limit the request. It supports wildcards (*) if <snapshot> isn't specified.

Query parameters

  • ignore_unavailable boolean

    If false, the request returns an error for any snapshots that are unavailable. If true, the request ignores snapshots that are unavailable, such as those that are corrupted or temporarily cannot be returned.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • snapshots array[object] Required
      Hide snapshots attributes Show snapshots attributes object
      • include_global_state boolean Required

        Indicates whether the current cluster state is included in the snapshot.

      • indices object Required
        Hide indices attribute Show indices attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • shards object Required
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shards_stats object Required
            Hide shards_stats attributes Show shards_stats attributes object
            • done number Required

              The number of shards that initialized, started, and finalized successfully.

            • failed number Required

              The number of shards that failed to be included in the snapshot.

            • finalizing number Required

              The number of shards that are finalizing but are not done.

            • initializing number Required

              The number of shards that are still initializing.

            • started number Required

              The number of shards that have started but are not finalized.

            • total number Required

              The total number of shards included in the snapshot.

          • stats object Required
            Hide stats attributes Show stats attributes object
      • repository string Required

        The name of the repository that includes the snapshot.

      • shards_stats object Required
        Hide shards_stats attributes Show shards_stats attributes object
        • done number Required

          The number of shards that initialized, started, and finalized successfully.

        • failed number Required

          The number of shards that failed to be included in the snapshot.

        • finalizing number Required

          The number of shards that are finalizing but are not done.

        • initializing number Required

          The number of shards that are still initializing.

        • started number Required

          The number of shards that have started but are not finalized.

        • total number Required

          The total number of shards included in the snapshot.

      • snapshot string Required

        The name of the snapshot.

      • state string Required

        The current snapshot state:

        • FAILED: The snapshot finished with an error and failed to store any data.
        • STARTED: The snapshot is currently running.
        • SUCCESS: The snapshot completed.
      • stats object Required
        Hide stats attributes Show stats attributes object
      • uuid string Required
GET /_snapshot/{repository}/_status 
curl \
 --request GET 'http://api.example.com/_snapshot/{repository}/_status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _snapshot/my_repository/snapshot_2/_status`. The response contains detailed status information for `snapshot_2` in the `my_repository` repository. 
{
  "snapshots" : [
    {
      "snapshot" : "snapshot_2",
      "repository" : "my_repository",
      "uuid" : "lNeQD1SvTQCqqJUMQSwmGg",
      "state" : "SUCCESS",
      "include_global_state" : false,
      "shards_stats" : {
        "initializing" : 0,
        "started" : 0,
        "finalizing" : 0,
        "done" : 1,
        "failed" : 0,
        "total" : 1
      },
      "stats" : {
        "incremental" : {
          "file_count" : 3,
          "size_in_bytes" : 5969
        },
        "total" : {
          "file_count" : 4,
          "size_in_bytes" : 6024
        },
        "start_time_in_millis" : 1594829326691,
        "time_in_millis" : 205
      },
      "indices" : {
        "index_1" : {
          "shards_stats" : {
            "initializing" : 0,
            "started" : 0,
            "finalizing" : 0,
            "done" : 1,
            "failed" : 0,
            "total" : 1
          },
          "stats" : {
            "incremental" : {
              "file_count" : 3,
              "size_in_bytes" : 5969
            },
            "total" : {
              "file_count" : 4,
              "size_in_bytes" : 6024
            },
            "start_time_in_millis" : 1594829326896,
            "time_in_millis" : 0
          },
          "shards" : {
            "0" : {
              "stage" : "DONE",
              "stats" : {
                "incremental" : {
                  "file_count" : 3,
                  "size_in_bytes" : 5969
                },
                "total" : {
                  "file_count" : 4,
                  "size_in_bytes" : 6024
                },
                "start_time_in_millis" : 1594829326896,
                "time_in_millis" : 0
              }
            }
          }
        }
      }
    }
  ]
}

Verify a snapshot repository Added in 0.0.0

POST /_snapshot/{repository}/_verify
Api key auth Basic auth Bearer auth

Check for common misconfigurations in a snapshot repository.

External documentation

Path parameters

  • repository string Required

    The name of the snapshot repository to verify.

Query parameters

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. To indicate that the request should never timeout, set it to -1.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • nodes object Required

      Information about the nodes connected to the snapshot repository. The key is the ID of the node.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
POST /_snapshot/{repository}/_verify 
curl \
 --request POST 'http://api.example.com/_snapshot/{repository}/_verify' \
 --header "Authorization: $API_KEY"

Create or update a policy Added in 7.4.0

PUT /_slm/policy/{policy_id}
Api key auth Basic auth Bearer auth

Create or update a snapshot lifecycle policy. If the policy already exists, this request increments the policy version. Only the latest version of a policy is stored.

Path parameters

  • policy_id string Required

    The identifier for the snapshot lifecycle policy you want to create or update.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

application/json

Body

  • config object
    Hide config attributes Show config attributes object
    • ignore_unavailable boolean

      If false, the snapshot fails if any data stream or index in indices is missing or closed. If true, the snapshot ignores missing or closed data streams and indices.

    • indices string | array[string]
    • include_global_state boolean

      If true, the current global state is included in the snapshot.

    • feature_states array[string]

      A list of feature states to be included in this snapshot. A list of features available for inclusion in the snapshot and their descriptions be can be retrieved using the get features API. Each feature state includes one or more system indices containing data necessary for the function of that feature. Providing an empty array will include no feature states in the snapshot, regardless of the value of include_global_state. By default, all available feature states will be included in the snapshot if include_global_state is true, or no feature states if include_global_state is false.

    • metadata object
      Hide metadata attribute Show metadata attribute object
      • * object Additional properties
    • partial boolean

      If false, the entire snapshot will fail if one or more indices included in the snapshot do not have all primary shards available.

  • name string
  • repository string

    Repository used to store snapshots created by this policy. This repository must exist prior to the policy’s creation. You can create a repository using the snapshot repository API.

  • retention object
    Hide retention attributes Show retention attributes object
    • expire_after string Required

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • max_count number Required

      Maximum number of snapshots to retain, even if the snapshots have not yet expired. If the number of snapshots in the repository exceeds this limit, the policy retains the most recent snapshots and deletes older snapshots.

    • min_count number Required

      Minimum number of snapshots to retain, even if the snapshots have expired.

  • schedule string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_slm/policy/{policy_id} 
curl \
 --request PUT 'http://api.example.com/_slm/policy/{policy_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"schedule\": \"0 30 1 * * ?\",\n  \"name\": \"\u003cdaily-snap-{now/d}\u003e\",\n  \"repository\": \"my_repository\",\n  \"config\": {\n    \"indices\": [\"data-*\", \"important\"],\n    \"ignore_unavailable\": false,\n    \"include_global_state\": false\n  },\n  \"retention\": {\n    \"expire_after\": \"30d\",\n    \"min_count\": 5,\n    \"max_count\": 50\n  }\n}"'
Request examples
Run `PUT /_slm/policy/daily-snapshots` to create a lifecycle policy. The `schedule` is when the snapshot should be taken, in this case, 1:30am daily. The `retention` details specify to: keep snapshots for 30 days; always keep at least 5 successful snapshots, even if they're more than 30 days old; keep no more than 50 successful snapshots, even if they're less than 30 days old. 
{
  "schedule": "0 30 1 * * ?",
  "name": "<daily-snap-{now/d}>",
  "repository": "my_repository",
  "config": {
    "indices": ["data-*", "important"],
    "ignore_unavailable": false,
    "include_global_state": false
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}
Run `PUT /_slm/policy/hourly-snapshots` to create a lifecycle policy that uses interval scheduling. It creates a snapshot once every hour. The first snapshot will be created one hour after the policy is modified, with subsequent snapshots every hour afterward. 
{
  "schedule": "1h",
  "name": "<hourly-snap-{now/d}>",
  "repository": "my_repository",
  "config": {
    "indices": ["data-*", "important"]
  }
}

Clear an SQL search cursor Added in 6.3.0

POST /_sql/close
Api key auth Basic auth Bearer auth
application/json

Body Required

  • cursor string Required

    Cursor to clear.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_sql/close 
curl \
 --request POST 'http://api.example.com/_sql/close' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"cursor\": \"sDXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAEWYUpOYklQMHhRUEtld3RsNnFtYU1hQQ==:BAFmBGRhdGUBZgVsaWtlcwFzB21lc3NhZ2UBZgR1c2Vy9f///w8=\"\n}"'
Request example
Run `POST _sql/close` to clear an SQL search cursor. 
{
  "cursor": "sDXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAEWYUpOYklQMHhRUEtld3RsNnFtYU1hQQ==:BAFmBGRhdGUBZgVsaWtlcwFzB21lc3NhZ2UBZgR1c2Vy9f///w8="
}

Get the async SQL search status Added in 7.15.0

GET /_sql/async/status/{id}
Api key auth Basic auth Bearer auth

Get the current status of an async SQL search or a stored synchronous SQL search.

Path parameters

  • id string Required

    The identifier for the search.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • expiration_time_in_millis number

      Time unit for milliseconds

    • id string Required

      The identifier for the search.

    • is_running boolean Required

      If true, the search is still running. If false, the search has finished.

    • is_partial boolean Required

      If true, the response does not contain complete search results. If is_partial is true and is_running is true, the search is still running. If is_partial is true but is_running is false, the results are partial due to a failure or timeout.

    • start_time_in_millis number

      Time unit for milliseconds

    • completion_status number
GET /_sql/async/status/{id} 
curl \
 --request GET 'http://api.example.com/_sql/async/status/{id}' \
 --header "Authorization: $API_KEY"

Delete a synonym set Added in 8.10.0

DELETE /_synonyms/{id}
Api key auth Basic auth Bearer auth

You can only delete a synonyms set that is not in use by any index analyzer.

Synonyms sets can be used in synonym graph token filters and synonym token filters. These synonym filters can be used as part of search analyzers.

Analyzers need to be loaded when an index is restored (such as when a node starts, or the index becomes open). Even if the analyzer is not used on any field mapping, it still needs to be loaded on the index recovery phase.

If any analyzers cannot be loaded, the index becomes unavailable and the cluster status becomes red or yellow as index shards are not available. To prevent that, synonyms sets that are used in analyzers can't be deleted. A delete request in this case will return a 400 response code.

To remove a synonyms set, you must first remove all indices that contain analyzers using it. You can migrate an index by creating a new index that does not contain the token filter with the synonyms set, and use the reindex API in order to copy over the index data. Once finished, you can delete the index. When the synonyms set is not used in analyzers, you will be able to delete it.

Path parameters

  • id string Required

    The synonyms set identifier to delete.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_synonyms/{id} 
curl \
 --request DELETE 'http://api.example.com/_synonyms/{id}' \
 --header "Authorization: $API_KEY"

Test a Grok pattern Added in 8.13.0

POST /_text_structure/test_grok_pattern
Api key auth Basic auth Bearer auth

Test a Grok pattern on one or more lines of text. The API indicates whether the lines match the pattern together with the offsets and lengths of the matched substrings.

External documentation

Query parameters

  • ecs_compatibility string

    The mode of compatibility with ECS compliant Grok patterns. Use this parameter to specify whether to use ECS Grok patterns instead of legacy ones when the structure finder creates a Grok pattern. Valid values are disabled and v1.

application/json

Body Required

  • grok_pattern string Required
  • text array[string] Required

    The lines of text to run the Grok pattern on.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • matches array[object] Required
      Hide matches attributes Show matches attributes object
      • matched boolean Required
      • fields object
        Hide fields attribute Show fields attribute object
        • * array[object] Additional properties
          Hide * attributes Show * attributes object
POST /_text_structure/test_grok_pattern 
curl \
 --request POST 'http://api.example.com/_text_structure/test_grok_pattern' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"grok_pattern\": \"Hello %{WORD:first_name} %{WORD:last_name}\",\n  \"text\": [\n    \"Hello John Doe\",\n    \"this does not match\"\n  ]\n}"'
Request example
Run `GET _text_structure/test_grok_pattern` to test a Grok pattern. 
{
  "grok_pattern": "Hello %{WORD:first_name} %{WORD:last_name}",
  "text": [
    "Hello John Doe",
    "this does not match"
  ]
}
Response examples (200)
A successful response from `GET _text_structure/test_grok_pattern`. 
{
  "matches": [
    {
      "matched": true,
      "fields": {
        "first_name": [
          {
            "match": "John",
            "offset": 6,
            "length": 4
          }
        ],
        "last_name": [
          {
            "match": "Doe",
            "offset": 11,
            "length": 3
          }
        ]
      }
    },
    {
      "matched": false
    }
  ]
}

Get transforms Added in 7.5.0

GET /_transform/{transform_id}
Api key auth Basic auth Bearer auth

Get configuration information for transforms.

Path parameters

  • transform_id string | array[string] Required

    Identifier for the transform. It can be a transform identifier or a wildcard expression. You can get information for all transforms by using _all, by specifying * as the <transform_id>, or by omitting the <transform_id>.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no transforms that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • from number

    Skips the specified number of transforms.

  • size number

    Specifies the maximum number of transforms to obtain.

  • exclude_generated boolean

    Excludes fields that were automatically added when creating the transform. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • transforms array[object] Required
      Hide transforms attributes Show transforms attributes object
      • authorization object
        Hide authorization attributes Show authorization attributes object
        • api_key object
          Hide api_key attributes Show api_key attributes object
          • id string Required

            The identifier for the API key.

          • name string Required

            The name of the API key.

        • roles array[string]

          If a user ID was used for the most recent update to the transform, its roles at the time of the update are listed in the response.

        • service_account string

          If a service account was used for the most recent update to the transform, the account name is listed in the response.

      • create_time number

        Time unit for milliseconds

      • create_time_string string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
      • description string

        Free text description of the transform.

      • dest object Required
        Hide dest attributes Show dest attributes object
      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • id string Required
      • latest object
        Hide latest attributes Show latest attributes object
        • sort string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • unique_key array[string] Required

          Specifies an array of one or more fields that are used to group the data.

      • pivot object
        Hide pivot attributes Show pivot attributes object
        • aggregations object

          Defines how to aggregate the grouped data. The following aggregations are currently supported: average, bucket script, bucket selector, cardinality, filter, geo bounds, geo centroid, geo line, max, median absolute deviation, min, missing, percentiles, rare terms, scripted metric, stats, sum, terms, top metrics, value count, weighted average.

        • group_by object

          Defines how to group the data. More than one grouping can be defined per pivot. The following groupings are currently supported: date histogram, geotile grid, histogram, terms.

          Hide group_by attribute Show group_by attribute object
      • retention_policy object
        Hide retention_policy attribute Show retention_policy attribute object
        • time object
          Hide time attributes Show time attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • max_age string Required

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • settings object
        Hide settings attributes Show settings attributes object
        • align_checkpoints boolean

          Specifies whether the transform checkpoint ranges should be optimized for performance. Such optimization can align checkpoint ranges with the date histogram interval when date histogram is specified as a group source in the transform config. As a result, less document updates in the destination index will be performed thus improving overall performance.

        • dates_as_epoch_millis boolean

          Defines if dates in the ouput should be written as ISO formatted string or as millis since epoch. epoch_millis was the default for transforms created before version 7.11. For compatible output set this value to true.

        • deduce_mappings boolean

          Specifies whether the transform should deduce the destination index mappings from the transform configuration.

        • docs_per_second number

          Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.

        • max_page_search_size number

          Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536.

        • unattended boolean

          If true, the transform runs in unattended mode. In unattended mode, the transform retries indefinitely in case of an error which means the transform never fails. Setting the number of retries other than infinite fails in validation.

      • source object Required
        Hide source attributes Show source attributes object
        • index string | array[string] Required
        • runtime_mappings object
          Hide runtime_mappings attribute Show runtime_mappings attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

            • input_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_index string
            • script object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • query object

          A query clause that retrieves a subset of data from the source index.

          Query DSL
      • sync object
        Hide sync attribute Show sync attribute object
        • time object
          Hide time attributes Show time attributes object
          • delay string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • version string
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
GET /_transform/{transform_id} 
curl \
 --request GET 'http://api.example.com/_transform/{transform_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response that contains configuration information for a transform. 
{
  "count": 1,
  "transforms": [
    {
      "id": "ecommerce_transform1",
      "authorization": {
        "roles": [
          "superuser"
        ]
      },
      "version": "8.4.0",
      "create_time": 1656023416565,
      "source": {
        "index": [
          "kibana_sample_data_ecommerce"
        ],
        "query": {
          "term": {
            "geoip.continent_name": {
              "value": "Asia"
            }
          }
        }
      },
      "dest": {
        "index": "kibana_sample_data_ecommerce_transform1",
        "pipeline": "add_timestamp_pipeline"
      },
      "frequency": "5m",
      "sync": {
        "time": {
          "field": "order_date",
          "delay": "60s"
        }
      },
      "pivot": {
        "group_by": {
          "customer_id": {
            "terms": {
              "field": "customer_id"
            }
          }
        },
        "aggregations": {
          "max_price": {
            "max": {
              "field": "taxful_total_price"
            }
          }
        }
      },
      "description": "Maximum priced ecommerce data by customer_id in Asia",
      "settings": {},
      "retention_policy": {
        "time": {
          "field": "order_date",
          "max_age": "30d"
        }
      }
    }
  ]
}

Delete a watch

DELETE /_watcher/watch/{id}
Api key auth Basic auth Bearer auth

When the watch is removed, the document representing the watch in the .watches index is gone and it will never be run again.

Deleting a watch does not delete any watch execution records related to this watch from the watch history.

IMPORTANT: Deleting a watch must be done by using only this API. Do not delete the watch directly from the .watches index using the Elasticsearch delete document API When Elasticsearch security features are enabled, make sure no write privileges are granted to anyone for the .watches index.

Path parameters

  • id string Required

    The watch identifier.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
DELETE /_watcher/watch/{id} 
curl \
 --request DELETE 'http://api.example.com/_watcher/watch/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE _watcher/watch/my_watch`. 
{
  "found": true,
  "_id": "my_watch",
  "_version": 2
}

Start the watch service

POST /_watcher/_start
Api key auth Basic auth Bearer auth

Start the Watcher service if it is not already running.

Query parameters

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_watcher/_start 
curl \
 --request POST 'http://api.example.com/_watcher/_start' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST _watcher/_start`. 
{
  "acknowledged": true
}

Get Watcher statistics Added in 5.5.0

GET /_watcher/stats
Api key auth Basic auth Bearer auth

This API always returns basic metrics. You retrieve more metrics by using the metric parameter.

Query parameters

  • emit_stacktraces boolean

    Defines whether stack traces are generated for each watch that is running.

  • metric string | array[string]

    Defines which additional metrics are included in the response.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object Required
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • manually_stopped boolean Required
    • stats array[object] Required
      Hide stats attributes Show stats attributes object
      • current_watches array[object]

        The current executing watches metric gives insight into the watches that are currently being executed by Watcher. Additional information is shared per watch that is currently executing. This information includes the watch_id, the time its execution started and its current execution phase. To include this metric, the metric option should be set to current_watches or _all. In addition you can also specify the emit_stacktraces=true parameter, which adds stack traces for each watch that is being run. These stack traces can give you more insight into an execution of a watch.

        Hide current_watches attributes Show current_watches attributes object
      • execution_thread_pool object Required
        Hide execution_thread_pool attributes Show execution_thread_pool attributes object
        • max_size number Required

          The largest size of the execution thread pool, which indicates the largest number of concurrent running watches.

        • queue_size number Required

          The number of watches that were triggered and are currently queued.

      • queued_watches array[object]

        Watcher moderates the execution of watches such that their execution won't put too much pressure on the node and its resources. If too many watches trigger concurrently and there isn't enough capacity to run them all, some of the watches are queued, waiting for the current running watches to finish.s The queued watches metric gives insight on these queued watches.

        To include this metric, the metric option should include queued_watches or _all.

        Hide queued_watches attribute Show queued_watches attribute object
      • watch_count number Required

        The number of watches currently registered.

      • watcher_state string Required

        Values are stopped, starting, started, or stopping.

      • node_id string Required
GET /_watcher/stats 
curl \
 --request GET 'http://api.example.com/_watcher/stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _watcher/stats`. 
{
  "watcher_state": "started",  
  "watch_count": 1, 
  "execution_thread_pool": {
    "size": 1000, 
    "max_size": 1 
  }
}
A successful response from `GET _watcher/stats?metric=current_watches`. 
{
  "watcher_state": "started",
  "watch_count": 2,
  "execution_thread_pool": {
      "queue_size": 1000,
      "max_size": 20
  },
  "current_watches": [ 
      {
        "watch_id": "slow_condition", 
        "watch_record_id": "slow_condition_3-2015-05-13T07:42:32.179Z", 
        "triggered_time": "2015-05-12T11:53:51.800Z", 
        "execution_time": "2015-05-13T07:42:32.179Z", 
        "execution_phase": "condition" 
      }
  ]
}
An abbreviated response from `GET _watcher/stats/queued_watches`. 
{
  "watcher_state": "started",
  "watch_count": 10,
  "execution_thread_pool": {
      "queue_size": 1000,
      "max_size": 20
  },
  "queued_watches": [ 
        {
            "watch_id": "slow_condition4", 
            "watch_record_id": "slow_condition4_223-2015-05-21T11:59:59.811Z", 
            "triggered_time": "2015-05-21T11:59:59.811Z", 
            "execution_time": "2015-05-21T11:59:59.811Z" 
        }
  ]
}