Return Stored Source Fields

If you configured the storedSource option in the index definition for a collection, MongoDB Search stores the specified fields on mongot. You can use the returnStoredSource boolean option in your MongoDB Search queries to retrieve only those fields, rather than fetching full documents from the collection.

By default, MongoDB Search performs full document lookup implicitly on the backend database after MongoDB Search matches documents for the query. This lookup could significantly degrade performance for subsequent aggregation pipeline stages that fetch all matched dataset from $search stage (such as $sort, $group) or filter a large part of it (such as $match, $skip). If you configured the storedSource option, you can use the returnStoredSource option to retrieve only parts of the documents directly stored on MongoDB Search and avoid full document lookup on the database. This allows you to perform most database-side filtering operations on documents with a minimum number of fields. You can then retrieve all the fields from the documents at a later stage in the pipeline using $lookup.

Note

returnStoredSource is only available on Atlas clusters running MongoDB 7.0 and later.
For sharded collections, $lookup is only available on Atlas clusters running MongoDB 5.1 and later.

Syntax

returnStoredSource has the following syntax in your queries:

{
  $search: {
    "<operator>": {
      <operator-specification>
    },
    "returnStoredSource": true | false // optional, defaults to "false"
  }
}

To learn more about query syntax, see $search.

Behavior

The returnStoredSource boolean option specifies whether MongoDB Search must perform a full document lookup on the database or return the stored fields directly from MongoDB Search. You can use the returnStoredSource option only if your index definition includes the configuration for storing fields on MongoDB Search. To learn more about storing fields on MongoDB Search, see Index Reference and Define Stored Source Fields in Your MongoDB Search Index.

You can set one of the following values for the returnStoredSource option:

true - to return only stored source fields directly from MongoDB Search
false - to do an implicit full document lookup on the backend database (default)

If you run MongoDB Search queries with the returnStoredSource boolean option set to true:

MongoDB Search returns an empty document if the document doesn't include the fields configured for storing.
MongoDB Search returns errors if the index definition doesn't include the Stored Source configuration.
MongoDB Search might return stale data due to a replication lag.
MongoDB Search might return duplicate data on sharded clusters.

If you perform a high volume and rate of data insert and update operations for your collection on the backend database, MongoDB Search might return stale data because the data stored on mongot might not be current due to a replication lag. You can view the approximate number of milliseconds that MongoDB Search is behind in replicating changes from the oplog of mongod in the Atlas UI. To learn more, see Review MongoDB Search Metrics.

If there are orphaned documents during chunk migration, MongoDB Search might return duplicate documents for queries against sharded cluster.

Sample Use

If your $search stage discards a lot of the results and you need to perform implicit document lookup on your database, we recommend using the returnStoredSource option. You can store fields required for sorting or filtering and use returnStoredSource option at query time to perform the following actions:

Intermediate operations on partial documents returned by MongoDB Search
$lookup at the end of the pipeline if full documents are needed.

Important

For efficiency, configure only a minimum number of fields for storage on MongoDB Search. Use this option if your documents are large enough to cause issues during lookup.

Examples

The examples in this section use the sample_mflix.movies collection. The examples show how to do a sort or match on the documents that MongoDB Search returns after the $search stage and then lookup documents on the database.

Create the index using the following index definition. The index definition for the collection specifies the following fields:
- Index title field
- Store year and title fields
```
{
  "mappings": {
    "fields": {
      "title": {
        "type": "string"
      }
    }
  },
  "storedSource": {
    "include": [
      "year",
      "title"
    ]
  }
}
```

Run the following query against the collection:

db.movies.aggregate([
  {
    // search and output documents
    $search: {
      "text": {
        "query": "baseball",
        "path": "title"
      },
      "returnStoredSource": true // return stored fields only
    }
  },
  // fetch all matched dataset from $search stage and sort it
  {
    $sort: {"year": 1, "title": 1}
  },
  // discard everything except top 10 results
  {
    $limit: 10
  },
  // perform full document lookup for top 10 documents only
  {
    $lookup: {
      from: "movies", localField: "_id", foreignField: "_id", as: "document"
    }
  }
])

The query returns the following documents:

1 [
2   {
3     _id: ObjectId("573a1399f29313caabced370"),
4     title: 'Mr. Baseball',
5     year: 1992,
6     document: [
7       { ... } // full document returned by $lookup
8     ]
9   },
10   {
11     _id: ObjectId("573a1399f29313caabcee1aa"),
12     title: 'Baseball',
13     year: 1994,
14     document: [
15       { ... } // full document returned by $lookup
16     ]
17   }
18 ]

Create the index using the following index definition. The index definition for the collection specifies the following fields:
- Index title field
- Store imdb.rating and imdb.votes fields
```
{
  "mappings": {
    "fields": {
      "title": {
        "type": "string"
      }
    }
  },
  "storedSource": {
    "include": [
      "imdb.rating",
      "imdb.votes"
    ]
  }
}
```

Run the following query against the collection:

db.movies.aggregate([
  {
    // search and output documents
    $search: {
      "text": {
        "query": "baseball",
        "path": "title"
      },
      "returnStoredSource": true // return stored fields only
    }
  },
  // filter dataset from $search stage using $match
  {
    $match: {$or: [ { "imdb.rating": { $gt: 8.2 } }, { "imdb.votes": { $gte: 4500 } } ]}
  },
  // perform full document lookup for matched documents only
  {
    $lookup: {
      from: "movies", localField: "_id", foreignField: "_id", as: "document"
    }
  }
])

The query returns the following documents:

1 [
2   {
3     _id: ObjectId("573a1399f29313caabcee1aa"),
4     imdb: { rating: 9.1, votes: 2460 },
5     document: [
6       { ... } // full document returned by $lookup
7     ]
8   },
9   {
10     _id: ObjectId("573a1399f29313caabced370"),
11     imdb: { rating: 5.8, votes: 7617 },
12     document: [
13       { ... } // full document returned by $lookup
14     ]
15   }
16 ]

Back

concurrent

Transform Documents & Filter Collections

1	[
2	{
3	_id: ObjectId("573a1399f29313caabced370"),
4	title: 'Mr. Baseball',
5	year: 1992,
6	document: [
7	{ ... } // full document returned by $lookup
8	]
9	},
10	{
11	_id: ObjectId("573a1399f29313caabcee1aa"),
12	title: 'Baseball',
13	year: 1994,
14	document: [
15	{ ... } // full document returned by $lookup
16	]
17	}
18	]

1	[
2	{
3	_id: ObjectId("573a1399f29313caabcee1aa"),
4	imdb: { rating: 9.1, votes: 2460 },
5	document: [
6	{ ... } // full document returned by $lookup
7	]
8	},
9	{
10	_id: ObjectId("573a1399f29313caabced370"),
11	imdb: { rating: 5.8, votes: 7617 },
12	document: [
13	{ ... } // full document returned by $lookup
14	]
15	}
16	]