05 - Mongodb - Backup
05 - Mongodb - Backup
MongoDB Tools
ImportExportTools
MiscellaneousTools
3
Back Up and Restore with MongoDB Tools
MongoDB Tools
mongoimport and mongoexport tools allow you to work with your data in a human-
readable JSON or CSV format.
This is useful for simple ingestion to or from a third-party system, and when you
want to backup or export a small subset of your data.
For more complex data migration tasks, you may want to write your own import
and export scripts using a client driver to interact with the database.
If you want to simply copy a database or collection from one instance to another,
consider using the copydb, clone, or cloneCollection commands, which may be
more suited to this task.
The mongo shell provides the db.copyDatabase() method.
4
Back Up and Restore with MongoDB Tools
mongoimport
5
Back Up and Restore with MongoDB Tools
mongoexport
6
Back Up and Restore with MongoDB Tools
MongoDB Tools
The mongorestore and mongodump utilities work with BSON data dumps, and are
useful for creating backups of small deployments.
For resilient and non-disruptive backups, use a file system or block-level disk
snapshot function.
mongodump excludes the content of the local database in its output.
mongodump can create a backup for an entire server, database or collection,
or can use a query to backup just part of a collection.
7
Back Up and Restore with MongoDB Tools
mongodump
8
Back Up and Restore with MongoDB Tools
mongorestore
9
mongoimport
MongoDB Import Data Tool
Back Up and Restore with MongoDB Tools
mongoimport
11
Back Up and Restore with MongoDB Tools
The mongoimport tool imports content from an JSON, CSV, or TSV export created
by mongoexport, or potentially, another third-party export tool.
Avoid using mongoimport and mongoexport for full instance production backups.
They do not reliably preserve all rich BSON data types, because JSON can only
represent a subset of the types supported by BSON.
To preserve type information, mongoexport and mongoimport uses the strict mode
representation for certain types.
The user must possess, at a minimum, the readWrite role on the database into
which they are importing data.
12
Back Up and Restore with MongoDB Tools
--verbose, -v
Increases the amount of internal reporting returned on standard output or in log
files. Increase the verbosity with the -v form by including the option multiple
times, (e.g. -vvvvv.)
--quiet
Runs the mongoimport in a quiet mode that attempts to limit the amount of output.
--version
Returns the mongoimport release number.
13
Back Up and Restore with MongoDB Tools
--port <port>
Default: 27017
Specifies the TCP port on which the MongoDB instance listens for client connections.
14
Back Up and Restore with MongoDB Tools
--type <json|csv|tsv>
Specifies the file type to import. The default format is JSON, but it’s possible to
import csv and tsv files.
--file <filename>
Specifies the location and name of a file containing the data to import. If you do not
specify a file, mongoimport reads data from standard input (e.g. “stdin”).
15
Back Up and Restore with MongoDB Tools
--stopOnError
Forces mongoimport to halt the insert operation at the first error rather than
continuing the operation despite errors.
--maintainInsertionOrder
If specified, mongoimport inserts the documents in the order of their appearance
in the input source, otherwise mongoimport may perform the insertions in an
arbitrary order.
--numInsertionWorkers <int>
Default: 1
Specifies the number of insertion workers to run concurrently.
16
Back Up and Restore with MongoDB Tools
--bypassDocumentValidation
Enables mongoimport to bypass document validation during the operation. This
lets you insert documents that do not meet the validation requirements.
17
Back Up and Restore with MongoDB Tools
--headerline
If using --type csv or --type tsv, uses the first line as field names. Otherwise,
mongoimport will import the first line as a distinct document.
18
Back Up and Restore with MongoDB Tools
--upsert
Modifies the import process to update existing objects in the database if they match an
imported object, while inserting all other objects. If you do not specify a field or fields
using the –upsertFields, mongoimport will upsert on the basis of the _id field.
--upsertFields <field1[,field2]>
Specifies a list of fields for the query portion of the upsert.
19
Back Up and Restore with MongoDB Tools
In the following example, mongoimport imports the data in the JSON data from the
contacts.json file into the collection contacts in the users database.
Normally, mongoimport restores a collection data from a backup taken with
mongoexport. Most of the arguments to mongoexport also exist for mongoimport.
20
Back Up and Restore with MongoDB Tools
In the following example, mongoimport imports data from the file mdb1-
examplenet.json into the contacts collection within the database marketing on a
remote MongoDB database with authentication enabled.
mongoimport connects to the mongod instance running on the host
mongodb1.example.net over port 37017. It authenticates with the username user
and the password pass.
21
Back Up and Restore with MongoDB Tools
In the following example, mongoimport imports the csv formatted data in the
contacts.csv file into the collection contacts in the users database on the
MongoDB instance running on the localhost port numbered 27017.
Specifying --headerline instructs mongoimport to determine the name of the fields
using the first line in the CSV file.
22
Back Up and Restore with MongoDB Tools
mongoimport uses the input file name, without the extension, as the collection name
if -c or --collection is unspecified.
Use the “--ignoreBlanks” option to ignore blank fields. For CSV and TSV imports, this
option provides the desired functionality in most cases because it avoids inserting
fields with null values into your collection.
23
mongoexport
MongoDB Export Data Tool
Back Up and Restore with MongoDB Tools
mongoexport
25
Back Up and Restore with MongoDB Tools
26
Back Up and Restore with MongoDB Tools
--verbose, -v
Increases the amount of internal reporting returned on standard output or in log
files. Increase the verbosity with the -v form by including the option multiple
times, (e.g. -vvvvv.)
--quiet
Runs the mongoexport in a quiet mode that attempts to limit the amount of output.
--version
Returns the mongoexport release number.
27
Back Up and Restore with MongoDB Tools
--port <port>
Default: 27017
Specifies the TCP port on which the MongoDB instance listens for client connections.
28
Back Up and Restore with MongoDB Tools
--type <json|csv>
Specifies the file type to export. Specify csv for CSV format or json for JSON format.
If you specify csv, then you must also use either the --fields or the --fieldFile option to
declare the fields to export from the collection.
29
Back Up and Restore with MongoDB Tools
--skip <number>
Use to control where mongoexport begins exporting documents.
--limit <number>
Specifies a maximum number of documents to include in the export.
--sort <json>
Specifies an ordering for exported results. If an index does not exist that can support the
sort operation, the results must be less than 32 megabytes.
30
Back Up and Restore with MongoDB Tools
--jsonArray
Modifies the output oto write the entire contents of the export as a single JSON array.
By default mongoexport writes data using one JSON document for every MongoDB
document.
31
Back Up and Restore with MongoDB Tools
For --csv output formats, mongoexport includes only the specified field(s), and the
specified field(s) can be a field within a sub-document.
--fieldFile <filename>
Allows you to specify in a file the field or fields to include in the export and is only valid
with the --type option with value csv. The file must have only one field per line
32
Back Up and Restore with MongoDB Tools
This example creates an export of the contacts collection in sales database from
the MongoDB instance running on the localhost port number 27017. This writes the
export to the contacts.json file in JSON format.
33
Back Up and Restore with MongoDB Tools
The following example exports the contacts collection from the marketing
database, which requires authentication.
This data resides on the MongoDB instance located on mongodb1.example.net
running on port 37017, which requires the username user and the password pass.
34
Back Up and Restore with MongoDB Tools
You can export only the results of a query by supplying a query filter with the --query
option, and limit the results to a single database using the --limit option.
For instance, this command returns all documents in the sales database’s contacts
collection that contain a field named field with a value of 1.
You must enclose the query in single quotes (e.g. ') to ensure that it does not interact
with your shell environment.
35
Back Up and Restore with MongoDB Tools
In the following example, mongoexport exports data from the contacts collection in
the users database in CSV format to the file contacts.csv.
The mongod instance that mongoexport connects to is running on the localhost port
number 27017.
When you export in CSV format, you must specify the fields in the documents to
export. The operation specifies the name and address fields to export.
36
Back Up and Restore with MongoDB Tools
For CSV exports only, you can also specify the fields in a file containing the line-
separated list of fields to export. The file must have only one field per line.
For example, you can specify the name and address fields in a file fields.txt:
name
address
Then, using the --fieldFile option, specify the fields to export with the file.
37
mongodump
MongoDB Backup Tool
Back Up and Restore with MongoDB Tools
mongodump
39
Back Up and Restore with MongoDB Tools
--verbose, -v
Increases the amount of internal reporting returned on standard output or in log
files. Increase the verbosity with the -v form by including the option multiple
times, (e.g. -vvvvv.)
--quiet
Runs the mongodump in a quiet mode that attempts to limit the amount of output.
--version
Returns the mongodump release number.
41
Back Up and Restore with MongoDB Tools
--port <port>
Default: 27017
Specifies the TCP port on which the MongoDB instance listens for client connections.
42
Back Up and Restore with MongoDB Tools
--excludeCollection <col>
Excludes the specified collection from the mongodump output. To exclude multiple
collections, specify the --excludeCollection multiple times.
--excludeCollectionsWithPrefix <col>
Excludes all collections with a specified prefix from the mongodump outputs. To specify
multiple prefixes, specify the --excludeCollectionsWithPrefix multiple times.
43
Back Up and Restore with MongoDB Tools
--gzip
Compresses the output. If mongodump outputs to the dump directory, the new feature
compresses the individual files. The files have the suffiz .gz
44
Back Up and Restore with MongoDB Tools
--queryFile <path>
Specifies the path to a file containing a JSON document as a query filter that limits the
documents included in the output of mongodump.
--dumpDbUsersAndRoles
Includes user and role definitions in the database’s dump directory when performing
mongodump on a specific database. This option apllies only when you specify a
database in the --db option.
45
Back Up and Restore with MongoDB Tools
The following operation creates a dump file that contains only the collection named
collection in the database named test. In this case the database is running on the
local interface on port 27017
46
Back Up and Restore with MongoDB Tools
The following operation dumps all collections in the test database except for users
and salaries
47
Back Up and Restore with MongoDB Tools
48
Back Up and Restore with MongoDB Tools
To output the dump to an archive file, run mongodump with the --archive option and
the archive filename. For example, the following operation creates a file
test.20150715.archive that contains the dump of the test database.
49
Back Up and Restore with MongoDB Tools
To compress the files in the output dump directory, run mongodump with the --gzip
option. For example, the following operation outputs compressed files into the default
dump directory.
To compress the archive file output by mongodump, use the --gzip option in
conjunction with the --archive option, specifying the name of the compressed file.
50
mongorestore
MongoDB Restore Tool
Back Up and Restore with MongoDB Tools
mongorestore
52
Back Up and Restore with MongoDB Tools
The mongorestore program writes data from a binary database dump created by
mongodump to a MongoDB instance.
mongorestore can create a new database or add data to an existing database.
However, mongorestore performs inserts only and does not perform updates.
If restoring documents to an existing database and collection and existing
documents have the same value _id field as the to-be-restored documents,
mongorestore will not overwrite those documents.
mongorestore recreates indexes recorded by mongodump.
53
Back Up and Restore with MongoDB Tools
--verbose, -v
Increases the amount of internal reporting returned on standard output or in log
files. Increase the verbosity with the -v form by including the option multiple
times, (e.g. -vvvvv.)
--quiet
Runs the mongorestore in a quiet mode that attempts to limit the amount of output.
--version
Returns the mongorestore release number.
54
Back Up and Restore with MongoDB Tools
--port <port>
Default: 27017
Specifies the TCP port on which the MongoDB instance listens for client connections.
55
Back Up and Restore with MongoDB Tools
--drop
Before restoring the collections from the dumped backup, drops the collections from the
target database. It does not drop collections that are not in the backup.
56
Back Up and Restore with MongoDB Tools
--dir <directory>
Specify the dump directory. You cannot specify both the --dir option and the <path>
argument. Also, you cannot use the --archive option with the --dir option.
57
Back Up and Restore with MongoDB Tools
--maintainInsertionOrder
If specified, mongorestore inserts the documents in the order of their appearance in the
input source. Otherwise, mongorestore may perform the insertions in arbitrary order.
--bypassDocumentValidation
Enables mongorestore to bypass document validation during the operation. This lets
you insert documents that do not meet the validation requirements
--gzip
Restores from compressed files or data stream created by mongodump --archive
58
Back Up and Restore with MongoDB Tools
--restoreDbUsersAndRoles
Restore user and role definitions for the given databases.
--numParallelCollections <num>
Number of collections mongorestore should restore in parallel.
--numInsertionWorkersPerCollection <num>
Specifies the number of insertion workers to run concurrently per collection. For large
imports, increasing the number of insertion workers may increase the speed of the
import
59
Back Up and Restore with MongoDB Tools
Here, mongorestore reads the database dump in the dump/ sub-directory of the
current directory, and restores only the documents in the collection named people
from the database named accounts.
mongorestore restores data to the instance running on the localhost interface on
port 27017.
60
Back Up and Restore with MongoDB Tools
61
Back Up and Restore with MongoDB Tools
To restore from an archive file, run restore with the new --archive option and the
archive filename.
For example, the following operation restores the test database from the file
test.20150715.archive.
62
Back Up and Restore with MongoDB Tools
To restore from a dump directory that contains compressed files, run mongorestore
with the new --gzip option. For example, the following operation restores the test
database from the compressed files located in the default dump directory.
To restore from a compressed archive file, run mongorestore with the --gzip option in
conjunction with the new --archive option. For example, the following operation
restores the test database from the archive file test.20150715.gz.
63
Exercises
Back Up and Restore with MongoDB Tools
65
Back Up and Restore with MongoDB Tools
Database: foursquare
mongoimport
66
Back Up and Restore with MongoDB Tools
Database: foursquare
Mongo Shell:
> show dbs
local 0.000GB
> db
foursquare
> db.createCollection(“restaurants”)
{“ok”: 1}
67
Back Up and Restore with MongoDB Tools
Database: foursquare
Command Prompt:
> mongoimport --help
68
Back Up and Restore with MongoDB Tools
Database: foursquare
Mongo Shell:
> show dbs
foursquare 0.005GB
local 0.000GB
> db.restaurants.count()
25359
> db.restaurants.findOne()
> db.restaurants.remove({})
WriteResult({ "nRemoved" : 25359 })
> db.restaurants.drop()
true
69
Back Up and Restore with MongoDB Tools
Database: twitter
mongoimport
70
Back Up and Restore with MongoDB Tools
Database: twitter
Mongo Shell:
> show dbs
foursquare 0.005GB
local 0.000GB
> db
twitter
> db.createCollection(“tweets”)
{“ok”: 1}
71
Back Up and Restore with MongoDB Tools
Database: twitter
Command Prompt:
> mongoimport --host 127.0.0.1 --port 27017
--db twitter --collection tweets
--type json --file tweets.json
--drop --stopOnError
--maintainInsertionOrder
72
Back Up and Restore with MongoDB Tools
Database: twitter
Mongo Shell:
> show dbs
foursquare 0.005GB
local 0.000GB
twitter 0.036GB
> db
twitter
> db.tweets.count()
51428
> db.tweets.findOne()
73
Back Up and Restore with MongoDB Tools
Database: digg
mongoimport
74
Back Up and Restore with MongoDB Tools
Database: imdb
mongoimport
75
Back Up and Restore with MongoDB Tools
Database: imdb
Mongo Shell:
> use imdb
switched to db imdb
> db
imdb
> db.createCollection(“movies”)
{“ok”: 1}
> db.createCollection(“oscars”)
{“ok”: 1}
> db.createCollection(“people”)
{“ok”: 1}
Database: imdb
Mongo Shell:
> db.movies.remove({})
WriteResult({ "nRemoved" : 614 })
> db.movies.drop()
true
> db.people.remove({})
WriteResult({ "nRemoved" : 2299 })
> db.people.drop()
true
> db.oscars.remove({})
WriteResult({ "nRemoved" : 492 })
> db.oscars.drop()
true
> db.dropDatabase()
{“ok”: 1}
77
Back Up and Restore with MongoDB Tools
Database: geo
mongoimport
78
Back Up and Restore with MongoDB Tools
Database: edx
mongoimport
79