A replica set in MongoDB is a group of mongod processes that maintain the same set of data. It provides redundancy and high availability and is the basis for all production deployments. By having several copies of your data across database servers, replication ensures a degree of fault tolerance, protecting against the loss of a single database server.

Maintaining copies of data in different data centers can increase data locality and availability for distributed applications. In some cases, replication can provide increased read capacity as clients are able to send read operations to different servers.

A replica set includes one primary node, which handles all write operations and logs all dataset changes in its operations log (oplog). The secondary nodes replicate the primary’s oplog and implement the operations on their own datasets. This ensures that they mirror the primary’s dataset. In the event that the primary becomes inaccessible, a qualified...

You can leverage compaction to release unneeded disk space, taken up by data and collection indexes, to the operating system. To do this, run the compact command on one of your deployment’s secondary nodes. The command has the following syntax:

db.runCommand({
   compact: <string>,
   dryRun: <boolean>,
   force: <boolean>,
   freeSpaceTargetMB: <int>,
   comment: <any>
}) 

You can specify the following compact options:

• compact: The name of the collection you want to run compaction on.
• dryRun: If set to true, returns an estimate of how much space, in bytes, compaction can reclaim from the targeted collection. If dryRun is enabled, the compact command doesn’t perform any kind of compaction and only returns the estimated value.
• force: (Optional) If enabled, forces compact to run on the primary member of your replica set.
• freeSpaceTargetMB: (Optional) Specifies the minimum amount of...

Starting in MongoDB 8.0, MongoDB uses an upgraded version of TCMalloc that uses per-CPU caches, which locally store memory for a specific CPU core and drastically reduce memory fragmentation when compared to the previously used per-thread caches. As a result, the new TCMalloc helps reduce memory fragmentation and makes your database more resilient to high-stress workloads. This reduced fragmentation can improve the read performance of your replica set, especially secondaries that serve read queries.

Platform support

All operating systems that support MongoDB 8.0 also support the updated TCMalloc, except for the following:

• RHEL 8/Oracle 8 on the PPC64LE and s390x architectures
• RHEL 9/CentOS 9 / Oracle 9 on the PPC64LE architecture
• All Windows operating systems

Enabling Transparent Huge Pages (THP)

If you’re running a self-managed MongoDB deployment on a Linux system, ensure that you enable Transparent...

Replication and sharding are often confused with each other, but they serve fundamentally different roles. Replication involves creating multiple copies of the same data across different servers. Its primary goal is to provide fault tolerance, high availability, and redundancy, ensuring your application remains operational even if a node goes down. On the other hand, sharding is a horizontal scaling method that breaks a large dataset into smaller, more manageable pieces, called shards, and distributes those shards across multiple server instances. Each shard holds only a subset of the overall data, which improves performance and allows the system to handle more data than a single server could manage.

However, replication and sharding work together in that each shard must implement replication to maintain data integrity and availability. Each shard in a sharded cluster should be configured as a replica set to protect against data loss and system outages...

MongoDB supports horizontal scaling through sharding, which involves the distribution of data across multiple servers, divided into smaller pieces known as shards. Sharding plays an essential role in managing and organizing large-scale data.

Moreover, sharding allows for the creation of distributed databases to support geographically distributed applications, enabling policies that enforce data residency within specific regions.

Why do you need sharding?

Imagine your application’s data is growing rapidly and starting to push the limits of your database. As data volume increases, you may encounter a range of issues, the most immediate being degraded performance. Queries that once returned results quickly may now take noticeably longer, slowing down your application and frustrating users with laggy or delayed responses.

Storage is another critical concern. Every system has practical limits on how much data it can store and manage efficiently. If your...

In this chapter, we covered the foundational elements of MongoDB’s architecture, defining replication and sharding, detailing key features that you can use in production, and introducing some new features in MongoDB 8.0 that can help maximize your deployment’s performance.

Regarding replication, we learned about the critical role that it plays in ensuring data availability and fault tolerance within MongoDB, including the replica set election process, the importance of the oplog, and how you can configure your replica set to enhance read scalability. Furthermore, we introduced new features, such as background compaction and the updated TCMalloc, that help manage memory usage and, in turn, increase the efficiency of your deployment.

With sharding, this chapter provided a comprehensive overview of sharding as a strategy for horizontal scaling. We learned how sharding distributes data across multiple servers to manage large datasets and high-traffic applications...