Mongosv 2011 - MongoDB on Amazon EC2

Mongosv 2011 - MongoDB on Amazon EC2

• 1.
  Amazon Web Services Jared Rosoff @forjared
• 2.
  Agenda • Single node – Instance sizing – Storage configuration – OS configuration • Replica Set – Availability Zones – Disaster Recovery – Multi Data-Center • Sharded Cluster
• 3.
  INSTANCES, STORAGE ANDSINGLE NODE CONFIGURATION
• 4.
  Instance Sizing 32-bit = Don’t Use Small Arbiter / Config Big 32-bit = Don’t Use Huge
• 5.
  OS • Amazon OSnow an option • Turn off atime • Raise file descriptor limits cat >> /etc/security/limits.conf << EOF * hard nofile 65536 * soft nofile 65536 EOF • DO NOT use large VM pages • Use ext4, xfs • Use RAID – RAID10 on MongoD – RAID1 on ConfigbDB • Warning! Known problems with Ubuntu 10.04 & EBS – https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853 – https://bugzilla.kernel.org/show_bug.cgi?id=16991
• 6.
  A MongoDB DataNode This is your typical database node. Used for actual data storage. We’ll use this icon to represent this type of node throughout this prezo. • 64-bit instance mongod • More RAM == better • Run EXT4 or XFS file system • Turn off atime & diratime EBS – RAID10 • EBS volumes in RAID10
• 7.
  A MongoDB ConfigServer Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements. • 64-bit instance – Micro is fine mongoc • EBS volumes in RAID1 EBS – RAID1
• 8.
  A MongoDB Arbiter Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements. • Micro is fine mongod -- • No storage requirements arbiter • Must be separate node from rest of replica set
• 9.
  REPLICA SET DESIGNS
• 10.
  Single Region ReplicaSet Region Availability Zone A Availability Zone B Availability Zone C mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10
• 11.
  Disaster Recovery Site Region 1 Region 2 Availability Zone A Availability Zone B Availability Zone A mongod mongod mongod Hidden EBS – RAID10 EBS – RAID10 EBS – RAID10
• 12.
  Multi Data Center Region 1 Region 2 Region 3 Availability Zone A Availability Zone A Availability Zone A mongod mongod mongod Priority 1 EBS – RAID10 EBS – RAID10 EBS – RAID10
• 13.
  SHARDED CLUSTERS INEC2
• 14.
  Single Region ShardedCluster Region Availability Zone A Availability Zone B Availability Zone C mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
• 15.
  Sharded with DisasterRecovery Site Region 1 Region 2 Availability Zone A Availability Zone B Availability Zone A mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
• 16.
  Multi Region Sharded Region 1 Region 2 Region 3 Availability Zone A Availability Zone A Availability Zone A mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
• 17.
  SECURITY
• 18.
  Security Groups Component Default Ports MongoD 27017, 28017 (Status page) MongoD Shard Server 27018 MongoS 27017 ConfigDB 27019 • All nodes listen on all interfaces unless –bind_ip is specified • Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s) • Use security groups as access control lists • (e.g. allow 27017 from app servers, but not everybody)
• 19.
  Provisioning Region 1 Region 2 Group: app-servers Group: mongodb 2.2.2.2 1.1.1.1 $ ec2-create-group mongodb --description "security group for mongodb nodes” $ ec2-create-group app-servers --description "security group for app servers ## Allow mongodb nodes to talk to each other $ ec2-authorize mongodb –o mongodb –u <your ec2 account id> ## Allow app servers to talk to mongodb nodes $ ec2-authorize mongodb –o app-servers –u <your ec2 account id> ## Allow SSH access to MongoDB nodes $ ec2-authorize mongodb –p 22 ## Allow remote MongoDB nodes to talk to cluster $ ec2-authorize mongodb –s 1.1.1.1 –p 27017 $ ec2-authorize mongodb –s 2.2.2.2 –p 27017
• 20.
  Questions? • I am – Jared Rosoff ([email protected]) • Try it out! – http://www.mongodb.org/downloads – http://www.mongodb.org/display/DOCS/Amazon +EC2