Download to read offline
Uploaded byedYoda
Database Migration to AWS at scale by Sameer
The document discusses a technical meetup on MySQL migration to AWS led by Sameer Kumar, a DevOps engineer with over 10 years of experience. It covers the reasons for migrating databases to the cloud, various migration approaches, architecture for scalability and availability, and performance statistics related to the migration process. Detailed methods and tools for backup and restoration, such as mysqldump and Percona XtraBackup, are also presented along with deployment strategies like blue/green deployment.
More Related Content
![[GS네오텍] Google Kubernetes Engine](https://cdn.slidesharecdn.com/ss_thumbnails/gsngooglecloudgamingworkshopgke-190909053356-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Database Migration to AWS at scale by Sameer
![[db tech showcase Tokyo 2017] C24:Taking off to the clouds. How to use DMS in...](https://cdn.slidesharecdn.com/ss_thumbnails/howtousedmsinawsmigrations-170911061230-thumbnail.jpg?width=640&height=640&fit=bounds)
Database Migration to AWS at scale by Sameer
- 1. MySQL Migration To AWS Asdelivered in tech meetup #01 organized by www.edYoda.com on 22nd April 2018 at zekeLabs Bangalore
- 2. About me: Sameer Kumar DevOpsEngineer @ Intuit 10+ years Ex- Siemens, Ex- Paytm, Ex - Snapdeal
- 3. Why Migrate Databasesto Cloud ?
- 4. Why Migrate Databasesto Cloud ? Business Growth
- 5. Why Migrate Databasesto Cloud ? Business Growth Vertical Scaling
- 6. Why Migrate Databasesto Cloud ? Business Growth Horizontal ScalingVertical Scaling
- 7. Availability Zone 3 POD-1POD-2 POD-N Proxy Layer (Routing) Availability Zone 1 Proxy Layer (Routing) POD-1 POD-2 POD-N Availability Zone 2 Proxy Layer (Routing) POD-1 POD-2 POD-N Architecture – Scalability & Availability
- 8. Some statistics ~700K Loginper day, ~1MM users login per day during Peak seasons. 103TB of financial data hosted in 118 MySQL physical servers. Size varies from 600 GB to 2 TB Nearly Zero Down Time
- 9.
- 10. Different Approaches DifferentTime 2015 Logical Backup Restore mysqldump
- 11. Different Approaches DifferentTime 2016 DMS 2015 Logical Backup Restore mysqldump
- 12. Different Approaches DifferentTime 2017 Percona Backup 2016 DMS 2015 Logical Backup Restore mysqldump
- 13. Logical Backup Restore •mysqldump • rsync • tar On-Premise Slave EC2 with EBS RDS • MySQL Client • tar
- 14. Logical Backup Restore •mysqldump • rsync • tar On-Premise Slave EC2 with EBS RDS • MySQL Client • tar • Dump : 18 Hours for 2 TB • Sync to AWS: 180G Per Hour • Restoration: 24~30 Hours • Replication Catchup : 2 Days
- 15. Logical Backup Restore •mysqldump • rsync • tar On-Premise Slave EC2 with EBS RDS • MySQL Client • tar • Dump : 18 Hours for 2 TB • Sync to AWS: 180G Per Hour • Restoration: 24~30 Hours • Replication Catchup : 2 Days • max_allowed_packet • No MyISAM • Ignore MySQL schema • Don’t change user permission during migration
- 16. Percona XtraBackup • PerconaXtrabackup • AWS CLI • tar On-Premise Slave AWS S3 RDS
- 17. Percona XtraBackup • PerconaXtrabackup • AWS CLI • tar On-Premise Slave AWS S3 RDS • Only MySQL 5.6 • 6 TB Limit • Functions, Stored Procedures are not imported automatically • Can’t use Partial Backup
- 18. Percona XtraBackup • PerconaXtrabackup • AWS CLI • tar On-Premise Slave AWS S3 RDS • Only MySQL 5.6 • 6 TB Limit • Functions, Stored Procedures are not imported automatically • Can’t use Partial Backup • S3 Copy: 180G Per Hour • Restoration: 8 Hours for 2 TB • Replication Catchup : 2 Days
- 19. Blue/Green Deployment withDNS switch DB-1 DB-2 DB-N Availability Zone 1 Proxy Layer (Routing & Pod-picking) App-1 App-2 App-N Availability Zone 3 Proxy Layer (Routing & Pod-picking) App-1 App-2 App-N Availability Zone 2 Proxy Layer (Routing & Pod-picking) App-1 App-2 App-N Route 53 Persistence Layer - RDS Jenkins Web and Application Layer Availability Zone 1 Proxy Layer (Routing & Pod-picking) App-1 App-1 App-1 Availability Zone 3 Proxy Layer (Routing & Pod-picking) App-1 App-2 App-N Availability Zone 2 Proxy Layer (Routing & Pod-picking) App-1 App-2 App-N Web and Application Layer 1. Deploy 2. Validate 3. Switch DNS Entry
- 20.
- 21. Reference links • DMSLimitations • Migration to RDS using Percona • RDS as Slave of External MySQL
- 22.