Download as PDF, PPTX
Uploaded byNaomi Yamasaki
What makes me to migrate entire VPC JAWS PANKRATION 2021
The document outlines the process and considerations for migrating an entire VPC architecture to AWS, detailing the use of tools like CloudEndure and challenges faced during migration, particularly with database migration using AWS DMS. It emphasizes the importance of network design, account management, and creating cloud infrastructure as code for successful implementation. Key issues such as resource duplication and KMS key permissions are also discussed, along with lessons learned for future migrations.
More Related Content
Similar to What makes me to migrate entire VPC JAWS PANKRATION 2021
What makes me to migrate entire VPC JAWS PANKRATION 2021
- 1. What makes meto migrate entire VPC JAWS-UG Architecture / JAWS-UG Systems Administrators Naomi Yamasaki
- 2. AWS SAMURAI 2015 JAWS-UG Architecture JAWS-UG System Administrators Naomi Yamasaki Infrastructure Engineer Digital Headquarters Consumers CO-OP Sapporo @nao_spon I ♡ Route53 IAM Organizations
- 3. Why do Imigrate entire VPC?
- 4. We are goingto AWS All in!! ● 190 Systems, 650 Servers at Datacenter ● Migrate with CloudEndure Migration ● Think CloudFirst for new systems
- 5. I love AWSOrganizations ● Control accounts, service and user role ○ Control Tower ○ SCP ○ OU ○ AWS SSO ● Standardize environments and Policy ○ CloudFormation Stack Sets ○ AWS Security Hub ○ AWS Config
- 6. Account designing ● Basically,3 accounts for 1 system ○ Separate accounts for development, staging, production ○ Some of them have fewer accounts for some reason ● Network ○ Each VPCs are connected with Transit Gateway ○ AWS and Datacenter is connected with Direct Connect
- 7. Increasing AWS accounts Migratedsystems accounts: 59 New systems accounts: 67 Other accounts: 46
- 8. Why do Imigrate entire VPC?
- 9. There are 3accounts created before 2020 ● They created before we started using AWS in earnest ● There was intended to be a small start at the time ● We want to put these accounts in Organizations too.
- 10. Why we includethose accounts in AWS Organizations Not under the control of Organizations means configure these settings individual ○ Control accounts, service and user role ■ Control Tower ■ SCP ■ OU ■ AWS SSO ○ Standardize environments and Policy ■ CloudFormation Stack Sets ■ AWS Security Hub ■ AWS Config
- 11. Most effect andwanted things
- 12. Most effect andwanted things
- 13. Problems in attachingto Transit Gateway
- 14. Problems in attachingto Transit Gateway ● Duplicated network segment ● Too much big CIDR blocks for VPC
- 15. We decided tomigrate VPC to another account ● Prefer ○ Separate accounts for each environments ○ Separate frontend and backend system’s resources being together ○ Less confusing than changing the current environment. ● Not prefer ○ Minimize the VPCs CIDR blocks ○ Change current go live production environment
- 16.
- 17.
- 18. Migration schedule 1st: Stagingenvironment 2nd: Production environment 3rd: Develop environment
- 19. The most difficultpart of the migration process was... Aurora migration
- 20. Which method tochoose for DB migration? ● Prefer simple and easy way ● Make it easy to create a replication environment ● Binary log replication seemed to be a pain to configure ● DMS is looks like easy way
- 21. Which method tochoose for DB migration? ● Prefer simple and easy way ● Make it easy to create a replication environment ● Binary log replication seemed to be a pain to configure ● DMS is looks like easy way♡
- 22.
- 23. Traps in DMS ●Defaults, unique keys, comments were disappear and the number of int digits had changed ● Replication was failed on Out Of Memory in nighttime batch.
- 24. Nighttime batch withDMS to be a nightmare batch ● After the nightmare batch, failed replication is replicated little by little ● I tried scaled up maximum the DMS replication instance, but nightmare batch made me nightmare… ● Out Of Memory with dms.r5.24xlarge: 96vCPUs, 768GiB memory
- 25. We gave upon using DMS ● 1 week left to the staging environment switchover ● 3 weeks left to production environment switchover ● We don't want to take a different approach to switchover each staging and production environment ● We had no time to try Binary log replication.
- 26. I couldn't say“Let’s do it!” anymore...
- 27. How can weresolve this issue? ● The service is stopped at 2:00 to 4:00 AM due to nighttime batch, so we can take down time at that time. ● We have a time to shift for the nighttime batches. ● We decided to sharing DB snapshots and restore from ● It was the best way for us at that time
- 28. Another issue withAWS KMS key DB snapshot cannot be shared to another account when using default aws/rds KMS key to encrypt the database
- 29. How to resolveit? ● Work at the current account ○ Create Customer managed key on KMS ○ Put permissions to IAM User or Role on Key Policy ○ Create DB snapshot ○ Copy DB snapshot and choose the KMS Key ○ Share the copied DB snapshot to new account ● Work at the new account ○ Copy the shared DB snapshot and choose KMS Key as ‘aws/rds’ ○ Create Aurora cluster from copied DB snapshot
- 30. CloudFormation issues ● Therewere manually created resources ● Some resources were created with CloudFormation, but some of them manually modified after all
- 31. CloudFormation issues ● Iwant to use the same templates for each staging, production and development environments ● I had to rewrite the all of the templates for import resources into CloudFormation stack, so I re-creating everything to new.
- 32. CloudFormation vs CDK ●Need low context description when I try to accurately resource settings in detail. ● I'm more familiar with YAML.
- 33. ● Using describecommands by AWS CLI ● Mappings like a variable How did I create CloudFormation templates
- 34. ● 1 templatesfor 1 resource ● I created: ○ Staging environment: 44 templates ○ Production environment: 60 templates ● Just copy templates and change the Mappings value How many CloudFormation templates I created
- 35. We’ve done it!! ●There was no big trouble. ● Also no big problems in service delivery after switched ● Development environment migration will be postponed until the major upcoming release have done
- 36. What I learned... ●Ensure good network design. ● Don't create a VPC with too large CIDR block ● DMS will match if there is not a large amount of processing at once ● Understand what your database doing ● Choose the best migration method as my DB ● Infrastructure as Code is soooooooooooo important!
- 37. The best wayis sometimes different from the best practice
- 38. 2 another accountsare left!! Again?!