Posts

Showing posts with the label Data Validation

Validate Your Migrations!

Image
Migrating data is  never  fun. It is, at best, a PITA, and at worst, a reason to go find a different job. But hey, sometimes you have no choice, and you just have to Go There, and that’s where  Scientist  comes in. Scientist ? Yeah,  Scientist . But do wait, we’ll get to it in just a moment. First, let’s just look at what you’re probably going to be doing to migrate your data between, oh, whatever it is — databases, DB versions, applications, whatever. Basically, you’ll be doing some variation fo the  Dual Write  approach  (•), which, roughly, translates to something like 1. Do all your  Writes  (Actually,  Create / Update / Delete ) to both the  Old , and the  New  targets, while serving  Reads  only from the  Old  target. 2. Start serving some of the  Reads  from the  New  target. 3. Eventually serve all the  Reads  from the  New  target. 4. Tu...