Cool Hacks — Timer Queues Edition
So, Uber released Yet Another Task Queue — called Cherami — back in 2016. It’s distributed, scalable, durable, written in Go, etc., and is basically something they wrote so that they could get off of Celery / Redis . Which is all well and good, and I’m not here to talk about it (if you want to know more, go take a look at this post , and the repo on github ). What I did find fascinating was this extremely nifty hack to deal with Timer Queues . Timer Queues are queues where each message is associated with a delay , and the message is delivered only after that delay has elapsed. In Cherami, they use RocksDB to implement the timer queues, but the really cool part is in the way they construct the keys. Y’see, the keys contain the delivery time in the high order bits, and a sequence number in the low-order bits. This way, when you sort the keys, they automagically get sorted in order of delivery time, an...