The 5 Stages of Load-Management
The load-management clock seems to perpetually stuck at Amateur Hour. Most of the implementation in this area seems to be something along the following lines 1. We’ll deal with load when we have load Which is not all that bad an idea, especially given that the other option tends to usually be “ Design something appropriate for Google ” 2. ( Now that we have load ) Put A Queue On It And spend a huge amount of time here, increasing the number of workers, optimizing their performance, “scaling horizontally”, and whatnot, till your queue-limit is the problem. Somewhere along the way, you also end up at 3. Increase the Buffer Size Which promptly leads to bufferbloat — basically, the latency that comes from having your requests spend waaaaay too much time in the queue, and all the emergent properties that come from this (the same request ends up in the queue multiple times as clients start saturate-bombing, etc.). The sad thing is that bufferbloat is quite straig...
