Queue-Based Load Leveling

A traffic spike sized for the front door overwhelms the back room

A flash sale can send ten times the normal request rate to a checkout service. If each request calls the inventory service directly and synchronously, the inventory service — sized for average load — falls over exactly when it matters most.

A queue as a shock absorber

The producer pushes work items onto a queue instead of calling the consumer directly, and returns immediately. The consumer pulls from the queue at whatever steady rate it can actually sustain, so bursts pile up safely in the queue rather than in the consumer's failure modes.

A simple in-memory work queue

The producer enqueues instantly regardless of burst size; the consumer drains at its own fixed pace on a timer.

class WorkQueue {
  #items = [];
  enqueue(item) { this.#items.push(item); }
  dequeue() { return this.#items.shift(); }
  get size() { return this.#items.length; }
}

const queue = new WorkQueue();

// Producer: a burst of 20 orders arrives instantly
for (let i = 1; i <= 20; i++) queue.enqueue(`order-${i}`);
console.log(`queued: ${queue.size}`);

// Consumer: processes at a steady, sustainable rate
function drain() {
  const item = queue.dequeue();
  if (!item) return;
  console.log(`processing ${item}`);
  setTimeout(drain, 200); // fixed pace, independent of arrival rate
}
drain();

Trade-offs

• Decouples arrival rate from processing rate — The consumer never sees a spike bigger than its own steady throughput.
• Absorbs failure too — If the consumer goes down briefly, work waits safely in the queue instead of being dropped.
• Added latency and a new component — Work isn't processed instantly anymore, and the queue itself needs monitoring and capacity planning.