Distributed Systems Series — Part 5.2: Scalability & Performance Why Latency at Scale Is a Different Problem Post 5.1 established what scalability means and identified Amdahl’s Law as the mathematical ceiling on parallelism. This post addresses the latency dimension of scalability — specifically why latency behaviour at scale is fundamentally different from latency at low … Read more
Distributed Systems Series — Part 4.9: Fault Tolerance & High Availability The Gap Between Designed Resilience and Actual Resilience Parts 4.1 through 4.8 have established the complete fault tolerance and high availability engineering stack. Post 4.1 defined the failure taxonomy. Posts 4.2 and 4.3 established the fault tolerance and redundancy foundations. Post 4.4 covered failure … Read more
Distributed Systems Series — Part 4.8: Fault Tolerance & High Availability Distributed Systems Without Observability Are Black Boxes Every mechanism covered in Part 4 — failure detection, redundancy, self-healing, high availability architecture, fault isolation — produces value only if engineers can observe whether it is working. A Raft cluster that is experiencing unnecessary leader elections … Read more
Distributed Systems Series — Part 4.4: Fault Tolerance & High Availability The Question That Has No Perfect Answer When a distributed node stops responding, other nodes face a question that cannot be answered with certainty: has this node failed, or is it merely slow? In a single-machine system, this question does not exist. The operating … Read more
Distributed Systems Series — Part 4.3: Fault Tolerance & High Availability Redundancy Is the Foundation, Not the Solution Post 4.1 established the taxonomy of failures — crash-stop, crash-recovery, omission, timing, gray, Byzantine, and correlated. Post 4.2 established the distinction between fault tolerance (correctness under failure) and high availability (uptime). This post addresses the structural mechanism … Read more
Distributed Systems Series — Part 4.2: Fault Tolerance & High Availability Two Goals That Sound the Same and Are Not Fault tolerance and high availability are the two most frequently conflated concepts in distributed systems engineering. Engineers use them interchangeably in architecture discussions, design documents, and system reviews. This conflation is not just imprecise — … Read more