Load Balancer — Simple

Problem statement (interviewer prompt)

Design a layer that fronts a pool of identical backend servers. Clients send to one virtual endpoint; the layer must distribute traffic, detect and bypass unhealthy nodes within seconds, support multiple algorithms, and survive its own host failures. Aim for sub-millisecond added latency at 100k+ RPS.

A load balancer distributes traffic across a pool of backend servers, providing horizontal scale, failover and a single virtual endpoint to clients.

flowchart LR
  C1([Client A])
  C2([Client B])
  C3([Client C])
  LB{{Load Balancer<br/>VIP}}
  S1[Server 1]
  S2[Server 2]
  S3[Server 3]
  C1 --> LB
  C2 --> LB
  C3 --> LB
  LB --> S1
  LB --> S2
  LB --> S3
  S1 -.health.-> LB
  S2 -.health.-> LB
  S3 -.health.-> LB

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class C1,C2,C3 client;
    class LB edge;
    class S1,S2,S3 service;