System Design
How a system is actually designed to handle scale: the estimation and tradeoff habits, the networking and API choices, load balancing, caching, data modeling and storage at scale, consistency/availability/consensus, asynchronous messaging, reliability patterns, architecture patterns, and observability and security at scale that a production system needs — closing with worked, end-to-end case studies of the classic system-design interview problems. Sixty lessons that assume the core data structures and algorithms, and general computer science, as a starting point (see datastructures-and-algorithms and computer-science).
What Is System Design?System design as a discipline — functional vs. non-functional requirements, and how to work through an open-ended design problem without guessing.Performance vs. ScalabilityWhy a fast system isn't automatically a scalable one, and the vertical-ceiling-vs-horizontal-growth framing that explains the difference.Latency vs. ThroughputWhy latency and throughput trade off through batching, why averages hide the truth, and how one slow service can dominate a whole request.Back-of-the-Envelope EstimationFast, rough math for QPS, storage, and bandwidth using powers-of-two/ten shortcuts — a habit for catching a wrong architecture early.DNS and Request Routing at ScaleHow anycast and GeoDNS turn DNS into a coarse load balancer, and the TTL tradeoff between fast failover and cache efficiency.Choosing Transport Protocols for ServicesTCP vs UDP for service-to-service calls, how HTTP/1.1 through HTTP/3 changed multiplexing, and why keep-alive matters at scale.RPC and gRPCWhat RPC buys you over raw sockets, how gRPC and Protocol Buffers fit together, and when synchronous RPC is the wrong choice.API Architectural Styles: REST, GraphQL, and gRPCWhat REST, GraphQL, and gRPC each optimize for, and how to pick between them for a given API.Idempotent APIs and Exactly-Once SemanticsWhy retries make idempotency essential, how idempotency keys work, and why "at-least-once plus idempotent" beats chasing true exactly-once.Load Balancing AlgorithmsHow round robin, weighted round robin, least connections, consistent hashing, and power-of-two-choices pick which server gets each request.Layer 4 vs. Layer 7 Load BalancingWhat a transport-layer load balancer can see and do versus an application-layer one — and the real cost of the difference.Reverse Proxies and API GatewaysWhat a reverse proxy adds over a plain load balancer, how it differs from a forward proxy, and what an API gateway layers on top.Service DiscoveryHow services find each other at runtime — client-side vs server-side discovery, service registries, and health checks that keep them accurate.Caching Layers and CDNsWhere caching sits in a system — browser, CDN, reverse proxy, application, database — and how a CDN serves content from the edge.Caching Strategies at ScaleCache-aside, write-through, write-behind, and refresh-ahead — how data actually gets into a cache and stays fresh, and when to use each.Distributed Caching SystemsHow Redis/Memcached-style caches spread keys with consistent hashing, and how locking, coalescing, and jitter stop a cache stampede.Client and Browser CachingHow Cache-Control, ETags, and service workers let the client itself skip the network entirely — the layer closest to the user.SQL vs NoSQL: Choosing How to Model Your DataRelational vs document, key-value, column-family, and graph databases — and the real criteria for picking one over another.Modeling Data: the Entity-Relationship Model and NormalizationHow to draw entities, attributes, and relationships, then normalize a schema through the first three normal forms — and when to break that rule.Indexes, Views, and Stored ProceduresHow a database index turns a table scan into a tree lookup, what views and stored procedures are for, and the DDL/DML/DQL/DCL split of SQL itself.Choosing a Data Model: Key-Value, Document, Column-Family, and GraphNoSQL isn't one thing — key-value, document, column-family, and graph stores each suit a different access pattern.Scaling Data: Replication, Sharding, and FederationThree distinct ways to grow a database past one machine — copying it, splitting it by key, and splitting it by function.Scaling, Load Balancing, and ClusteringHorizontal vs vertical scaling, how load balancers spread traffic across servers, and what clustering actually buys you.Connection Pooling and Read/Write SplittingWhy opening a connection per request doesn't scale, how pooling fixes it, and the staleness tradeoff of routing reads to replicas.Stateless Services and Session AffinityStatelessness is what lets a load balancer treat instances as interchangeable; sticky sessions are the fallback when it's missing.ACID Transactions, BASE, and the CAP TheoremWhat a database transaction actually guarantees, how BASE relaxes it for scale, and why CAP forces a choice when the network splits.Consistency Patterns: Strong, Eventual, and the Space BetweenHow CAP's consistency-vs-availability choice becomes strong, eventual, and causal consistency, plus read-your-writes and session guarantees.Availability Patterns and FailoverActive-passive vs active-active topologies, how failover is actually detected and triggered, and the split-brain risk when it goes wrong.Measuring Availability: The NinesWhat 99.9% vs 99.99% uptime means in real downtime, how availability multiplies across a dependency chain, and SLA vs SLO.Distributed Consensus: Paxos and RaftWhy distributed systems need to agree on one value despite failures, and the core ideas behind Paxos and Raft at a conceptual level.Distributed Transactions: Two-Phase Commit and SagasWhy two-phase commit blocks when its coordinator fails, and how the saga pattern trades atomicity for availability with compensating transactions.Message Queues and Asynchronous CommunicationWhy decoupling producers from consumers matters, and how message queues implement point-to-point and publish-subscribe messaging.Pub/Sub and Event-Driven ArchitectureHow publish-subscribe differs from point-to-point queues — fan-out to many consumers, event buses, and choreography vs orchestration.Backpressure and Flow ControlWhat happens when a producer outpaces a consumer, why bounded queues and load shedding are the real fix, and why a bigger queue usually isn't.Change Data Capture for Data PipelinesStreaming a database's write-ahead log to other systems without dual writes — and the transactional outbox pattern that backs it.Circuit Breakers and RetriesThe circuit breaker state machine, retry with exponential backoff and jitter, and why naive retries turn a blip into a full outage.Rate Limiting AlgorithmsToken bucket, leaky bucket, fixed-window, and sliding-window rate limiters — how each one actually behaves at a burst boundary.Bulkheads and Graceful DegradationIsolating resource pools so one failing dependency can't exhaust everything, plus feature flags and fallbacks for degrading gracefully under partial failure.Health Checks and Chaos EngineeringLiveness vs readiness checks that feed load balancers and orchestrators, and chaos engineering as deliberately testing whether resilience assumptions hold.Performance AntipatternsRecognizable failure patterns — busy database, chatty I/O, sync-where-async-would-do, noisy neighbor, extraneous fetching — and how to fix each.Leader ElectionPractical leader-election mechanisms — the bully algorithm and lease-based election via etcd/ZooKeeper-style ephemeral nodes and TTLs.Monolith vs. Microservices: The Real Trade-offsWhat splitting a monolith into microservices actually buys and costs you, and the concrete signals that tell you a system is ready to split.CQRS and Event SourcingWhy separating reads from writes (CQRS) and treating an event log as the source of truth (event sourcing) are a natural pairing — and their real costs.Strangler Fig and Anti-Corruption LayerMigrating a legacy monolith incrementally by routing growing slices of traffic to new services, and shielding new code from the old system's data model.Sidecar, Ambassador, and Service MeshAttaching networking, observability, and security to a service without touching its code — from one sidecar to a mesh across an entire fleet.Backend for Frontend and Gateway PatternsWhy one API gateway serving every client type becomes a bottleneck, and how a tailored backend per client (BFF) fixes it.Monitoring, Logging, and AlertingMetrics, logs, and traces as observability's pillars, why structured logging matters, and what makes an alert actionable instead of noise.Distributed TracingWhy one request spanning a dozen services needs a trace ID and per-hop spans, and how a trace waterfall reveals which hop is slow.SLIs, SLOs, and Error BudgetsHow SLIs, SLOs, and SLAs turn "how reliable should this be" into a number, and how an error budget governs release velocity.DDoS Mitigation and Rate Limiting for SecurityDefending against DDoS and abusive bot traffic with scrubbing, edge shielding, IP reputation, and challenge pages beyond ordinary rate limiting.Authentication and Federated Identity at ScaleHow a distributed system centralizes authentication with an identity provider and tokens instead of every service checking passwords itself.Secrets and Config ManagementWhy secrets and per-environment config don't belong in code or images, and how centralized secrets stores and rotation reduce real risk.Case Study: Design a URL ShortenerA worked design for a bit.ly-style shortener, covering short-code generation, collision handling, and the cache that keeps redirects fast.Case Study: Design a Rate LimiterA worked design for a shared, distributed rate limiter that stays low-latency and avoids becoming the bottleneck it was built to prevent.Case Study: Design a Key-Value StoreA worked, Dynamo-style design for a distributed key-value store, covering partitioning, the replication/consistency tradeoff, and conflict resolution.Case Study: Design a Distributed CacheA worked design for a Redis/Memcached-style distributed cache, covering eviction, cross-node invalidation, and resizing without a full reshuffle.Case Study: Design a Chat SystemA worked design for a WhatsApp/Messenger-style chat system, covering real-time delivery, message ordering, and presence at scale.Case Study: Design a News Feed / Notification SystemA worked design for a social feed and notification pipeline, centered on the fan-out-on-write vs. fan-out-on-read decision.Case Study: Design a Web CrawlerA worked design for a large-scale web crawler, covering the URL frontier, dedup, per-domain politeness, and the graph's unbounded size.Case Study: Design a Search AutocompleteA worked design for a type-ahead search suggestion system, covering the trie structure, how suggestions stay fresh, and latency budget.
Design fundamentals & estimation
- What Is System Design?System design as a discipline — functional vs. non-functional requirements, and how to work through an open-ended design problem without guessing.
- Performance vs. ScalabilityWhy a fast system isn't automatically a scalable one, and the vertical-ceiling-vs-horizontal-growth framing that explains the difference.
- Latency vs. ThroughputWhy latency and throughput trade off through batching, why averages hide the truth, and how one slow service can dominate a whole request.
- Back-of-the-Envelope EstimationFast, rough math for QPS, storage, and bandwidth using powers-of-two/ten shortcuts — a habit for catching a wrong architecture early.
Networking, APIs & communication
- DNS and Request Routing at ScaleHow anycast and GeoDNS turn DNS into a coarse load balancer, and the TTL tradeoff between fast failover and cache efficiency.
- Choosing Transport Protocols for ServicesTCP vs UDP for service-to-service calls, how HTTP/1.1 through HTTP/3 changed multiplexing, and why keep-alive matters at scale.
- RPC and gRPCWhat RPC buys you over raw sockets, how gRPC and Protocol Buffers fit together, and when synchronous RPC is the wrong choice.
- API Architectural Styles: REST, GraphQL, and gRPCWhat REST, GraphQL, and gRPC each optimize for, and how to pick between them for a given API.
- Idempotent APIs and Exactly-Once SemanticsWhy retries make idempotency essential, how idempotency keys work, and why "at-least-once plus idempotent" beats chasing true exactly-once.
Load balancing & service discovery
- Load Balancing AlgorithmsHow round robin, weighted round robin, least connections, consistent hashing, and power-of-two-choices pick which server gets each request.
- Layer 4 vs. Layer 7 Load BalancingWhat a transport-layer load balancer can see and do versus an application-layer one — and the real cost of the difference.
- Reverse Proxies and API GatewaysWhat a reverse proxy adds over a plain load balancer, how it differs from a forward proxy, and what an API gateway layers on top.
- Service DiscoveryHow services find each other at runtime — client-side vs server-side discovery, service registries, and health checks that keep them accurate.
Caching at scale
- Caching Layers and CDNsWhere caching sits in a system — browser, CDN, reverse proxy, application, database — and how a CDN serves content from the edge.
- Caching Strategies at ScaleCache-aside, write-through, write-behind, and refresh-ahead — how data actually gets into a cache and stays fresh, and when to use each.
- Distributed Caching SystemsHow Redis/Memcached-style caches spread keys with consistent hashing, and how locking, coalescing, and jitter stop a cache stampede.
- Client and Browser CachingHow Cache-Control, ETags, and service workers let the client itself skip the network entirely — the layer closest to the user.
Data modeling & storage
- SQL vs NoSQL: Choosing How to Model Your DataRelational vs document, key-value, column-family, and graph databases — and the real criteria for picking one over another.
- Modeling Data: the Entity-Relationship Model and NormalizationHow to draw entities, attributes, and relationships, then normalize a schema through the first three normal forms — and when to break that rule.
- Indexes, Views, and Stored ProceduresHow a database index turns a table scan into a tree lookup, what views and stored procedures are for, and the DDL/DML/DQL/DCL split of SQL itself.
- Choosing a Data Model: Key-Value, Document, Column-Family, and GraphNoSQL isn't one thing — key-value, document, column-family, and graph stores each suit a different access pattern.
Scaling data & services
- Scaling Data: Replication, Sharding, and FederationThree distinct ways to grow a database past one machine — copying it, splitting it by key, and splitting it by function.
- Scaling, Load Balancing, and ClusteringHorizontal vs vertical scaling, how load balancers spread traffic across servers, and what clustering actually buys you.
- Connection Pooling and Read/Write SplittingWhy opening a connection per request doesn't scale, how pooling fixes it, and the staleness tradeoff of routing reads to replicas.
- Stateless Services and Session AffinityStatelessness is what lets a load balancer treat instances as interchangeable; sticky sessions are the fallback when it's missing.
Consistency, availability & consensus
- ACID Transactions, BASE, and the CAP TheoremWhat a database transaction actually guarantees, how BASE relaxes it for scale, and why CAP forces a choice when the network splits.
- Consistency Patterns: Strong, Eventual, and the Space BetweenHow CAP's consistency-vs-availability choice becomes strong, eventual, and causal consistency, plus read-your-writes and session guarantees.
- Availability Patterns and FailoverActive-passive vs active-active topologies, how failover is actually detected and triggered, and the split-brain risk when it goes wrong.
- Measuring Availability: The NinesWhat 99.9% vs 99.99% uptime means in real downtime, how availability multiplies across a dependency chain, and SLA vs SLO.
- Distributed Consensus: Paxos and RaftWhy distributed systems need to agree on one value despite failures, and the core ideas behind Paxos and Raft at a conceptual level.
- Distributed Transactions: Two-Phase Commit and SagasWhy two-phase commit blocks when its coordinator fails, and how the saga pattern trades atomicity for availability with compensating transactions.
Async systems & messaging
- Message Queues and Asynchronous CommunicationWhy decoupling producers from consumers matters, and how message queues implement point-to-point and publish-subscribe messaging.
- Pub/Sub and Event-Driven ArchitectureHow publish-subscribe differs from point-to-point queues — fan-out to many consumers, event buses, and choreography vs orchestration.
- Backpressure and Flow ControlWhat happens when a producer outpaces a consumer, why bounded queues and load shedding are the real fix, and why a bigger queue usually isn't.
- Change Data Capture for Data PipelinesStreaming a database's write-ahead log to other systems without dual writes — and the transactional outbox pattern that backs it.
Reliability & resilience patterns
- Circuit Breakers and RetriesThe circuit breaker state machine, retry with exponential backoff and jitter, and why naive retries turn a blip into a full outage.
- Rate Limiting AlgorithmsToken bucket, leaky bucket, fixed-window, and sliding-window rate limiters — how each one actually behaves at a burst boundary.
- Bulkheads and Graceful DegradationIsolating resource pools so one failing dependency can't exhaust everything, plus feature flags and fallbacks for degrading gracefully under partial failure.
- Health Checks and Chaos EngineeringLiveness vs readiness checks that feed load balancers and orchestrators, and chaos engineering as deliberately testing whether resilience assumptions hold.
- Performance AntipatternsRecognizable failure patterns — busy database, chatty I/O, sync-where-async-would-do, noisy neighbor, extraneous fetching — and how to fix each.
- Leader ElectionPractical leader-election mechanisms — the bully algorithm and lease-based election via etcd/ZooKeeper-style ephemeral nodes and TTLs.
Architecture patterns at scale
- Monolith vs. Microservices: The Real Trade-offsWhat splitting a monolith into microservices actually buys and costs you, and the concrete signals that tell you a system is ready to split.
- CQRS and Event SourcingWhy separating reads from writes (CQRS) and treating an event log as the source of truth (event sourcing) are a natural pairing — and their real costs.
- Strangler Fig and Anti-Corruption LayerMigrating a legacy monolith incrementally by routing growing slices of traffic to new services, and shielding new code from the old system's data model.
- Sidecar, Ambassador, and Service MeshAttaching networking, observability, and security to a service without touching its code — from one sidecar to a mesh across an entire fleet.
- Backend for Frontend and Gateway PatternsWhy one API gateway serving every client type becomes a bottleneck, and how a tailored backend per client (BFF) fixes it.
Observability & security at scale
- Monitoring, Logging, and AlertingMetrics, logs, and traces as observability's pillars, why structured logging matters, and what makes an alert actionable instead of noise.
- Distributed TracingWhy one request spanning a dozen services needs a trace ID and per-hop spans, and how a trace waterfall reveals which hop is slow.
- SLIs, SLOs, and Error BudgetsHow SLIs, SLOs, and SLAs turn "how reliable should this be" into a number, and how an error budget governs release velocity.
- DDoS Mitigation and Rate Limiting for SecurityDefending against DDoS and abusive bot traffic with scrubbing, edge shielding, IP reputation, and challenge pages beyond ordinary rate limiting.
- Authentication and Federated Identity at ScaleHow a distributed system centralizes authentication with an identity provider and tokens instead of every service checking passwords itself.
- Secrets and Config ManagementWhy secrets and per-environment config don't belong in code or images, and how centralized secrets stores and rotation reduce real risk.
Case studies
- Case Study: Design a URL ShortenerA worked design for a bit.ly-style shortener, covering short-code generation, collision handling, and the cache that keeps redirects fast.
- Case Study: Design a Rate LimiterA worked design for a shared, distributed rate limiter that stays low-latency and avoids becoming the bottleneck it was built to prevent.
- Case Study: Design a Key-Value StoreA worked, Dynamo-style design for a distributed key-value store, covering partitioning, the replication/consistency tradeoff, and conflict resolution.
- Case Study: Design a Distributed CacheA worked design for a Redis/Memcached-style distributed cache, covering eviction, cross-node invalidation, and resizing without a full reshuffle.
- Case Study: Design a Chat SystemA worked design for a WhatsApp/Messenger-style chat system, covering real-time delivery, message ordering, and presence at scale.
- Case Study: Design a News Feed / Notification SystemA worked design for a social feed and notification pipeline, centered on the fan-out-on-write vs. fan-out-on-read decision.
- Case Study: Design a Web CrawlerA worked design for a large-scale web crawler, covering the URL frontier, dedup, per-domain politeness, and the graph's unbounded size.
- Case Study: Design a Search AutocompleteA worked design for a type-ahead search suggestion system, covering the trie structure, how suggestions stay fresh, and latency budget.