System design: a reading path from components to scalable architecture
This curriculum takes an intermediate engineer from solid distributed-systems intuition to production-grade architecture thinking across four tightly sequenced stages. Each stage builds on the vocabulary and mental models of the previous one, moving from foundational principles → real-world patterns → deep internals → interview-ready synthesis.
Foundations & Mental Models
IntermediateEstablish the core vocabulary of distributed systems — reliability, scalability, consistency, and trade-offs — so every later concept has a firm conceptual home.
▸ Study plan for this stage
Pace: 8–10 weeks, ~40–50 pages/day (DDIA: 5–6 weeks, ~40 pages/day; Art of Scalability: 3–4 weeks, ~50 pages/day)
- Reliability, availability, and fault tolerance: how systems degrade gracefully under failure (DDIA Chapters 1–2)
- Scalability dimensions: load parameters, performance metrics, and latency vs. throughput trade-offs (DDIA Chapter 1, Art of Scalability Chapters 1–2)
- Data models and query languages: relational, document, graph models and their consistency implications (DDIA Chapter 2)
- Storage engines and indexing: how databases physically organize data to balance read/write performance (DDIA Chapter 3)
- Replication strategies: leader-based, multi-leader, and leaderless replication with consistency guarantees (DDIA Chapter 5)
- Partitioning and sharding: distributing data across nodes and handling skew, hot spots, and rebalancing (DDIA Chapter 6)
- Consistency models: strong, eventual, causal, and read-your-writes consistency in distributed systems (DDIA Chapter 7)
- Scalability patterns and organizational trade-offs: vertical vs. horizontal scaling, cost, and complexity (Art of Scalability Chapters 3–5)
- What is the difference between reliability, availability, and fault tolerance, and why can't a system guarantee all three simultaneously?
- How do latency and throughput relate to scalability, and what load parameters matter most for your system?
- What are the trade-offs between relational, document, and graph data models in terms of consistency and query flexibility?
- How do leader-based replication, multi-leader replication, and leaderless replication differ in handling consistency and availability?
- What is the difference between strong consistency, eventual consistency, and causal consistency, and when would you choose each?
- How do partitioning strategies (range-based, hash-based, directory-based) affect hot spots, rebalancing, and query performance?
- What are the key differences between vertical and horizontal scaling, and what organizational and technical challenges does each introduce?
- Design a simple key-value store with replication: sketch how you'd implement leader-based replication with failover, and document the consistency guarantees you can offer.
- Analyze a real system you use (e.g., a social media platform, payment system, or cache): identify its likely data model, replication strategy, and partitioning approach, then list the failure modes it might face.
- Build a small distributed system simulator (in Python, Go, or your language of choice) that demonstrates the CAP theorem: show how adding a network partition forces you to choose between consistency and availability.
- Create a load-testing scenario for a hypothetical e-commerce database: define load parameters (concurrent users, read/write ratio, query patterns), measure latency and throughput, and propose scaling strategies.
- Compare two data models (e.g., relational vs. document) for a specific use case (e.g., user profiles, product catalogs): document the consistency, query, and scalability trade-offs of each.
- Design a sharding strategy for a growing user table: choose a partitioning key, estimate hot spots, and plan a rebalancing approach when a new shard is added.
Next up: This stage builds the shared vocabulary and mental models—reliability, scalability, consistency, and trade-offs—that the next stage will apply to specific architectural patterns (caching, messaging, service communication) and real-world system design problems.
The single most important book for this curriculum. It builds rigorous intuition around databases, replication, partitioning, consistency, and distributed transactions — the backbone of every large-scale system discussion.
Introduces the Scale Cube and organizational/process thinking around scaling, complementing Kleppmann's technical depth with architectural decision-making frameworks used by real engineering teams.
System Design Patterns in Practice
IntermediateLearn the canonical building blocks — load balancers, caches, queues, CDNs, API gateways — and how they are composed into real production architectures.
▸ Study plan for this stage
Pace: 8–10 weeks, ~40–50 pages/day (with 2–3 days per week for exercises and architecture reviews)
- Load balancing strategies (round-robin, least connections, consistent hashing) and their trade-offs in distributed systems
- Caching layers (in-memory caches like Redis/Memcached) and cache invalidation patterns (TTL, LRU, write-through, write-behind)
- Message queues and asynchronous processing for decoupling services and handling burst traffic
- Content Delivery Networks (CDNs) for geographic distribution and reducing latency
- API gateways as the entry point for routing, rate limiting, authentication, and request transformation
- Composing patterns into cohesive architectures: how load balancers, caches, queues, and CDNs work together in production
- Scaling databases and handling data consistency in distributed systems
- Monitoring, logging, and observability as essential components of production systems
- What are the main load balancing algorithms, and when would you use consistent hashing versus round-robin?
- How do different cache invalidation strategies (TTL, LRU, write-through, write-behind) affect system performance and consistency?
- Why are message queues essential for scalability, and what problems do they solve in distributed architectures?
- How does a CDN reduce latency and improve availability, and what are the trade-offs of using one?
- What responsibilities does an API gateway handle, and how does it fit into a larger system architecture?
- How would you design a scalable system that combines load balancing, caching, queues, and CDNs for a high-traffic application?
- Design a load balancing strategy for a microservices architecture with 3–5 services; justify your choice of algorithm and explain how you'd handle sticky sessions if needed
- Implement a simple in-memory cache (or use Redis) with TTL and LRU eviction; measure cache hit rates under different workloads
- Set up a message queue (RabbitMQ, Kafka, or SQS) and build a producer-consumer system that decouples a web service from a background job processor
- Analyze a real CDN configuration (e.g., CloudFront, Cloudflare) for a website you use; document cache headers, origin behavior, and edge locations
- Build a mock API gateway that handles request routing, rate limiting (token bucket algorithm), and basic authentication; test it under load
- Design an end-to-end architecture for a social media feed service: include load balancing, caching strategy, queue for notifications, and CDN for media; document trade-offs
Next up: This stage equips you with the practical vocabulary and mental models of production system components; the next stage will deepen your ability to reason about trade-offs, failure modes, and optimization strategies when these patterns are pushed to extreme scale.
Provides concrete, worked examples of designing real systems (URL shorteners, news feeds, rate limiters) that make abstract patterns tangible; read first in this stage to anchor the vocabulary.
Bridges theory and practice by walking through caching strategies, database scaling, and asynchronous processing in a pragmatic, engineer-to-engineer voice that reinforces the patterns from the Xu books.
Deep Internals — Databases, Queues & Reliability
ExpertUnderstand how the components you've been using actually work under the hood — storage engines, consensus protocols, message brokers, and failure modes — enabling principled trade-off decisions.
▸ Study plan for this stage
Pace: 8–10 weeks, ~40–50 pages/day (alternating between Database Internals and Release It! every 2–3 weeks to balance theory with practical patterns)
- Storage engine architecture: B-trees, LSM trees, and how data is physically organized on disk for read/write performance trade-offs
- Consensus protocols (Raft, Paxos) and their role in distributed databases for achieving consistency and fault tolerance
- Write-ahead logging (WAL), crash recovery, and durability guarantees (ACID properties) in practice
- Message broker internals: partitioning, replication, ordering guarantees, and exactly-once semantics
- Failure modes and cascading failures: timeouts, resource exhaustion, slow clients, and circuit breakers
- Stability patterns: bulkheads, timeouts, retries with backoff, and graceful degradation to prevent system collapse
- Observability and instrumentation: logging, metrics, and tracing to detect and diagnose failures in production
- Trade-offs between consistency, availability, and partition tolerance (CAP theorem) and how they manifest in real systems
- How do B-trees and LSM trees differ in their write and read performance characteristics, and when would you choose one over the other?
- What is write-ahead logging and why is it essential for crash recovery and durability?
- Explain how a consensus protocol like Raft ensures that a distributed database maintains consistency across replicas even when nodes fail.
- What are the key differences between at-most-once, at-least-once, and exactly-once message delivery semantics, and what are the trade-offs?
- Describe a cascading failure scenario: how can a slow database client cause an entire system to collapse, and what patterns prevent this?
- How do circuit breakers, bulkheads, and timeouts work together to improve system stability and prevent resource exhaustion?
- What instrumentation (logging, metrics, tracing) would you add to a distributed system to detect and diagnose failures in production?
- Read Database Internals chapters 1–4 on storage engines; sketch out the on-disk layout of a B-tree and an LSM tree, then write pseudocode for a point lookup and a range scan in each.
- Implement a simple write-ahead log (WAL) in your language of choice: write records, simulate a crash, and verify recovery restores the correct state.
- Study a real consensus protocol (Raft is recommended in Database Internals); trace through a leader election and log replication scenario step-by-step on paper.
- Deploy a message broker (Kafka or RabbitMQ) locally; produce and consume messages with different delivery guarantees; observe partition behavior under network delays.
- Read Release It! chapters on stability patterns; design a failure scenario (e.g., slow downstream service) and implement circuit breakers and bulkheads in a toy service.
- Instrument a multi-service application with structured logging, metrics (latency, error rates), and distributed tracing; simulate a failure and verify you can diagnose the root cause.
- Conduct a chaos engineering exercise: kill a database replica mid-transaction, introduce network latency, or exhaust a connection pool; observe and document system behavior.
Next up: This stage equips you with the mental models and failure-handling patterns needed to design resilient, observable systems; the next stage will apply these principles to building end-to-end architectures that scale and degrade gracefully under real-world constraints.
Dives into B-trees, LSM trees, write-ahead logs, and distributed consensus (Raft, Paxos) — the machinery inside every database you'll choose or design around. Read after Kleppmann to go one level deeper.
Teaches stability patterns (circuit breakers, bulkheads, timeouts, back-pressure) through post-mortem-style case studies, making reliability a first-class design concern rather than an afterthought.
Architecture Thinking & Synthesis
ExpertSynthesize everything into coherent, defensible architectural decisions — evaluating trade-offs, communicating designs to stakeholders, and thinking like a staff-level or principal engineer.
▸ Study plan for this stage
Pace: 8–10 weeks, ~40–50 pages/day (mix of dense architecture theory and practical microservices patterns)
- Architectural thinking: defining fitness functions, making explicit trade-offs, and evaluating architectural decisions against business and technical constraints
- The four dimensions of software architecture (structure, architecture characteristics, architecture decisions, design principles) and how they interact
- Monolithic vs. distributed architecture patterns: when each is appropriate and the hidden costs of distribution
- Microservices decomposition strategies: domain-driven design, service boundaries, and avoiding the distributed monolith anti-pattern
- Architecture characteristics (scalability, resilience, maintainability, security) and how to measure and communicate them to stakeholders
- Deployment, monitoring, and operational complexity: the true cost of microservices and when the trade-off is worth it
- Communication and stakeholder alignment: presenting architectural decisions with clear rationale, trade-offs, and risk mitigation
- What are the four dimensions of software architecture, and how do they inform an architectural decision?
- How would you evaluate whether a monolithic or microservices architecture is the right choice for a given business context, and what are the hidden costs of each?
- What is a fitness function in architecture, and how would you define one for a system you're designing?
- How do you identify service boundaries in a microservices system, and what happens when boundaries are poorly chosen?
- What are the key operational and organizational challenges of microservices, and how do you mitigate them?
- How would you present an architectural decision to a non-technical stakeholder, including trade-offs and risks?
- Read and annotate Part I of 'Fundamentals' (chapters 1–5) focusing on the four dimensions; create a one-page summary of how each dimension applies to a system you know well
- Work through a real or hypothetical system: define 3–5 fitness functions for it, then evaluate a monolithic vs. microservices approach against those functions
- Map a moderately complex domain (e.g., an e-commerce platform or SaaS product) using domain-driven design principles; identify candidate service boundaries and justify each boundary
- Create a detailed architectural decision record (ADR) for a significant architectural choice: include problem statement, options considered, trade-offs, and rationale
- Design a microservices deployment and monitoring strategy for a system with 8–12 services; document operational complexity, failure modes, and mitigation strategies
- Conduct a mock architecture review: present a design to peers, defend trade-offs, and respond to critical questions about scalability, resilience, and maintainability
Next up: This stage equips you with the frameworks and communication skills to make and defend architectural decisions at scale; the next stage will likely focus on specialized domains (event-driven systems, data-intensive architectures, or organizational scaling) where you apply these principles to specific technical challenges.
Provides a structured vocabulary for architecture styles (event-driven, microservices, layered, space-based) and teaches how to evaluate and communicate trade-offs — the meta-skill that ties all prior learning together.
The definitive guide to decomposing large systems into services, covering service boundaries, inter-service communication, data ownership, and operational concerns — the capstone for real-world large-scale design.
Discussion
Keep reading
Paths that share books, cover the same subject, or open a related topic.
