System Design Basics

System design is the practice of deciding how the parts of a software system fit together before you build them. Instead of focusing on a single function or class, you reason about components: clients, servers, databases, caches, queues, and the networks that connect them. The central question is not "what does this line of code do?" but "how do these services cooperate to satisfy a requirement under real load?"

A useful mental model is to separate concerns into three layers. The interface layer accepts requests from users or other systems. The logic layer applies business rules and coordinates work. The storage layer persists and retrieves state. Good designs keep these layers loosely coupled so each can change independently. You also think in terms of contracts: an API is a promise about inputs, outputs, and errors, and it should stay stable even when the implementation behind it changes. This abstraction lets teams work in parallel and lets systems evolve without rewrites. Ultimately, system design is about managing complexity and making explicit, defensible choices.

A design that works for ten users often fails for ten million. At scale, the dominant costs shift from CPU instructions to network round trips, disk seeks, and coordination between machines. Latency, throughput, availability, and consistency become first-class concerns rather than afterthoughts. If a single database handles all writes, it becomes a bottleneck and a single point of failure. If every request fans out to many services, tail latency compounds and users wait.

Scalability is usually pursued in two directions. Vertical scaling means adding resources to one machine; it is simple but has hard limits and remains a single failure domain. Horizontal scaling means adding more machines; it offers elasticity but introduces partitioning, replication, and consistency problems. Reliability also matters: systems are expected to run continuously, so you plan for partial failure, retries, and graceful degradation. Cost is another force. Over-provisioning wastes money, while under-provisioning causes outages. Understanding these pressures early prevents expensive redesigns later and helps you justify each component you add.

Most large systems are built from a small set of recurring patterns. Load balancers distribute incoming traffic across identical server instances, enabling horizontal scaling and health-based routing. Stateless services are preferred because any instance can handle any request, which simplifies scaling and recovery. When state is required, it lives in a database or an external store.

Databases scale through replication and partitioning. Replication copies data to multiple nodes for read scaling and failover; partitioning (sharding) splits data by a key so no single node holds everything. Caches store frequently read results in fast memory to reduce database load, but they introduce staleness that you must bound with expiration or invalidation. Message queues decouple producers from consumers, absorbing bursts and enabling asynchronous processing. Content delivery networks push static assets close to users. Together these patterns form a vocabulary: you choose a combination that meets your latency, consistency, and availability targets, then measure and adjust.

Suppose you are designing a URL shortener. A client sends a long URL and receives a short code; later, anyone visiting the short link is redirected. The read-to-write ratio is heavily skewed toward reads, so caching is valuable.

A minimal service might look like this in pseudocode:

def shorten(long_url):
    code = base62(hash(long_url))[:7]
    store.put(code, long_url)
    return "https://ex.co/" + code
def resolve(code):
    url = cache.get(code)
    if url is None:
        url = store.get(code)
        cache.set(code, url, ttl=3600)
    return redirect(url)

The write path computes a short code, persists the mapping, and returns it. The read path checks the cache first, falls back to durable storage, and repopulates the cache. To scale reads, you add replicas of the store and a distributed cache. To avoid collisions, you can append a counter or use a unique ID generator instead of a raw hash. This small example already exposes the core trade-offs: speed versus consistency, and simplicity versus scale.

Every design choice trades one property for another. Strong consistency makes reads slower and can reduce availability during partitions; eventual consistency is faster and more available but forces your application to tolerate stale data. Caching improves latency but risks serving outdated values. Synchronous calls are easy to reason about but couple services tightly, so one slow dependency can stall everything. Asynchronous messaging decouples services but adds complexity in ordering, duplication, and debugging.

Common pitfalls include premature optimization, where teams add caches and queues before measuring a real bottleneck, and single points of failure hidden behind a load balancer. Another trap is ignoring idempotency: retries are inevitable in distributed systems, so operations like payments must be safe to repeat. Over-normalizing a database can also hurt read performance, while denormalizing too early creates update anomalies. Finally, avoid designing for hypothetical scale you do not have. Measure first, identify the actual constraint, and then apply the simplest pattern that addresses it.

Start by clarifying requirements: expected traffic, data size, latency targets, and consistency needs. Write these down as numbers, because vague goals produce vague designs. Sketch the main components and the flow of a typical request, then trace what happens when one component fails. Identify the bottleneck and choose a pattern to relieve it.

Prefer simple, well-understood building blocks. Add a cache only when reads dominate and staleness is acceptable. Introduce partitioning only when a single node cannot hold or serve the data. Use queues when work can be deferred. Define clear APIs and make operations idempotent where retries are possible. Instrument everything with metrics, logs, and traces so you can verify assumptions. Iterate: a design is a hypothesis, and production traffic is the experiment. Document the decisions and the reasons behind them, because future maintainers need the context, not just the diagram.

Learners often ask whether they need to memorize specific technologies. You do not; interviews and real work reward understanding of patterns such as caching, replication, and partitioning, which apply across databases and languages. Another common question is how to know when to scale. The honest answer is to measure: watch latency percentiles, error rates, and resource saturation, and scale when a metric crosses a threshold you defined. Finally, people wonder if system design is only for large companies. It is not. Even a small application benefits from separating concerns, avoiding single points of failure, and keeping operations idempotent, because these habits make the system easier to change as it grows.