Databases

A backend server is just a running process under operating system control. Its in-memory variables disappear when the process restarts, crashes, or is redeployed. That makes memory ideal for computation, but completely unreliable for long-term storage.

Production systems require data durability. User accounts must still exist after a reboot. Orders must remain recorded after deployment. Logs must remain accessible for debugging and auditing.

A database solves this by writing structured data to persistent storage (typically SSD or disk). While memory is optimized for speed, storage is optimized for durability. The database engine manages how data is written, indexed, and recovered safely.

Without a database, your system has no long-term memory. It becomes stateless in the worst possible way — unable to survive failure.

A database is not just storage; it is a controlled execution engine for data.

In the relational model, data is stored in tables composed of rows and columns. Each row represents a record, and each column represents an attribute. A schema defines structure, data types, and constraints, preventing invalid or inconsistent data from being inserted.

The database also provides a query engine. Instead of reading raw files, applications issue structured queries to retrieve or modify specific subsets of data efficiently.

At a systems level, the database is the authoritative source of truth. It ensures durability, maintains structural integrity, and enables predictable state management.

SQL databases (relational systems) organize data into predefined tables with strict schemas. Relationships between tables are explicitly modeled, and strong transactional guarantees are standard. This makes them well-suited for systems requiring integrity and complex queries.

NoSQL databases relax rigid schemas. Many use document-based or key-value structures, allowing fields to vary between records. This flexibility can simplify development for rapidly evolving data models and can support horizontal scaling across distributed clusters more naturally.

The real difference is architectural emphasis. SQL prioritizes structure and strong consistency. NoSQL prioritizes flexibility and distributed scalability. Production systems often combine both depending on workload.

When a client sends an HTTP request, it eventually reaches your server process. Your handler executes application logic — but most meaningful operations require reading or writing persistent data.

The server sends a query to the database engine. That query may involve parsing SQL, checking permissions, locating relevant data pages, using indexes, reading from disk, and assembling a result set.

During this time, the server thread is typically waiting. In synchronous models, it blocks. In asynchronous models, the event loop awaits the result. Either way, progress depends on the database finishing its work.

This is critical: database interaction is frequently the largest contributor to latency in backend systems. Poor queries, missing indexes, network round-trips, or disk I/O delays can easily outweigh application computation time.

If your system feels slow, the database layer is often the first place to investigate.

Suppose you have a table with one million users and you search for a specific email address.

Without an index, the database may need to look at each row until it finds a match. That’s slow — especially as the table grows.

With an index on the email column, the database maintains a separate organized structure (similar to a book index). Instead of scanning everything, it uses that structure to quickly locate the correct row.

Indexes make reads much faster, but they are not free. Every time you insert, update, or delete data, the index must also be updated.

If a query is unexpectedly slow, one of the first questions to ask is: “Is there an index on the column being searched?”

Consider transferring money from Account A to Account B.

Step 1: Subtract $100 from Account A.
Step 2: Add $100 to Account B.

If the system crashes after Step 1 but before Step 2, the money disappears. That is data corruption.

A transaction prevents this. Both operations are wrapped inside a transactional boundary. The database guarantees that either:

• Both steps complete successfully, or
• Neither step is permanently applied.

This “all-or-nothing” behavior is called atomicity. It ensures the system remains logically consistent even during failures.

Transactions are essential anywhere data correctness matters — finance, inventory, authentication, and beyond.

ACID is a practical reliability contract, not theory.

Atomicity prevents partial updates. If one step fails, everything rolls back.

Consistency ensures that constraints, relationships, and rules are preserved after a transaction completes.

Isolation protects transactions from corrupting each other when multiple users operate simultaneously.

Durability guarantees that once the database confirms success, the data is written to persistent storage and will survive system failure.

Together, these properties make databases dependable foundations for real-world systems.

As usage grows, query volume increases. Eventually, a single database server cannot keep up with incoming requests.

One approach is vertical scaling — upgrading the existing machine with more CPU, memory, or faster storage. This is the simplest solution because the architecture stays the same. However, hardware upgrades have limits and costs rise quickly.

Another strategy is adding read replicas. In this model, a primary database handles writes while replicated copies serve read-only queries. This reduces pressure on the primary server, though it can introduce small delays as data propagates between nodes.

For even larger growth, systems may use sharding. Instead of storing all data on one machine, the dataset is divided across multiple database instances. For example, one shard might store users A–M and another N–Z. This increases total capacity but requires careful routing logic and operational management.

Each of these approaches improves scalability, but they also introduce trade-offs in cost, complexity, and consistency.

The database is often the most critical dependency in a backend system. When it slows down or fails, the application reflects the impact immediately.

If the database becomes unreachable, request handlers cannot complete their queries and the application begins returning 500-level errors.

Even when the database is running, slow queries, missing indexes, or locking contention can increase latency. From the outside, the server appears slow, but it is actually waiting on the database.

Under heavy load, resource limits such as connection exhaustion, replication lag, or disk saturation can introduce instability and inconsistent behavior.

Many “application issues” originate in the persistence layer. Understanding this dependency chain is essential for diagnosing production problems.