Scalability Fundamentals
Scalability is a system's ability to handle growth: more users, more data, more requests, without falling over. It is not a property of code alone. It comes from the architecture and the bottlenecks you allow.
Vertical vs Horizontal Scaling
Vertical scaling (scale up) means giving a single machine more CPU, memory, or disk. It is simple, fast to implement, and avoids distributed-systems pain. The ceiling is the largest server you can buy or rent.
Horizontal scaling (scale out) means adding more machines that share the work. The ceiling is much higher, but you need a load balancer, a way to share state, and tolerance for partial failures.
Most production systems mix both: scale up the database for a while, then scale out the application tier behind a load balancer.
Stateless Services
A stateless service stores nothing about the client between requests. Any server can handle any request. Adding capacity is as simple as starting another instance and pointing the load balancer at it.
A stateful service remembers things like sessions or in-memory caches. To scale it you must either route the same client back to the same server (sticky sessions) or move the state into a shared store like Redis.
The strong default is to keep your application servers stateless and push state into a database, cache, or queue.
Bottlenecks
The slowest part of your system caps the rest. Common bottlenecks include the database, a single shared lock, a slow downstream API, or network bandwidth. Profile under load before guessing.
The example simulation shows how peak load on any one server drops as you add servers. With 1000 requests across 8 servers, no server handles more than 125.
Try It Yourself
- Modify the simulation to use random server selection. Compare peak load to round-robin.
- Add a server that processes requests at half the speed and watch what happens.
- Identify one stateful piece of code in a recent project and sketch how to make it stateless.