: Why do big sites use multiple servers instead of one server with better specifications? I read that Stack Overflow uses 10 or more servers for serving the Stack Overflow site. Different servers

@Samaraweera270

If one tries to have one machine do the work of two, some parts of the machine will need to be larger but run at the same speed, some can stay the same size but will need to run faster, and some will need to be larger and faster. The extent to which it makes sense to combine the roles of smaller machines into a larger one, or split the roles of larger machines into smaller ones, depends in large part upon what sort of scaling would apply to the most expensive parts of the machines. If the workloads of too many machines are combined into one huge colossus, then costs will be dominated by things which would need to get larger and faster to handle increased workloads. Even if the costs of such things were linear with respect to speed and size, doubling the workload would more than double the cost of a machine to process it. The fact that speed increases beyond a certain point result a (much) greater-than-linear cost increase magnify the effect.

There isn't really a fixed point where practicality compels subdivision of labor; depending upon the type of work to be done, a machine that combines the workloads of two might get by with less than twice as much memory, or running at less than twice the speed. On the other hand, the more tasks a machine is given to do, the greater the extent to which memory and speed requirements start scaling linearly with workload. The further one goes beyond that, the greater the increase in relative cost for each doubling of workload.

Vote Up Vote Down

@Gretchen104

Let's take the problem at small scale. A tiny office with one server running mail, ActiveDirectory, file share, and the web site for the company.

Hackers hit it and you have to reboot because IIS is messed up. Or Exchange needs an update and a reboot. Or Active Directory got corrupted.

Any of these isolated "one service is down" problems affects the entire server, so anything sharing on that server is going to impact them by virtue of having to reboot or anything else.

Once a real IT guy shows up and sees that server, he's going to recommend splitting them up into separate servers (and having a backup domain controller server).

It's the old adage of "don't put all your eggs in one basket"

Now that philosophy is applied to web servers. If I have just a single web server and I publish my web app (the new MyFaceLink.com) and it gets really popular, I've got new troubles. I can't take the site down to do maintenance while users are on it. And if it crashes or I get too many users, I'm hosed. Even the worlds biggest single server will get overwhelmed by 1 billion FB converts coming over.

So, load balancing comes into play, for the same "eggs in basket" reason. Spread the site across 3 servers, and if one goes down, the remaining 2 handle the capacity. If I need to do patches, I just do one at a time, and nobody notices.

At the simplest, it isn't about price of the mega-server or whether it can truly handle the load (though it can be). It's about single point of failure. Once business is busy enough, and happening 24x7 instead of for 5 users working 8-5, downtime is not acceptable. Scheduled outages are harder to schedule. So, you spread the load.

Vote Up Vote Down

@Sims2060225

Size limit. We like to pretend that a single box with multiple processors, memory chips and disks is uniform. This isn't entirely true, but it's true enough if your numbers don't get too big. There are technical limits on heat, energy, proximity etc. which means there'll always be a practical limit on how big a single server can be.
Scalability - there's a vast difference between a single server system, using shared memory for IPC and a multi server system that uses networking or clustering. However the difference between two servers and 200 is considerably smaller - if you've built a system that scales, you can scale it MUCH larger before there's a problem... and if you have, then there's actually no need for a huge single server in the first place.
Resilience - One server is a place that one admin might 'oops'. Or there's a physical problem that means service to that whole piece of tin is interrupted. (Datacentre water leak, someone crashing into a rack and knocking it over, that kind of thing). Multiple servers can be distributed within a datacentre, or better yet distributed geographically. And if you're already distributing your app, scaling on 'medium' sized machines is almost always cheaper than the same amount of CPU/memory/IO on a smaller number of bigger machines.
Updates - If I patch a server, this may make a service unstable, require a reboot, or otherwise demand some downtime. If I have 4 servers running the same thing, I can take one out of service for a while to do this. And leave it out of service if the patching/update cycle goes wrong.

Vote Up Vote Down

@Megan663

From Rear Admiral Grace Hopper:


On the building of bigger computers: "In pioneer days they used oxen for heavy pulling, and when one ox couldn't budge a log, they didn't try to grow a larger ox. We shouldn't be trying for bigger computers, but for more systems of computers."


source

Vote Up Vote Down

@Steve110

Stephen explains the major consideration to make when deciding on a systems architecture: the tradeoff in vertical and horizontal scaling. I'll add a few other considerations:


Separation of concerns: you mention multiple radically different systems: reverse proxies, DB, content servers, etc. From a maintenance and security standpoint it is clearly advantageous to keep these responsibilities spread out over different systems so they can run a different OS (version) if necessary, can be updated separately and don't impact other services when compromised.
Content delivery: this is the ultimate goal of a web server and it lends itself well to a distributd model. Systems can be duplicated and spread out geographically so that the latency of long-distance connections is minimised. It also allows for redundancy. Large websites use load balancers (yet another set of servers!) to allow for automatic failover to keep the service up at all times.


There is actually an entire server class that takes vertical scaling to another level: mainframes. They have a variety of advantages (speed, reliability) and disadvantages (cost) but on the whole they are usually used when tremendous volumes of data have to handled via Input-Output processing in what we call transaction processing (think credit card purchases, banking, election and census data). Banks for instance serve sites from vertically scaled web servers while the back-end would end up processing transactions via the mainframe.

Interestingly companies like Paypal and Visa have been moving away from mainframe towards clustered systems of thousands of horizontally scaled systems. In the rapidly evolving digital world even mainframes are hitting the horizontal scaling ceiling:


“With all the availability and performance requirements, we couldn’t keep processing payments on mainframes,


Source: Adam Banks, in ComputerWorldUK

Vote Up Vote Down

@Heady270

A single powerful server can only be upgraded so far. Once you have the most powerful server available, your site cannot grow more without splitting it between servers or making it more efficient.

There is also the cost factor. A single server that is super powerful may cost ten times as much as two servers that are half as powerful. You want to be able to buy your hardware at the price point that is cheapest and not be locked into a higher price point because it's the only thing that will work.

Uptime and reliability also come into play. With two or more servers, one can fail or be taken off-line for maintenance and the site can stay up. You can't do that with a single server.

Most large websites use load balancers and multiple servers. I used to work for TripAdvisor. They published a great article about the TripAdvisor architecture and how they make it highly scalable with multiple servers.

It is possible to run an advanced service on a single server. One example that I know of is Mailinator. The author published an article about the architecture of Mailinator. He focuses on making his code more efficient rather than buying new servers. This ends up being a limitation that dictates how his service works. It keeps mail only a few hours before the single machine deletes it to make room for more.

Upgrading a single server is known as scaling vertically. Adding more servers is known as scaling horizontally. For more information on this topic, here are some articles that compare the two:


Wikipedia: Horizontal and vertical scaling
IBM thoughts on cloud: How to explain vertical and horizontal scaling in the cloud
Internap video: vertical vs horizontal scaling

Vote Up Vote Down