How often have you heard people throw around the scalable word? Often used in a boastful manner to illustrate they are ready for scale. Being a technologist I treat such boasts with much skepticism; because scaling an infrastructure or enterprise is bloody difficult and near on impossible if not been properly designed from the outset.
Every deal book we see at ParkerGale they all describe their systems as a “scalable platform” – but rarely they are. Too much faith has been put in the wonders of the cloud, which many read and assume that their scaling problems are magically over.
In this article I am going to help the CEO to determine if their platform is ready for the scale they have been told it is.
Because the cloud makes it quick, easy and cheap to spin up servers/resources the assumption is made that scaling is linear. Let’s look at the root of the problem with a simple example and maths that makes it look easy on paper.
Your web site is running on a single server and has never shown any sort of strain. Every one of your 1000 customers get a fast response and never been denied their experience. Great job engineering team.
“No problem” yells the CEO, “add another server“
The business decides to double its clients – 2000 customers hitting on your site. “No problem” yells the CEO, “add another server“. Since 1 server handled 1000 customers, 2 servers can handle 2000 clients. Problem solved.
This logic is hard to argue with on face value. However the devil is in the detail as you recall the classic phrase about how 9 mothers can’t make a baby in 1 month no matter how dedicated each mother is to the process.
While many systems can have more “mothers” thrown at it, a lot of the core systems can’t scale like that. The big one that most struggle with is the good old database; the heart that pumps the data around the company. You generally want all your data in one place so it maybe an analyzed and queried. This comes at a huge cost. Often we see lots of front-end application servers all properly load balanced and designed, only to be failed by the single database serving them all.
Application servers can prove troublesome too; nothing worse than having your session data loaded on one server only to have the another server deal with your request because the load balancer decided to balance you off to another. Not insurmountable, but you have to design for this from the start.
There is an assumption that because a company is utilizing the infrastructure of a cloud provider they have outsourced the scaling problem. If I have 100 sales people doesn’t mean I am going to have x100 the amount of sales. They still need managed and properly allocated their workload. All a cloud provider does for a company is to provision resources quickly.
You can validate your scalability readiness by asking your engineering team some questions.
- What steps did they take during development to provision for scale?
- Teams will often outsource this to the underlying platform/framework they chose. For example if say that .NET/JEE takes care of it automatically; #RedFlag
- What is the weakest point of the system?
- No matter how great the system is, there is always at least one weakness. If they can’t identify any; #RedFlag
- What level of scale can the system cope with?
- Any type of scale! #RedFlag While you don’t need the scale of Facebook, they even have scaling woes.
- How much testing was done to confirm the scaling touch points?
- If they look at you, head tilted like a dog trying to understand what its master just said; #RedFlag The great follow-up question here is to ask at what point did the system break? If they couldn’t get it to break then the testing was not thorough enough.
- What about down scaling?
- This is a great question and usually puts a team into a bit of a panic. Bit like asking if anyone has tried to restore any of the backups. If they have never done this; #RedFlag Scaling back down, turning off resources and making sure you lose nothing, is actually as hard, if not harder, than continually adding to a system.
Scaling an infrastructure is a very difficult task, even if you have the luxury of green-field developing it from scratch, but it is 10x harder if you are trying to make a legacy system scale.
Keep some perspective and do not assume the problem to cope with demand is someone elses – cloud platforms will only take you so far. Development and consultancy shops talk a great game but they too can fail the 5 Quick Scale Questions.
No matter how convincing someone is, scaling an enterprise is surprisingly difficult.
alan williamson 