Keep It Simple — Issue #46

"How Will It Scale?"

This is the age-old question.

At most larger companies it is a great question to ask. You have tons of customers it is important to make sure your services can deliver the best experience to them without hiccups or slow downs.

However – the "big tech" mindset has become pervasive. Bleeding its way through the tech industry.

Most engineers don't understand how much traffic a single server is capable of handling.

Culture Shock

Before I started at Amazon I had a pretty simple track record in software.

Then I got recruited by Amazon.

Many companies have priority lists that look something like:

1. Grow the company
2. Deliver projects as fast as possible
3. Decouple teams to move independently fast
4. Increase the lifetime value of each customer
5. Build great software

​
Which is fair from a business perspective. Cash flow is the blood of a company.
Customers don't care about it, but without cash flow the company will die.

It also turns out when you have as many engineers as Amazon, Google, or Netflix – moving fast is hard.

Big companies seem to have a mindset where a micro-service architecture is the one architecture to rule them all.

I saw this first hand at a Startup. A recent big-tech hire recommended micro-services when our single server was handling all the traffic fine (<20% capacity on peak hours).

Why are we trying to solve problems we don't yet have.

Big tech engineers get hired by a startup that has no customers, and suggest a FAANG-sized architectures for startup-sized problems. They spend more time standing up infrastructure than delivering features to acquire new customers.

They are not being malicious, they have good intent. They are suggesting what worked for them in the past.
The difference is that big tech companies have lots of cash. Startups don't.

This can be a painful (if not fatal) blow to a startup. ​
Complexity from the start.
Engineering moves slower.
Engineering costs burn in to capital.
The company is forced to make difficult decisions.

An Absurd Case of Micro-serivices

Recently I worked on a profile service at Amazon. The legacy system was split up in to FOUR micro-service.

1. A service to store profile names
2. A service to store profile images
3. A service to store profile follows + privacy settings
4. A service that is a proxy to those top 3

What.

To top it off – these services are owned by different teams. Which means the team managing the "profile service" was actually just managing a proxy. It also means getting anything is a now a careful coordinated effort.

Part of our work was undoing the mess. We consolidated profile names, profile images, and privacy setting in to one service. In other words, we undid what was the result of peak micro-service mindset.

If you're wondering – the answer is yes – there was a service that just stored and returned strings.

What's Your Proof?

There is a big shift happening in tech right now. We are being forced to ask questions about our habits as big companies struggle to keep their velocity moving faster than a crawl.

The prevailing big-tech dogma of micro-services and serverless architecture is costing us.

literally.

Earlier this year, Amazon Prime Video did a move from micro-service to monolith that turned a lot of heads in the industry.

They had a service to monitor the quality of audio and video streams that was costing them a lot of money. Before, they relied on serverless lambdas and passing frames to S3 buckets over the internet.

It's important to remember this.
​When you are passing data between services on AWS (or any cloud platform) you are traversing the internet.

Traversing the internet is the most costly move you can make in computing. With how expensive AWS is getting you want to spend as little time executing on your accounts as possible.

So what did Prime Video do? They took this same architecture and moved it to a monolith in ECS.

Same architecture. One instance. Traversing memory instead of the internet for every step.

(did you know that RAM is faster than the internet?) 🤔

This move to a monolith reduced the costs of this process by 90%.

That's massive savings.

I'm not pretending to be better than this team. I could be victim to group think, team preferences, and other convincing factors that make us reach for a micro-service architecture when it's not necessary. Often – its just the way things have been done and teams don't think twice.

But Prime video isn't the only case of a monolith prevailing.

The entire Stack Exchange website run on 9 servers and 64GB of ram.
​https://stackexchange.com/performance​

Turns out you can get a lot done with a little.

The Takeaway

There are few main things I want readers to take away from this.

1. Just because Google or Amazon does it doesn't mean it is the best way to do it
2. Distributed systems are complicated. They are a last line of defense for complicated problems.
3. If you are building something new, and have no customers, there is no reason to think hard about scalability.

It's okay to build a monolith to start and use conventional coding wisdom to optimize your code.

Computers can handle more than we give them credit for.

At risk of getting killed for saying this – Deal with scale when it is time to scale.

As a final note check out this thread from @levelsio (a popular indie hacker) on twitter.

$35,000/month on one machine.

Worry about scale then (if it is a problem). Don't overcomplicate things when you have 0 users. And dont underestimate how much a single server can handle.

______________________________________

Thank you for reading this week's newsletter.
I appreciate all of you who read to the end.

How I can help you:

​Book A Coaching Call ☎️​
​Free Course 📚​
​Email Me​ 💌​

Until next week 👋