Scaling Microservices at Deployment

Hi, I’m DJ Spiess. The speed at which we deploy
our microservices is critical for creating a functioning system. If we can’t deploy fast,
we can’t scale. In this video we’re going to look at a few scaling issues we must to
address when deploying our microservices. When you think of a microservice, think of
a cat. A little effort, and you probably can get it to do what you want. Now imagine adding
another cat. It’s a takes a bit more effort, but you could probably manage them. Now think
of adding hundreds of cats, maybe even thousands. That’s the challenge we face with large microservice
systems. Lots little slinkies with fur, and without a plan to manage them it’s chaos. There’s many areas to discuss when scaling
your microservices, and we’ll eventually discuss all the ones I know about, but for now we’re
going to focus on one pain point at a time. In this video we’re going to address scaling
issues while deploying microservices. To succeed at cat herding, I mean, microservice architecting,
we need a controlled way to introduce new microservices into our system. Here’s the first area we need to focus. Continuous
integration. The way this works is our developer issues a pull request for a Git repository.
To keep the workflow simple, we’ll assume this work was for an approved feature or bug
fix. The pull request triggers a workflow, usually executed by something like Jenkins,
but we can insert our favorite scheduler here. The pull request then goes to a peer review.
That’s why I’d use a pull request instead of just pushing code. Pull requests make the
peer review simple. On approval the code gets built and packaged into a Docker container,
it’s tested, stored into an artifact repository, and then it gets deployed. Every step in this
process we make automatic will make our lives much better. Once the code is accepted, it
should be automatic all the way to production. So what does this buy us? Well, with an automatic
workflow we can shepard new code into our system very quickly. This is true for any
system, not just microservice architectures. Here’s why it’s critical for deploy mincroservices. Continuous Integration allows us to deploy
to production hundreds of times a day. The more automatic we make deploying to production,
the less it’s a pain point for us. That’s important because, we’re trying to deploy
hundreds of microservices. Our process should not take days, weeks, or heaven forbid months,
to deploy a release. If it takes longer than the time it takes to push a button, our deployment
process is too long and it will kill us. Push button deployments will also make it
easy for us to rollback if we need to. Just push the button to deploy the previous version.
Kill the new version instance. Done. This also allows us to forget about instances
that are going sideways. If the container has a glitch, or the machine the service is
running on is failing, kill it. Start new instances on another machine. Done. Putting our code into Docker containers makes
the act of deployment the same for every microservice every time. The action is, “run this container”.
We can specify compute requirements, but at the end of the day we don’t care where it’s
run. At some point in our system’s life, our system
administrator will come to us and say, we don’t have any more room for compute. We can’t
scale until we get more hardware, and that process takes weeks. If we don’t care where
our microservice is run, we can deploy anywhere. Running in containers allow us to do something
called cloud bursting. That means when we run out of compute in our datacenter, we can
scale up on cloud services like Amazon or Google. We’ll use something like Kubernetes
to orchestrate scaling up, scaling down, and cloud bursting. Another thing we’ll need to do to keep deployments
easy is more of an architectural issue to address. When we design our microservices,
they need to be stateless with zero dependencies. I don’t mean the data the microservice processes
cannot be in a state. Stateless means that any microservice instance can handle any request
at any time. It doesn’t need to be a particular instance.
If our sales microservice needs to talk to a particular order microservice and only that
microservice, we’d have to replace both just to replace one. This makes the two microservices,
pretty much one microservice. Keep doing that for all your microservices, and suddenly you’re
replacing the whole system everytime. That’s no good because it forces us to deploy more
than we need to. We do not want deep dependencies between microservices
either for almost the same reason. The more coupled our microservices are, the harder
it is to deploy one without coordination with the other. The more this happens in our system,
the more our system looks less like microservices and looks alot more like the monolith system
we’re trying to escape. This means we need to design our microservices to fail gracefully,
and function when other microservices fail. To make deployments truly independent, each
microservice must be able to function in a vacuum and wait until other services are available. So that’s a few things we need to be able
to scale easily. I’m sure I’ve missed a few things. Let me know what I’ve missed in the
comments below! If you liked this video, make sure you like
and share. This helps get the video in front of other viewers! New videos come out each
week, so make sure you subscribe. You don’t want to miss a video! Thanks for watching,
and I’ll see you in the next video!


I watched it in double speed – I guess my fast videos are making my ears take in audio faster. 🙂 I like the length and that you are keeping me in the loop. Are those your cats? My cat is looking at the window at a bunny right now! LOL.

Top notch quality videos, how do you not have more views?

Could you do a video on dealing with eventual consistency, as well as "NoSQL in the real world for SQL-devs"? It's killing me because I am so used to SQL and JOIN's being strongly consistent, and if I am to use a NoSQL-based store, I'd have to do application-joins and deal with eventual consistency and it just feels so damn wrong to me.

You got my sub!

Leave a Reply