Scaling your backend service — System Design (part 1)
Various organization starts with a monolithic app design and eventually when the load on the server increases, the capacity to manage and handle the request becomes a problem. However, sometimes the app architecture becomes so large that converting from monolithic to microservice architecture requires intensive company development resources and time investment. In this blog, I am going to scale a backend service from a simple monolithic architecture design to a complex design that can handle a lot of concurrent requests.
I am going to split up the blog into multiple blogs where I am going to code everything in a step-by-step manner.
This blog is mainly to understand a complete high-level overview of how to achieve the scaling on web-app along with a discussion on technologies which we are going to use in our blog to scale the app.
Let’s roll!
Here’s the design of the app which we are going to build
Step 1 — Adding Kafka, Redis layer in API services:
Kafka is an amazing framework that helps us to achieve the pub-sub model in our micro-services. Basically, every request which comes in needs to be produced by Kafka producer and must be sent to a relevant topic which is then consumed by Kafka consumer listening to it and further processed. This is an important layer in micro-service to achieve scaling. A new micro-service could be simply attached to the relevant producer by making it listen to a specific topic where Kafka producer is going to send a message. Kafka message brokers are responsible for creating topics and handling all operations of incoming and outgoing requests. A topic can also have multiple partitions which is a split inside the topics which can be further targeted by the Kafka producer. Partitions are primarily useful for helping in creating a replication of message queue to handle failure mechanisms and a topic can have partitions across various message brokers as well.
High level Kafka architecture —
I will further create a blog post that will give you a complete overview of how to set up and use Kafka.
Redis:
A layer of Redis in our micro-service can help us to serve the request much faster by making frequently accessed info stored in its key-value store. It can be used as a caching mechanism that can be added to our database layer as well as the micro-service layer. It uses a key-value storage mechanism and supports various data structures. Redis has various other usages as well such as it can also be used as locks & counters, to achieve a pub-sub design in our service and also it can act as a messaging queue.
High-level Redis architecture overview —
Step 2— Dockerizing the project using docker
Dockers are used to creating a virtual environment where our project runs. Setting up Docker requires docker daemon service installation which helps in the creation and management of Docker containers, images. An image of the project needs to be created by writing the Dockerfile inside the project with necessary instructions on environment setup as well as a way to run the project inside a container. This is then useful to create an image of our project. An image can be deployed further than inside a docker container which acts as a virtual environment for our project. Multiple containers can be created where our images can be deployed separately into individual containers and in this way we can easily achieve horizontal scaling of our project.
Dockerizing your app also helps in easy environment setup as well as it’s very helpful for organization’s new employee on-boarding since installations requirement on a new machine could heavily be reduced from days of work to just a couple of hours by simple image installations into containers.
Docker Architecture —
I have made a video to understand on how to setup docker. Please give it a watch for better understanding
Step 3— Load Balancing using Nginx
After you have dockerized your project, the next step is to connect instances of your containers hosted in multiple machines with Nginx which acts like a proxy server. Nginx is helpful to load balance the request coming and re-direct to different servers or could be connected to docker containers running instance of images of our service. There are multiple load balancing techniques such as round-robin, weighted round-robin, etc., to identify servers where requests must go, and also there are multiple load balancers such as L4, l7, SSL, geo-based etc., which you can select from.
Load balancing architecture:
I have discussed more in-depth detail about load balancers, their types and how to set it up from scratch in the below blog
Also, you can find my video on the same here
Step 4— Handling orchestration via Kubernetes:
As and then when several docker containers to manage grows, we need to handle and manage deployment and additions of new containers of our projects. Kubernetes helps us in a great way to orchestrate our containers when they grow in such cases of scaling. We need to specify the description of containers in YAML file configurations and it helps us in the dynamic creation of containers as well as allocating containers to pods based on resource availability via “Kubernetes scheduler”. “Kubernetes controller manager” helps in maintaining clusters. Everything in Kubernetes is further accessible via Kubernetes CLI, API, UI services. etcd service of Kubernetes helps us to create a backing store.
High-level Kubernetes architecture —
Step 5— Elastic DB, Kibana:
Adding a layer of Elastic DB, the No SQL DB can act to store every incoming request which can be further visualized via the Kibana framework. Kibana is used to display and manage all data stored in the elastic DB in the form of UI.
Above is the high-level overview for your to understand why this layer plays a vital role and I will create an in-depth blog further if you wish to understand how exactly to do it.
Step 6 — Database scaling:
Database scaling is a huge topic in itself but it is very important to know how
to scale your database well to achieve better performance. Consider you have a simple database inside a container that is then facing a huge number of incoming requests. Following are a few of the first initial steps which can be adopted to scale our DB
Step 1. Database pooling — Increasing the number of pool connections can be done by setting the necessary attributes in the connection request. The number of pool connections is equal to the number of concurrent requests our DB can handle and it should be well noted that increasing pool size must support hardware resources of the system.
Step 2. Vertical scaling — increasing hardware load of the system. i.e., RAM size, cache size, hard disk space etc, to handle incoming requests.
Step 3. CQRS — Command Query Responsibility Segregation —
Separating reads and writes into different servers. All read requests will go through one server and write via another. This helps in achieving a better level of database access and concurrent request.
Step 4. Multi-Master replication technique — creating multiple master-slave DB machines and maintaining consistency amongst them is another step towards database scaling.
Step 5. Partitioning — creating separate DB for high-demand requests. This can simply help in scaling our DB further bypassing the high-demand request to specific DB servers.
Step 6. Horizontal scaling — via DB Sharding key
A sharding key acting as a primary key to redirect a particular request always to a specific server can introduce further layers of scaling. In this way, the load will be distributed across multiple servers well.
Step 7. Data-wise partition — Geographical-based data storage center to map the request coming from one region to a specific server only helps us in achieving more DB scaling.
