Top 5 things to consider while designing micro services — Better Architecting
Micro-service design is a familiar concept which is one of the buzzing words as soon as you enter the tech industry but I believe it is worth the buzz.
Why is it so?
Most of the companies I have seen over my experience and networking spend Lakhs of dollars later to improvise their initial written monolith service to make things work better. This can be a very familiar thing to many people working at startups. If the company’s Lead Engineers had a better understanding at the start of the project of a scalable architecture then the whole base design concept could have gotten a lot better and the codes they designed would have ended up being well-adaptive for horizontal and vertical scaling as well saving a lot of dollars in the future. The whole re-architecting cost of investment would generally get saved in such a case so yes — it’s worth a read to understand these concepts.
Please find my set of blogs related to practical course of desiging micro service step by step — https://m-chetandwarkani.medium.com/
Here are the key points to give it a read end to end before you start designing microservices
1. LLD patterns:
Well, Design pattern is really a base concept to learn before you hop it into any micro-service design because it improves your thinking in terms of how to exactly write a scalable code in every step of implentation you take.
You will be able to better map your code arrangements if you give it a read on this concept. Key concepts around LLD involves learning Creational, Behavioural and Structural Design Pattern.
I have written a blog which gives insights about various Low level Design patterns.
Do give it a read — Link 1, Link 2, Link 3, Link 4
One of the other websites I love on Design pattern learning is https://refactoring.guru/
2. HLD patterns:
HLD is one of the key concepts in designing a super-scalable micro-services. After we are good with designing a scalable code using LLD, next lies importance of designing the whole infra supporting scaling on top of well designed LLD code.
Some of the various key concepts in good HLD design starts from proxy servers, Orchestration, Docker, Caching, distributed logging, queuing and choosing right framework based on your business need.
I did give a headstart to write blogs relating on the same — Link to my blog Please give it a read.
One of the amazing resource I like to read to understand better HLD is https://www.youtube.com/@gkcs
3. Data Algorithms:
After a great HLD and LLD design infra is laid out for our micro service, datas come into picture. It is very important to learn basics of algorithms which are very scalable for storing datas based on our need. Some of the important algorithm for starters includes CAP theorem, Quorum, Bloom filter, Gossip protocol, Consistent Hashing.
I have written a blog for a better understanding of this — Link to my blog
Also, you can find video recording of my blog around the same at — Youtube Link
4. Database Scaling:
After all above three steps, there comes into picture when our data starts growing enormously. In such a case, understanding on how to scale your DB becomes important. Before you dive into Database Scaling, it is very important to understand right database to select based on your business need and matching algorithm which is again explained in step 3 above.
After your selected Database starts hitting storage peak, then here are some of the strategies you can adopt for scaling your DB
- Increasing pool size
- Vertical Scaling
- CQRS
- Multi master replication
- Partitoning
- Horizontal Scaling
- Data-wise partition
Here’s my blog to give it a full read about the same
Please check step 6 of my blog — Link to understand in detail about some of the steps to scale a structure Data system.
5. Logging
Logging plays a very crucial role in any SaaS product to be reliable. If we miss out on any logs then it may lead to a vital case of data loss. Setting up logging right from scratch becomes important for the design of any micro-service. In general, I believe for a good design, we should have logs enabled for
1. Proxy:
Every request that comes in gets logged inside a file. Sometimes whole payload logging may become too much in case your traffic is in millions so logging at least request headers helps.
2. Logging at every code level:
It is vital to log and catch errors well in every piece at every function level of our code. This is a crucial deal to constantly analyse logs and improvise our system for better error catch
3. Logging at various other framework levels :
If we are using queues or any other framework such as Kubernetes or docker container as explained in HLD Design then it is super important to turn on the logs for the same.
For every third-party framework we use, adding a log there before its been processed by the same helps in case of any edge case issues hitting on our system.
I will be writing blogs soon on how to build a better logging system and you can find the same in various internet resources about the same as well.
