How to Handle 160 Million Users a Month? With K8s and Docker!

Highly Available SaaS Infrastructure for Over 2 Billion Requests Per Month

In today’s digital world, resilience and scalability are indispensable features of any successful Software-as-a-Service (SaaS) platform. Prime Insights, a leading provider of data analytics solutions, recognized this need early on and sought an advanced infrastructure to efficiently manage its rapidly growing number of user requests. This is where ayedo comes into play – our experts developed and operate a highly available, geo-redundant SaaS infrastructure that meets the demands of Prime Insights.

The Challenge: Exponential Growth and High Availability

Prime Insights is experiencing exponential growth in its user base and faced the challenge of accommodating this growth with appropriate infrastructure. The existing systems reached their limits, and it was clear that a new, robust infrastructure was required. The central requirement was not only scalability but especially high availability and geo-redundancy to ensure that the platform remains available even during regional outages and peak times.

The first iterations of the new hosting infrastructure were implemented with Docker Swarm, which quickly proved to be too inflexible.

The Solution: Kubernetes, HAProxy, KeyDB, MariaDB, and Maxscale

Ultimately, we decided to use Kubernetes, a proven container orchestration platform, to ensure the desired scalability and resilience. With Kubernetes, applications can be seamlessly scaled and automatically restored if needed, providing the perfect foundation for Prime Insights’ demanding environment.

As the core of the database architecture, we chose MariaDB, a powerful relational database management system. MariaDB not only offers excellent performance and flexibility but is also well-suited for handling large volumes of data. By using Maxscale, a proxy server for MariaDB, we implemented an additional layer of load balancing and resilience that can automatically adjust the topology of the MariaDB cluster. This allows a new primary node to be determined automatically, and all replicas to be adjusted in the event of a primary node failure.

To comprehensively monitor the PHP-based backend systems and seamlessly track user requests as well as the load of FPM workers, special dashboards and alert rules were developed using VictoriaMetrics and Grafana. This ensures that no user requests fail unnoticed due to resource limits.

To protect the systems from damage and ensure maximum availability for the end-user, we route ingress traffic through a combination of Cloudflare as an edge proxy and HAProxy as a central load balancer within the customer-owned infrastructure. This not only ensures maximum security but also zero-downtime updates and deployments.

KeyDB is used as a replacement for classic Redis. KeyDB brings many of the Redis Enterprise features to the community edition, including active-active clustering of KeyDB instances. KeyDB is better suited for the very CPU-intensive workloads of Prime Insights than Redis due to its multi-threading capabilities and clusterability and has also impressed with its high reliability and excellent debugging tools.

Implementing a Geo-Redundant Infrastructure

To meet the requirement of geo-redundancy, we involved multiple data centers at different geographic locations. These data centers are configured to work seamlessly together, ensuring that data is always synchronized through replication. If one data center fails, another immediately takes over the requests without interruption, ensuring data integrity and availability at all times. All locations operate in an active-active setup, so no unnecessary resources are wasted.

Result: Higher Reliability and Customer Satisfaction

Since the implementation of this solution, Prime Insights has seen a significant improvement in the overall performance and reliability of its platform. The infrastructure now effortlessly handles over 2 billion requests per month, and downtime has been reduced to nearly zero. Prime Insights’ customers benefit from a smooth and always-available service, leading to a significant increase in customer satisfaction.

Ongoing Optimization and Support

The partnership between ayedo and Prime Insights does not end with the successful completion of the implementation. Our team continuously monitors the infrastructure and performs regular maintenance to ensure the platform always functions optimally. Additionally, we work closely with Prime Insights as part of our Priority Support to identify further opportunities for optimization and adaptation to future requirements.

Conclusion: A Success Story of Digital Transformation

Through the successful integration of Kubernetes, KeyDB, HAProxy, MariaDB, and Maxscale, ayedo has created a groundbreaking, highly available, and scalable SaaS infrastructure for Prime Insights. This collaboration is a prime example of successful digital transformation and demonstrates how advanced technology companies like ayedo can offer innovative solutions to meet the growing demands of modern businesses. Prime Insights is now well-equipped to continue growing and providing its customers with top-notch data analytics services.