Planning Failover and Health Checks for Backend Pools

TL;DR

Load balancing failover is not an automatic guarantee for high availability. What matters is which errors a health check detects, how quickly it reacts, and which targets remain reachable afterward. A robust failover architecture separates technical accessibility from operational functionality and clearly defines behavior for L4 and L7 traffic.

Introduction

A backend can be reachable yet unable to process requests. An open TCP port says little about whether an application can establish database connections, fully respond to requests, or meet its operational dependencies. This is where many failover concepts fail: the load balancer only detects the process failure, not the application failure.

Backend health checks must therefore match the error domain of the respective service. At the same time, failover needs clear rules: When is a target considered unavailable? Which pool is switched to? How is it prevented that a short-term error triggers a failover cascade? These questions belong to operational design and not just to incident analysis.

1. Health Checks Assess Different Availabilities

A health check can operate at various levels. For an L4 service, it typically checks whether a TCP connection can be established to an address and port. This detects an unreachable host or a non-running listener. However, for many applications, this check is insufficient: a process can accept connections even if its request processing is blocked.

L7 health checks, on the other hand, examine the behavior of an HTTP or HTTPS endpoint. Besides the status code, path, protocol, and response behavior can be relevant. A meaningful checkpoint should represent a function significant for operations but should not invoke unnecessarily deep dependencies. For example, if a complex business process is started with every check, the check itself can generate additional load.

For failover planning, error intervals, repetitions, and the return of a backend are also important. Too aggressive checks react quickly but can remove healthy targets from the pool during short network disturbances. Too generous values extend the time traffic reaches an already disturbed backend.

2. Failover Needs Defined Error and Return Rules

Failover does not merely mean sending traffic to “another backend.” First, it must be determined which targets belong to the same availability domain. A pool can contain multiple instances of the same application, or separate pools can be designated for different locations, clusters, or providers. This structure determines how much failure a system can actually tolerate.

Equally important is the return to normal operation. If a backend receives traffic immediately after a short error, an unstable service can oscillate between available and unavailable. This flapping complicates diagnosis and additionally burdens the application. A controlled resumption should only occur when the target reliably responds again.

For L4 and L7 load balancing, the consequences of existing connections also differ. New connections can be more easily distributed to healthy targets. Established sessions, however, may break or fail due to state dependencies. Failover must therefore consider whether the application operates statelessly, holds sessions externally, or binds connection states to a specific backend.

3. Failover is Part of the Edge Architecture

The ayedo Edge Cloud evaluates backend targets via health checks and can distribute traffic within backend pools or switch to available targets in case of failures. This applies in the context of its Anycast-based Layer-4 and Layer-7 load balancing. The Edge Cloud handles the public ingress and forwarding; the actual applications continue to run in the compute infrastructure.

This separation is crucial for the failover architecture. A failed Kubernetes pod , an unreachable cluster, or a disturbed provider are different failure scenarios. The Edge can control public traffic ingress and the selection of reachable backends but does not replace redundancy within the application or its data storage. A second pool does not help if both pools use the same failed database.

Due to the distributed multi-PoP architecture and the active-active principle, the Edge itself is not considered a single handover point. Anycast routing, backend health checks, and failover must still be planned together: a target can be reachable from one Edge location while another path shows errors. For operations, central statistics and clear state definitions are therefore required.

4. Error Detection Must Match Failure Behavior

A good health check does not answer the question “Is the server alive?” but “Can this target currently process this type of traffic meaningfully?” For an API endpoint, an authentication-free readiness path might be suitable. For a transactional application, it must also be checked whether critical dependencies are available. It is important to distinguish between readiness, liveness, and actual service availability.

Failover should also not blindly occur for all requests. For non-idempotent write operations, a timeout may leave it unclear whether processing has already started or completed. A retry on a second backend can then lead to double processing. The Edge can forward the traffic, but the application must clearly define repeatability, idempotency, and error responses.

For operations, the following questions should be included in the design: Which errors trigger failover? Which responses remain with the original backend? How is a pool treated as completely unhealthy? And how are wrong decisions made visible? Without these rules, technical switching occurs, but not controlled high availability.

Operational Scenario: Primary Cluster and Backup Pool

An API primarily runs in a Kubernetes cluster and additionally in a separate cluster with another provider. Both environments are connected as backend targets of an Edge configuration. An L7 check examines a dedicated readiness endpoint. This confirms that the API can accept requests and that its immediately necessary dependencies are available.

If the primary pool fails, the Edge distributes new requests to the backup pool. Existing connections are not automatically continued operationally; clients must be able to handle timeouts and retries. Before activating the design, it is also checked whether both clusters meet the same data states, configurations, and security requirements. Otherwise, the failover merely shifts the failure to another location.

FAQ

Are TCP Checks Sufficient for Backend Health Checks?

Only for the reachability of a port. They do not reliably detect whether the application can process requests. For HTTP services, a suitable L7 check is usually more meaningful.

What Happens if All Backends of a Pool Fail?

The behavior must be defined in advance, such as an error response or switching to a separate pool. Without a rule, the failure is recognizable but operationally uncontrolled.

Does Failover Prevent the Loss of Ongoing Sessions?

No. Failover primarily redistributes new connections or requests. Session states, retries, and possible double processing must be considered by the application and client.

Conclusion

Load balancing failover is an operational decision with technical, organizational, and economic consequences. Health checks must assess the actual availability of a service, while pools, return rules, and session behavior limit the risk of failure. The ayedo Edge Cloud integrates these mechanisms into an Anycast-based L4 and L7 load balancing with distributed Edge ingress. High availability, however, only arises through the interaction of Edge, compute, data storage, and robust operational processes.