New Approaches in AI Management: The Gateway API Inference Extension
Modern generative AI and large language models (LLMs) present unique traffic management challenges …
We are excited to announce the general availability of Gateway API v1.3.0! Released on April 24, 2025, this version brings exciting new features.
Version 1.3.0 of the Gateway API introduces percentage-based request mirroring as a new feature in the Standard channel. Additionally, there are three experimental features: CORS filters, a standardized mechanism for merging listeners and gateways, and retry budgets.
Inclusion in the Standard channel indicates a high level of confidence in the API surface and guarantees backward compatibility. Developers can thus rely on the new features being stable and future-proof.
A highlight of the new version is percentage-based request mirroring. This enhancement allows HTTP requests to be duplicated to another backend, which is particularly useful in blue-green deployments. This way, you can test the impact of request scaling on application performance without affecting client responses.
Previously, mirroring was performed for all requests to a backendRef. With the new feature, users can specify a portion of the requests they want to mirror, either as a percentage or a fraction. This is especially useful when services receive a high number of requests.
Here is an example where 42% of requests to “foo-v1” are mirrored to “foo-v2”:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: http-filter-mirror
labels:
gateway: mirror-gateway
spec:
parentRefs:
- name: mirror-gateway
hostnames:
- mirror.example
rules:
- backendRefs:
- name: foo-v1
port: 8080
filters:
- type: RequestMirror
requestMirror:
backendRef:
name: foo-v2
port: 8080
percent: 42 # This value must be an integer.
You can also configure partial mirroring with a fraction. Here is an example where 5 out of 1000 requests to “foo-v1” are mirrored to “foo-v2”:
rules:
- backendRefs:
- name: foo-v1
port: 8080
filters:
- type: RequestMirror
requestMirror:
backendRef:
name: foo-v2
port: 8080
fraction:
numerator: 5
denominator: 1000
These new features provide developers and DevOps teams with the ability to make their applications more flexible and robust. At ayedo, we are thrilled with the advancements of the Gateway API and are here to support you in effectively utilizing these new features!
Source: Kubernetes Blog
Modern generative AI and large language models (LLMs) present unique traffic management challenges …
In the world of Kubernetes development, there’s exciting news: JobSet has been introduced, an …
Efficient management of Kubernetes clusters is becoming increasingly important, especially as …