Technical implementation

Goals

We considered several migration paths, each with their own advantages and drawbacks.
In the end we decided to implement the migration mode as defined below since this addresses our main goals:

• You can do a controlled migration on a domain by domain basis
• During the migration all Ingress NGINX resources keep working as expected

Before the migration

Before the migration all traffic goes through the Ingress NGINX controllers.

During the migration

To facilitate this migration we created custom frontends in the HAProxy ingress controllers.

These migration HTTP(S) frontends route all HTTP(S) traffic to the Ingress NGINX controller if:

The domain is not known in the HAProxy ingress controller:

• i.e. no Ingress resource exists with ingressClassName: haproxy for this domain
• We capture the domain by reading the Host header on HTTP request and the TLS SNI field on HTTPS requests

The request path contains .well-known/acme-challenge (HTTP only):

• This is needed to ensure certificates can get requested/renewed during the migration period
• During the migration cert-manager uses Ingress NGINX to request certificates, after the migration this is switched to HAProxy ingress.

If the domain is known to HAProxy ingress (and it's not an acme-challenge), the traffic is handled by the HAProxy ingress controller.

All traffic going into the cluster is routed through this migration HTTP(S) frontend.
At the start of the migration none of your domains will have an Ingress with ingressClassName: haproxy so all traffic is routed to the Ingress NGINX controller.

The following scheme shows the flow of traffic through the cluster in migration mode:

After the migration

After the migration is done all traffic will go to the HAProxy ingress, the custom migration frontends are removed and cert-manager is configured to use HAProxy ingress resources.