Create a mixed-protocol external load balancer

Autopilot Standard

This document shows you how to expose an application running in a Google Kubernetes Engine (GKE) cluster to the internet by using a mixed-protocol, external LoadBalancer Service for both TCP and UDP traffic.

To learn more about external passthrough Network Load Balancers, see Backend service-based external passthrough Network Load Balancer.

Overview

You can expose applications that use both TCP and UDP protocols by using two separate GKE LoadBalancer Services with a manually coordinated, shared IP address. However, this approach is inefficient because it requires managing multiple Services for a single application and might lead to issues such as configuration errors or exhausted IP address quotas.

Mixed-protocol LoadBalancer Services let you use a single Service to manage traffic for both TCP and UDP. Using a single Service simplifies your configuration by letting you use a single IPv4 address and a consolidated set of forwarding rules for both protocols. This feature is supported for external passthrough Network Load Balancer.

Before you begin

Before you start, make sure that you have performed the following tasks:

Enable the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.
Note: For existing gcloud CLI installations, make sure to set the compute/region property. If you use primarily zonal clusters, set the compute/zone instead. By setting a default location, you can avoid errors in the gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location. You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.

Ensure that you have an existing Autopilot or Standard cluster. To create a new cluster, see Create an Autopilot cluster.

Requirements

To create an external LoadBalancer Service that uses mixed protocols, your cluster must meet the following requirements:

Mixed-protocol load balancing is available only on newly-created clusters on version 1.34.1-gke.2190000 or later.
You must have the HttpLoadBalancing addon enabled on your cluster.
For new external LoadBalancer Services, to implement the load balancer, set the spec.loadBalancerClass field to networking.gke.io/l4-regional-external in the Service manifest. For existing Services, your manifest already has the cloud.google.com/l4-rbs: "enabled" annotation, and you can leave the annotation as is.

Limitations

Mixed-protocol load balancers support only IPv4 addresses.
You can't use mixed protocols in a Service manifest with the following finalizers:
- gke.networking.io/l4-ilb-v1
- gke.networking.io/l4-netlb-v1
If your manifest has these finalizers, you must delete and re-create the Service according to the preceding requirements.

Pricing

Google Cloud bills you per forwarding rule, for any external IP addresses, and for data sent. The following table describes the number of forwarding rules and external IP addresses used for specified configurations. For more information, see VPC network pricing.

Type	Transport Layer	Internet Layer	Number of Forwarding Rules	Number of External IP Addresses
External	Single (Either TCP or UDP)	IPv4	1	1
		IPv6	1	1
		IPv4 and IPv6(DualStack)	2	2
	Mixed (Both TCP and UDP)	IPv4	2	1

Deploy a workload

This section shows how to deploy a sample workload that listens on both TCP and UDP ports. Note that the Deployment configuration is the same whether you are using a mixed-protocol LoadBalancer Service or two separate single-protocol LoadBalancer Services.

The following manifest is for a sample application that listens on port 8080 for both TCP and UDP traffic. Save the following manifest as mixed-app-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mixed-app-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: mixed-app
  template:
    metadata:
      labels:
        app: mixed-app
    spec:
      containers:
      - image: gcr.io/kubernetes-e2e-test-images/agnhost:2.6
        name: agnhost
        args: ["serve-hostname", "--port=8080", "--tcp=true", "--udp=true", "--http=false"]
        ports:
          - name: tcp8080
            protocol: TCP
            containerPort: 8080
          - name: udp8080
            protocol: UDP
            containerPort: 8080

Apply the manifest to your cluster:

kubectl apply -f mixed-app-deployment.yaml

Create a mixed-protocol load balancer

Create a Service of type LoadBalancer that exposes the deployment to both TCP and UDP traffic.

Save the following manifest as mixed-protocol-lb.yaml:

apiVersion: v1
kind: Service
metadata:
  name: mixed-protocol-lb
spec:
  loadBalancerClass: "networking.gke.io/l4-regional-external"
  type: LoadBalancer
  selector:
    app: mixed-app
  ports:
  - name: tcp-port
    protocol: TCP
    port: 8080
  - name: udp-port
    protocol: UDP
    port: 8080

The preceding Service has two ports, one for TCP and one for UDP, both on port 8080.

Apply the manifest to your cluster:

kubectl apply --server-side -f mixed-protocol-lb.yaml

Verify the mixed-protocol load balancer

After you create the Service, verify that GKE created the load balancer successfully.

Inspect the Service:
```
kubectl describe service mixed-protocol-lb
```
The output shows the external IP address of the load balancer and the forwarding rules for both TCP and UDP. Verify the following details in the output:
- The status.loadBalancer.ingress.ip field is populated.
- Verify that the following annotations for your external load balancer are present:
  - service.kubernetes.io/tcp-forwarding-rule
  - service.kubernetes.io/udp-forwarding-rule
- The Events section contains no error messages.

Update the mixed-protocol load balancer

You can update the ports on a mixed-protocol load balancer by editing the Service manifest. To edit the Service, run the following command:

kubectl edit service SERVICE_NAME

Replace SERVICE_NAME with the name of your Service.

Update ports

To update the ports on a mixed-protocol load balancer, modify the ports section of the Service manifest. You can add, remove, or modify ports.

The following example adds a UDP port for streaming and a TCP port for game-server metadata:

apiVersion: v1
kind: Service
metadata:
  name: mixed-protocol-lb
spec:
  loadBalancerClass: "networking.gke.io/l4-regional-external"
  type: LoadBalancer
  selector:
    app: mixed-app
  ports:
  - name: tcp-port
    protocol: TCP
    port: 8080
  - name: streaming
    protocol: UDP
    port: 10100
  - name: gameserver-metadata
    protocol: TCP
    port: 10400
  - name: https
    protocol: TCP
    port: 443

Update a single-protocol load balancer to mixed-protocol

To change a single-protocol load balancer to a mixed-protocol load balancer, edit the Service to include ports for both the TCP and UDP protocols.

The following example adds a UDP port for DNS to an existing TCP-only load balancer:

apiVersion: v1
kind: Service
metadata:
  name: already-existing-single-protocol-lb
spec:
  loadBalancerClass: "networking.gke.io/l4-regional-external"
  type: LoadBalancer
  selector:
    app: mixed-app
  ports:
  - name: http
    protocol: TCP
    port: 80
  - name: https
    protocol: TCP
    port: 443
  - name: dns
    protocol: UDP
    port: 53

Update a mixed-protocol load balancer to single-protocol

To change a mixed-protocol load balancer to a single-protocol load balancer, remove all ports for one of the protocols.

The following example removes the UDP port for DNS, which converts the load balancer to TCP-only:

apiVersion: v1
kind: Service
metadata:
  name: already-existing-mixed-protocol-lb
spec:
  loadBalancerClass: "networking.gke.io/l4-regional-external"
  type: LoadBalancer
  selector:
    app: mixed-app
  ports:
  - name: http
    protocol: TCP
    port: 80
  - name: https
    protocol: TCP
    port: 443

Delete the mixed-protocol LoadBalancer

To delete the mixed-protocol-lb external LoadBalancer Service, run the following command:

kubectl delete service mixed-protocol-lb

GKE automatically removes all load balancer resources created for the Service.

Troubleshooting

This section describes how to resolve common issues with mixed-protocol LoadBalancer Services.

Check for error events

The first step in troubleshooting is to check the events associated with your Service.

Get the details of your Service:

kubectl describe service mixed-protocol-lb

Review the Events section at the end of the output for any error messages.

Error: Mixed Protocol is not supported for LoadBalancer

If you created the Service with the cloud.google.com/l4-rbs: "enabled" annotation, you might see a warning event from the original service controller after you create the mixed-protocol load balancer: mixed-protocol is not supported for LoadBalancer.

You can safely ignore this message because the new controller, which supports mixed-protocols, correctly provisions the load balancer.

Port definition is missing after an update

Symptom:

When you update a Service that uses the same port for both TCP and UDP (for example, port 8080), one of the port definitions is missing from the updated Service.

Cause:

This is a known issue in Kubernetes. When you update a Service with multiple protocols on the same port, the client-side patch calculation can incorrectly merge the port list, which causes one of the port definitions to be removed. This issue affects clients that use client-side patching, such as kubectl apply and the Go client with merge patches.

Solution:

The workaround for this issue depends on your client.

For kubectl: Use the --server-side flag with kubectl apply:
```
kubectl apply --server-side -f YOUR_SERVICE_MANIFEST.yaml
```
Replace YOUR_SERVICE_MANIFEST with the name of your Service manifest.
For go-client: Don't use merge patches. Instead, use an update call to replace the Service. This requires an HTTP PUT request with the entire Service object specification.

What's next

Learn more about exposing applications using Services.
Read about LoadBalancer Services.