Configure and Deploy Confluent Gateway using Confluent for Kubernetes

This guide provides instructions for configuring and deploying Confluent Gateway using Confluent for Kubernetes (CFK).

For the configuration and deployment steps using Docker, see Deploy Confluent Gateway.

A separate guide is provided for the security-related tasks listed below. See Configure Security for Confluent Gateway using CFK:

Configure secret stores
Configure TLS/SSL
Configure passwords
Configure authentication swapping

Confluent Gateway supports deployment on-premises, in private cloud VPCs, or in hybrid environments.

The high-level steps to deploy Confluent Gateway are:

Configure Confluent Gateway to connect to Kafka clusters, specifying authentication, routing policies, and security policies according to your organizational needs.

Additionally, Configure security for the Confluent Gateway.
Provision the Confluent Gateway service using the configuration settings from the previous step.
Reconfigure clients to communicate with the Confluent Gateway endpoint.

This simplifies credential management and strengthens overall security posture.

Note that the Kubernetes commands used in this guide assume that the default namespace is set and omits specifying the namespace for brevity.

Requirements and considerations

Before you begin, make sure that the following requirements are satisfied.

System requirements
- CPU and memory
  - Minimum: 2 vCPUs, 4 GB RAM
  - Recommended: 4 vCPUs, 8 GB RAM
- Network throughput: 45 MB/sec of sustained workload for a 1 Gigabit Link (1Gbps)
- Storage: 10 GB of disk space
Network access

The Confluent Gateway container must have access to necessary Kafka clusters. Configure firewall ports for bidirectional traffic as needed.
CFK and Confluent Platform

Ensure your Kubernetes environment is set up and CFK and Confluent Platform are installed.

For details on installing CFK, see Deploy Confluent for Kubernetes.

Use the appropriate version of CFK that supports your Confluent Platform and Confluent Gateway requirements.

Configure Confluent Gateway

Provide the following configuration settings in the Confluent Gateway custom resource (CR) file.

The top-level layout for the Confluent Gateway CR is as follows:

kind: Gateway
metadata:
  name:                   --- [1]
  namespace:              --- [2]
spec:
  image:
    application:          --- [3]
    init:                 --- [4]
  streamingDomains:       --- [5]
  secretStores:           --- [6]
  routes:                 --- [7]
  admin:                  --- [8]
  advanced:               --- [9]

[1] The Confluent Gateway name.
[2] The Confluent Gateway namespace.
[3] The Confluent Gateway application image in the following pattern.
- For Confluent Private Cloud Gateway, specify confluentinc/cpc-gateway:<version-tag>.
- For Confluent Cloud Gateway, specify confluentinc/confluent-gateway-for-cloud:<version-tag>.
<version-tag> is the Confluent Gateway release, currently 1.0.0.
[4] Set to the Confluent Gateway CFK init container image and its release tag, confluentinc/confluent-init-container:3.1.0.
[5] A list of streaming domains that represent Kafka clusters. See Configure streaming domains section for details.
[6] A list of credential stores. Required for authentication swapping. See Secret store configuration section for details.
[7] A list of client-traffic listeners that enforces governance policies and forwards requests to the appropriate streaming domain. See Configure routes section for details.
[8] Admin/management configuration. See Configure administration and metrics section for details.
[9] Advanced settings.

Configure streaming domains

Streaming domains are logical representations of your Kafka clusters in Confluent Gateway.

kind: Gateway
spec:
  streamingDomains:
    - name:               --- [1]
      type:               --- [2]
      kafkaCluster:       --- [3]
        name:             --- [4]
        bootstrapServers: --- [5]
          - id:           --- [6]
            endpoint:     --- [7]
            ssl:          --- [8]
        nodeIdRanges:     --- [9]
          - name:         --- [10]
            start:        --- [11]
            end:          --- [12]

[1] A unique name for the streaming domain.
[2] The type of the streaming domain. The default value is kafka.
[3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or Apache Kafka®, that the Confluent Gateway can route traffic to.
[4] The name of the Kafka cluster.
[5] A list of Kafka bootstrap servers. At minimum, one bootstrap server is required per Kafka cluster. For each security protocol listener, one can define one bootstrap server.
[6] A unique identifier for the bootstrap-server. Recommended to use the protocol, such as SASL_PLAINTEXT, SSL or PLAINTEXT_SASL_PLAIN for clear distinction between both TLS channel type and SASL mechanism.
[7] The bootstrap endpoint of the Kafka broker in the format, protocol://host:port
[8] Only required for SSL/SASL_SSL configuration.

See the TLS/SSL configuration section for details.
[9] Optional. The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.
[10] The name of the node ID range.
[11] The start of the node ID range, inclusive.
[12] The end of the node ID range, inclusive.

The following is an example configuration for a Confluent Gateway streaming domain:

kind: Gateway
spec:
  streamingDomains:
    - name: sales # unique across gateway
      type: kafka # default: kafka
      kafkaCluster:
        name: sales-cluster # default: <streamingDomain.name>
        bootstrapServers:
          - id: PLAINTEXT-1
            endpoint: kafka0.example.com:9092
          - id: SASL_PLAINTEXT-1
            endpoint: kafka0.example.com:9093
        nodeIdRanges:
          - name: default
            start: 0              # inclusive
            end: 3                # inclusive

Configure routes

Routes are Confluent Gateway endpoints where client applications connect to stream data, and they define how client applications connect to Kafka clusters. Clients connect to the Confluent Gateway as if it were a Kafka cluster, while the Confluent Gateway handles routing and governance.

kind: Gateway
spec:
  routes:
    - name:                         --- [1]
      endpoint:                     --- [2]
      brokerIdentificationStrategy: --- [3]
        type:                       --- [4]
        pattern:                    --- [5]
      streamingDomain:              --- [6]
        name:                       --- [7]
        bootstrapServerId:          --- [8]
      security:                     --- [9]
    logNetwork:                     --- [10]
    logFrames:                      --- [11]

[1] The unique name for the route.
[2] The host:port combination that Confluent Gateway will listen on. This is the external address clients use to bootstrap to the Kafka cluster.
[3] Specifies the strategy for mapping client requests to a specific Kafka broker.
[4] The type of broker identification strategy. Set to port (default) or host.
- port strategy: Each Kafka broker is identified using a unique port number. This is the default strategy.
  
  Clients connect to different ports to reach specific brokers (for example, port 9092 to connect with broker-0, port 9093 to connect with broker-1).
  
  The nodeIdRanges for the streaming domain you set in Configure streaming domains is used.
  
  nodeIdRanges should be present in all the clusters associated with the route’s streaming domain.
- host strategy: Each Kafka broker is represented using a unique hostname.
  
  Clients use different host names to reach specific brokers (for example, broker-0.kafka.company.com, broker-1.kafka.company.com), and the Confluent Gateway routes based on the SNI header.
  
  The pattern setting ([5]) is used.
[5] The pattern for the broker identification strategy. Required if the type ([4]) is host. For example, broker-$(nodeId).eu-gw.sales.example.com:9092.
[6] The reference to a streaming domain.
[7] The name of the streaming domain. Must be a valid name from the gateway.streamingDomains[].name.
[8] The bootstrap server ID. Must match one of kafkaCluster.bootstrapServers[].id.
[9] The security configuration. See Configure Security for Confluent Gateway using Confluent for Kubernetes section for details.
[10] Optional. Enable network logging of the TCP-level connection details. The default value is false.
[11] Optional. Enable frame logging of Kafka protocol frames. The default value is false.

The following is an example configuration for a Confluent Gateway routes:

kind: Gateway
spec:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_SSL-1
      logNetwork: true
      logFrames: true

Configure administration and metrics

kind: Gateway
spec:
  admin:
    bindAddress:             --- [1]
    port:                    --- [2]
    endpoints:
      metrics:               --- [3]
    jvmMetrics:              --- [4]
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:              --- [5]
      host:
      region:

[1] The local address to bind the administration service and to listen on for incoming connections. The default value is 0.0.0.0.

This address determines which network interfaces the gateway will use to expose its administrative endpoints, such as metrics and liveness checks.
[2] The TCP port for exposing admin endpoints. The default value is 9190.

The Confluent Gateway will serve metrics at http://{bindAddress}:{port}/metrics and liveness check at http://{bindAddress}:{port}/livez.
[3] The metrics endpoints to be enabled and exposed. The default value is true.
[4] The standard JVM Metrics (uses Micrometer classes). By default, JvmGcMetrics, JvmMemoryMetrics, JvmThreadMetrics, ProcessorMetrics, and UptimeMetrics are enabled.
[5] Optional. The common tags to be applied to all the metrics.

The following is an example configuration for a Confluent Gateway administration and metrics:

kind: Gateway
spec:
  admin:
    bindAddress: 0.0.0.0
    port: 9190
    endpoints:
      metrics: true
    jvmMetrics:
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:
      host: pod-0
      region: us-west-2

Configure Confluent Gateway license

The table below summarizes the license modes supported by Confluent Gateway.

Deployment mode	Trial mode (default)	Enterprise mode for Confluent Private Cloud deployments	Enterprise mode for Confluent Cloud deployments
License required	No license is required	Valid Confluent Private Cloud license token for Confluent Private Cloud deployments	Valid Confluent Cloud Gateway license token for Confluent Cloud deployments
Limitation	Maximum of 4 routes can be configured	Supports only Confluent Private Cloud streaming domains with unlimited routes	Supports only Confluent Cloud streaming domains with unlimited routes
Purpose	Evaluation and testing	Gateway forwarding to self-managed Kafka clusters	Gateway forwarding to Confluent Cloud clusters
Duration	Indefinite	As specified in the license claim	As specified in the license claim
Image	Use `confluentinc/cpc-gateway` or `confluentinc/confluent-gateway-for-cloud`	Use `confluentinc/cpc-gateway`	Use `confluentinc/confluent-gateway-for-cloud`

To configure the Enterprise license mode:

Contact Confluent to obtain one or more valid license tokens for production use.

Create a Kubernetes secret with your valid license token(s) you obtained in the previous step.

Option A: Create the license secret from a file.

Create licenses.txt with your token(s), one per line.

echo "your-license-token-here" > licenses.txt

If multiple tokens, append them to the file.

echo "your-second-license-token-here" >> licenses.txt

Create the secret from the file.

kubectl create secret generic confluent-gateway-licenses \
  --from-file=licenses.txt=licenses.txt

Verify the secret.

kubectl get secret confluent-gateway-licenses

Option B: Create the license secret directly.

kubectl create secret generic confluent-gateway-licenses \
  --from-literal=licenses.txt='your-license-token'

Configure the Confluent Gateway CR to use the license secret created in the previous step.

kind: Gateway
spec:
  podTemplate:
    envVars:
      - name: GATEWAY_LICENSES
        valueFrom:
          secretKeyRef:
            name: confluent-gateway-licenses
            key: licenses.txt

To verify the license status, check the Confluent Gateway logs for license information after starting the Confluent Gateway.
```
kubectl logs -l app=confluent-gateway | grep -i license
```

For example license configurations, see Confluent Gateway examples in GitHub.

Install Confluent Gateway

Deploy Confluent Gateway using the Confluent Gateway CR file you set up in the previous section, Configure Confluent Gateway.

kubectl apply -f <gateway-CR-file>