k8sobjectsreceiver

Kubernetes Objects Receiver

Status
Stability	beta: logs
Distributions	contrib, k8s
Issues
Code coverage
Code Owners	@dmitryax, @hvaghani221, @TylerHelmuth, @ChrsMark, @krisztianfekete

The kubernetes Objects receiver collects(pull/watch) objects from the Kubernetes API server.

Currently this receiver supports authentication via service accounts only. See example for more information.

Getting Started

The following is example configuration

  k8sobjects:
    auth_type: serviceAccount
    k8s_leader_elector: k8s_leader_elector
    objects:
      - name: pods
        mode: pull
        label_selector: environment in (production),tier in (frontend)
        field_selector: status.phase=Running
        interval: 15m
      - name: events
        mode: watch
        group: events.k8s.io
        namespaces: [default]

Brief description of configuration properties:

• auth_type (default = serviceAccount): Determines how to authenticate to the K8s API server. This can be one of none (for no auth), serviceAccount (to use the standard service account token provided to the agent pod), or kubeConfig to use credentials from ~/.kube/config.
• error_mode (default = propagate): Determines how to handle errors when the receiver is unable to pull or watch objects due to missing resources. This can be one of propagate, ignore, or silent.
  • propagate will propagate the error to the collector as an Error.
  • ignore will log and ignore the error and continue.
  • silent will ignore the error and continue without logging.
• name: Name of the resource object to collect
• mode: define in which way it collects this type of object, either "pull" or "watch".
  • pull mode will read all objects of this type use the list API at an interval.
  • watch mode will do setup a long connection using the watch API to just get updates.
• include_initial_state (default = false): When set to true (watch-mode only) the receiver sends a one-time snapshot of the current objects before it starts processing watch events.
• label_selector: select objects by label(s)
• field_selector: select objects by field(s)
• interval: the interval at which object is pulled, default 60 minutes. Only useful for pull mode.
• exclude_watch_type: allows excluding specific watch types. Valid values are ADDED, MODIFIED, DELETED, BOOKMARK, and ERROR. Only usable in watch mode.
• resource_version allows watch resources starting from a specific version (default = 1). Only available for watch mode. If not specified, the receiver will do an initial list to get the resourceVersion before starting the watch. See Efficient Detection of Change for details on why this is necessary.
• namespaces: An array of namespaces to collect events from. (default = all)
• group: API group name. It is an optional config. When given resource object is present in multiple groups, use this config to specify the group to select. By default, it will select the first group. For example, events resource is available in both v1 and events.k8s.io/v1 APIGroup. In this case, it will select v1 by default.
• k8s_leader_elector (default: none): if specified, will enable Leader Election by using k8sleaderelector extension

The full list of settings exposed for this receiver are documented in config.go with detailed sample configurations in testdata/config.yaml.

Follow the below sections to setup various Kubernetes resources required for the deployment.

Supported Kubernetes objects

The k8sobjectsreceiver supports collecting a wide range of standard Kubernetes objects from the API server. For example, the mostly used supported objects (in addition to other standard Kubernetes objects) are:

• pods
• events
• nodes
• jobs
• cronjobs
• deployments
• services
• configmaps
• secrets
• statefulsets
• replicasets
• daemonsets
• namespaces

This receiver supports both pull and watch modes, allowing for flexible and real-time monitoring of these objects. Please note that custom resources are supported only if their CRDs are available in the cluster.

Configuration

Create a ConfigMap with the config for otelcontribcol. Replace OTLP_ENDPOINT with valid value.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
data:
  config.yaml: |
    receivers:
      k8sobjects:
        objects:
          - name: pods
            mode: pull
          - name: events
            mode: watch
    exporters:
      otlp:
        endpoint: <OTLP_ENDPOINT>
        tls:
          insecure: true

    service:
      pipelines:
        logs:
          receivers: [k8sobjects]
          exporters: [otlp]
EOF

Service Account

Create a service account that the collector should use.

<<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: otelcontribcol
  name: otelcontribcol
EOF

RBAC

Use the below commands to create a ClusterRole with required permissions and a ClusterRoleBinding to grant the role to the service account created above. Following config will work for collecting pods and events only. You need to add appropriate rule for collecting other objects.

When using watch mode you must also specify list verb so that the receiver has permission to do its initial list if no resource_version was supplied or a list to recover from 410 Gone scenarios.

<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
rules:
- apiGroups:
  - ""
  resources:
  - events
  - pods
  verbs:
  - get
  - list
  - watch
- apiGroups: 
  - "events.k8s.io"
  resources:
  - events
  verbs:
  - watch
  - list
EOF

<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: otelcontribcol
subjects:
- kind: ServiceAccount
  name: otelcontribcol
  namespace: default
EOF

Deployment

Create a Deployment to deploy the collector. Note: This receiver must be deployed as one replica, otherwise it'll be producing duplicated data.

<<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otelcontribcol
  template:
    metadata:
      labels:
        app: otelcontribcol
    spec:
      serviceAccountName: otelcontribcol
      containers:
      - name: otelcontribcol
        # This image is created by running `make docker-otelcontribcol`.
        # If you are not building the collector locally, specify a published image: `otel/opentelemetry-collector-contrib`
        image: otelcontribcol:latest
        args: ["--config", "/etc/config/config.yaml"]
        volumeMounts:
        - name: config
          mountPath: /etc/config
        imagePullPolicy: IfNotPresent
      volumes:
        - name: config
          configMap:
            name: otelcontribcol
EOF

Troubleshooting

If receiver returns error similar to below, make sure that resource is added to ClusterRole.

{"kind": "receiver", "name": "k8sobjects", "pipeline": "logs", "resource": "events.k8s.io/v1, Resource=events", "error": "unknown"}