Transforming telemetry

The OpenTelemetry Collector is a convenient place to transform data before sending it to a vendor or other systems. This is frequently done for data quality, governance, cost, and security reasons.

Processors available from the Collector Contrib repository support dozens of different transformations on metric, span and log data. The following sections provide some basic examples on getting started with a few frequently-used processors.

The configuration of processors, particularly advanced transformations, may have a significant impact on collector performance.

Basic filtering

Processor: filter processor

The filter processor allows users to filter telemetry based on include or exclude rules. Include rules are used for defining “allow lists” where anything that does not match include rules is dropped from the collector. Exclude rules are used for defining “deny lists” where telemetry that matches rules is dropped from the collector.

For example, to only allow span data from services app1, app2, and app3 and drop data from all other services:

processors:
  filter/allowlist:
    spans:
      include:
        match_type: strict
        services:
          - app1
          - app2
          - app3

To only block spans from a service called development while allowing all other spans, an exclude rule is used:

processors:
  filter/denylist:
    spans:
      exclude:
        match_type: strict
        services:
          - development

The filter processor docs have more examples, including filtering on logs and metrics.

Adding or Deleting Attributes

Processor: attributes processor or resource processor

The attributes processor can be used to update, insert, delete, or replace existing attributes on metrics or traces. For example, here’s a configuration that adds an attribute called account_id to all spans:

processors:
  attributes/accountid:
    actions:
      - key: account_id
        value: 2245
        action: insert

The resource processor has an identical configuration, but applies only to resource attributes. Use the resource processor to modify infrastructure metadata related to telemetry. For example, this inserts the Kubernetes cluster name:

processors:
  resource/k8s:
    attributes:
      - key: k8s.cluster.name
        from_attribute: k8s-cluster
        action: insert

Renaming Metrics or Metric Labels

Processor: metrics transform processor

The metrics transform processor shares some functionality with the attributes processor, but also supports renaming and other metric-specific functionality.

processors:
  metricstransform/rename:
    transforms:
      include: system.cpu.usage
      action: update
      new_name: system.cpu.usage_time

The metrics transform processor also supports regular expressions to apply transform rules to multiple metric names or metric labels at the same time. This example renames cluster_name to cluster-name for all metrics:

processors:
  metricstransform/clustername:
    transforms:
      - include: ^.*$
        match_type: regexp
        action: update
        operations:
          - action: update_label
            label: cluster_name
            new_label: cluster-name

Enriching Telemetry with Resource Attributes

Processor: resource detection processor and k8sattributes processor

These processors can be used for enriching telemetry with relevant infrastructure metadata to help teams quickly identify when underlying infrastructure is impacting service health or performance.

The resource detection processor adds relevant cloud or host-level information to telemetry:

processors:
  resourcedetection/system:
    # Modify the list of detectors to match the cloud environment
    detectors: [env, system, gcp, ec2, azure]
    timeout: 2s
    override: false

Similarly, the K8s processor enriches telemetry with relevant Kubernetes metadata like pod name, node name, or workload name. The collector pod must be configured to have read access to certain Kubernetes RBAC APIs, which is documented here. To use the default options, it can be configured with an empty block:

processors:
  k8sattributes/default:

Advanced Transformations

More advanced attribute transformations are also available in the transform processor. The transform processor allows end-users to specify transformations on metrics, logs, and traces using the OpenTelemetry Transformation Language.