feat: logs SDK observability - otlploggrpc exporter metrics #7353
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7353 +/- ##
=======================================
+ Coverage 85.5% 85.7% +0.1%
=======================================
Files 279 284 +5
Lines 24711 25001 +290
=======================================
+ Hits 21143 21436 +293
+ Misses 3189 3186 -3
Partials 379 379
🚀 New features to boost your workflow:
|
flc1125
reviewed
Sep 12, 2025
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation_test.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
yumosx
commented
Sep 12, 2025
MrAlias
reviewed
Sep 15, 2025
Contributor
MrAlias
left a comment
There was a problem hiding this comment.
Thank you for the contribution. This is a significant step towards resolution, but there are few outstanding issues that need to be addressed:
- The scope name needs to be fixed
- Tests need to be updated to evaluate the scope and any other telemetry that is missing
- There are no benchmarks
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
flc1125
reviewed
Sep 16, 2025
flc1125
approved these changes
Sep 26, 2025
Member
flc1125
left a comment
There was a problem hiding this comment.
Apart from some remaining issues, I have no problems with other aspects on my end.
Thank you for your hard work and contributions.
9f46a5a to
6bb0d6b
Compare
MrAlias
added a commit
to MrAlias/opentelemetry-go
that referenced
this pull request
Sep 26, 2025
In order to add gRPC server attributes for exporter observability, this information needs to be parsed into a host and port. The added generated files provides ParseCanonicalTarget for this functionality. This is added as a generated template as it is expected to be needed for all OTLP exporters (e.g. open-telemetry#7404, open-telemetry#7353).
MrAlias
added a commit
that referenced
this pull request
Sep 29, 2025
In order to add [gRPC server attributes for exporter observability](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/otel/sdk-metrics.md#metric-otelsdkexporterspaninflight), this information needs to be parsed into a host and port. The added generated files provides `ParseCanonicalTarget` for this functionality. This is added as a generated template as it is expected to be needed for all OTLP exporters (e.g. #7404, #7353). Split from work added to #7404 cc @yumosx ### Benchamarks ``` goos: linux goarch: amd64 pkg: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc/internal/observ cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz │ otlptracegrpc-observ-target.bmark.result │ │ sec/op │ ParseTarget/HostName-8 80.90n ± 1% ParseTarget/HostPort-8 123.2n ± 4% ParseTarget/IPv4WithoutPort-8 94.25n ± 2% ParseTarget/IPv4WithPort-8 136.2n ± 1% ParseTarget/IPv6Bare-8 195.5n ± 2% ParseTarget/IPv6Bracket-8 191.2n ± 3% ParseTarget/IPv6WithPort-8 128.6n ± 4% ParseTarget/UnixSocket-8 15.73n ± 4% ParseTarget/UnixAbstractSocket-8 15.71n ± 6% ParseTarget/Passthrough-8 129.3n ± 18% geomean 84.98n │ otlptracegrpc-observ-target.bmark.result │ │ B/op │ ParseTarget/HostName-8 48.00 ± 0% ParseTarget/HostPort-8 48.00 ± 0% ParseTarget/IPv4WithoutPort-8 16.00 ± 0% ParseTarget/IPv4WithPort-8 48.00 ± 0% ParseTarget/IPv6Bare-8 16.00 ± 0% ParseTarget/IPv6Bracket-8 16.00 ± 0% ParseTarget/IPv6WithPort-8 48.00 ± 0% ParseTarget/UnixSocket-8 0.000 ± 0% ParseTarget/UnixAbstractSocket-8 0.000 ± 0% ParseTarget/Passthrough-8 48.00 ± 0% geomean ¹ ¹ summaries must be >0 to compute geomean │ otlptracegrpc-observ-target.bmark.result │ │ allocs/op │ ParseTarget/HostName-8 1.000 ± 0% ParseTarget/HostPort-8 1.000 ± 0% ParseTarget/IPv4WithoutPort-8 1.000 ± 0% ParseTarget/IPv4WithPort-8 1.000 ± 0% ParseTarget/IPv6Bare-8 1.000 ± 0% ParseTarget/IPv6Bracket-8 1.000 ± 0% ParseTarget/IPv6WithPort-8 1.000 ± 0% ParseTarget/UnixSocket-8 0.000 ± 0% ParseTarget/UnixAbstractSocket-8 0.000 ± 0% ParseTarget/Passthrough-8 1.000 ± 0% geomean ¹ ¹ summaries must be >0 to compute geomean ```
MrAlias
reviewed
Sep 29, 2025
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
exporters/otlp/otlplog/otlploggrpc/internal/observ/instrumentation.go
Outdated
Show resolved
Hide resolved
MrAlias
reviewed
Sep 30, 2025
Contributor
MrAlias
left a comment
There was a problem hiding this comment.
Looks good outside of the un-export of the added function.
MrAlias
approved these changes
Oct 2, 2025
46 tasks
Merged
MrAlias
added a commit
that referenced
this pull request
Dec 8, 2025
## Overview ### Added - Greatly reduce the cost of recording metrics in `go.opentelemetry.io/otel/sdk/metric` using hashing for map keys. (#7175) - Add `WithInstrumentationAttributeSet` option to `go.opentelemetry.io/otel/log`, `go.opentelemetry.io/otel/metric`, and `go.opentelemetry.io/otel/trace` packages. This provides a concurrent-safe and performant alternative to `WithInstrumentationAttributes` by accepting a pre-constructed `attribute.Set`. (#7287) - Add experimental observability for the Prometheus exporter in `go.opentelemetry.io/otel/exporters/prometheus`. Check the `go.opentelemetry.io/otel/exporters/prometheus/internal/x` package documentation for more information. (#7345) - Add experimental observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#7353) - Add temporality selector functions `DeltaTemporalitySelector`, `CumulativeTemporalitySelector`, `LowMemoryTemporalitySelector` to `go.opentelemetry.io/otel/sdk/metric`. (#7434) - Add experimental observability metrics for simple log processor in `go.opentelemetry.io/otel/sdk/log`. (#7548) - Add experimental observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. (#7459) - Add experimental observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. (#7486) - Add experimental observability metrics for simple span processor in `go.opentelemetry.io/otel/sdk/trace`. (#7374) - Add experimental observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#7512) - Add experimental observability metrics for manual reader in `go.opentelemetry.io/otel/sdk/metric`. (#7524) - Add experimental observability metrics for periodic reader in `go.opentelemetry.io/otel/sdk/metric`. (#7571) - Support `OTEL_EXPORTER_OTLP_LOGS_INSECURE` and `OTEL_EXPORTER_OTLP_INSECURE` environmental variables in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#7608) - Add `Enabled` method to the `Processor` interface in `go.opentelemetry.io/otel/sdk/log`. All `Processor` implementations now include an `Enabled` method. (#7639) - The `go.opentelemetry.io/otel/semconv/v1.38.0` package. The package contains semantic conventions from the `v1.38.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.38.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.37.0.`(#7648) ### Changed - `Distinct` in `go.opentelemetry.io/otel/attribute` is no longer guaranteed to uniquely identify an attribute set. Collisions between `Distinct` values for different Sets are possible with extremely high cardinality (billions of series per instrument), but are highly unlikely. (#7175) - `WithInstrumentationAttributes` in `go.opentelemetry.io/otel/trace` synchronously de-duplicates the passed attributes instead of delegating it to the returned `TracerOption`. (#7266) - `WithInstrumentationAttributes` in `go.opentelemetry.io/otel/meter` synchronously de-duplicates the passed attributes instead of delegating it to the returned `MeterOption`. (#7266) - `WithInstrumentationAttributes` in `go.opentelemetry.io/otel/log` synchronously de-duplicates the passed attributes instead of delegating it to the returned `LoggerOption`. (#7266) - Rename the `OTEL_GO_X_SELF_OBSERVABILITY` environment variable to `OTEL_GO_X_OBSERVABILITY` in `go.opentelemetry.io/otel/sdk/trace`, `go.opentelemetry.io/otel/sdk/log`, and `go.opentelemetry.io/otel/exporters/stdout/stdouttrace`. (#7302) - Improve performance of histogram `Record` in `go.opentelemetry.io/otel/sdk/metric` when min and max are disabled using `NoMinMax`. (#7306) - Improve error handling for dropped data during translation by using `prometheus.NewInvalidMetric` in `go.opentelemetry.io/otel/exporters/prometheus`.⚠️ **Breaking Change:** Previously, these cases were only logged and scrapes succeeded. Now, when translation would drop data (e.g., invalid label/value), the exporter emits a `NewInvalidMetric`, and Prometheus scrapes **fail with HTTP 500** by default. To preserve the prior behavior (scrapes succeed while errors are logged), configure your Prometheus HTTP handler with: `promhttp.HandlerOpts{ ErrorHandling: promhttp.ContinueOnError }`. (#7363) - Replace fnv hash with xxhash in `go.opentelemetry.io/otel/attribute` for better performance. (#7371) - The default `TranslationStrategy` in `go.opentelemetry.io/exporters/prometheus` is changed from `otlptranslator.NoUTF8EscapingWithSuffixes` to `otlptranslator.UnderscoreEscapingWithSuffixes`. (#7421) - Improve performance of concurrent measurements in `go.opentelemetry.io/otel/sdk/metric`. (#7427) - Include W3C TraceFlags (bits 0–7) in the OTLP `Span.Flags` field in `go.opentelemetry.io/exporters/otlp/otlptrace/otlptracehttp` and `go.opentelemetry.io/exporters/otlp/otlptrace/otlptracegrpc`. (#7438) - The `ErrorType` function in `go.opentelemetry.io/otel/semconv/v1.37.0` now handles custom error types. If an error implements an `ErrorType() string` method, the return value of that method will be used as the error type. (#7442) ### Fixed - Fix `WithInstrumentationAttributes` options in `go.opentelemetry.io/otel/trace`, `go.opentelemetry.io/otel/metric`, and `go.opentelemetry.io/otel/log` to properly merge attributes when passed multiple times instead of replacing them. Attributes with duplicate keys will use the last value passed. (#7300) - The equality of `attribute.Set` when using the `Equal` method is not affected by the user overriding the empty set pointed to by `attribute.EmptySet` in `go.opentelemetry.io/otel/attribute`. (#7357) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#7372) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#7372) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. (#7372) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#7372) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. (#7372) - Return partial OTLP export errors to the caller in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. (#7372) - Fix `AddAttributes`, `SetAttributes`, `SetBody` on `Record` in `go.opentelemetry.io/otel/sdk/log` to not mutate input. (#7403) - Do not double record measurements of `RecordSet` methods in `go.opentelemetry.io/otel/semconv/v1.37.0`. (#7655) - Do not double record measurements of `RecordSet` methods in `go.opentelemetry.io/otel/semconv/v1.36.0`. (#7656) ### Removed - Drop support for [Go 1.23]. (#7274) - Remove the `FilterProcessor` interface in `go.opentelemetry.io/otel/sdk/log`. The `Enabled` method has been added to the `Processor` interface instead. All `Processor` implementations must now implement the `Enabled` method. Custom processors that do not filter records can implement `Enabled` to return `true`. (#7639)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for experimental metrics in
otlploggrpcotel.sdk.exporter.log.inflightotel.sdk.exporter.log.exportedotel.sdk.exporter.operation.durationReferences:
goos: darwin goarch: arm64 pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc cpu: Apple M3 │ disabled.txt │ enabled.txt │ │ sec/op │ sec/op vs base │ ExporterExportLogs/Observability-8 681.5µ ± 3% 684.3µ ± 6% ~ (p=0.315 n=10) │ disabled.txt │ enabled.txt │ │ B/op │ B/op vs base │ ExporterExportLogs/Observability-8 672.8Ki ± 0% 673.6Ki ± 1% ~ (p=0.247 n=10) │ disabled.txt │ enabled.txt │ │ allocs/op │ allocs/op vs base │ ExporterExportLogs/Observability-8 9.224k ± 0% 9.232k ± 0% +0.09% (p=0.000 n=10)goos: darwin goarch: arm64 pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/observ cpu: Apple M3 │ bench.txt │ │ sec/op │ InstrumentationExportLogs/NoError-8 162.6n ± 3% InstrumentationExportLogs/PartialError-8 705.5n ± 5% InstrumentationExportLogs/FullError-8 592.1n ± 1% geomean 408.0n │ bench.txt │ │ B/op │ InstrumentationExportLogs/NoError-8 152.0 ± 0% InstrumentationExportLogs/PartialError-8 697.0 ± 0% InstrumentationExportLogs/FullError-8 616.0 ± 0% geomean 402.6 │ bench.txt │ │ allocs/op │ InstrumentationExportLogs/NoError-8 3.000 ± 0% InstrumentationExportLogs/PartialError-8 10.00 ± 0% InstrumentationExportLogs/FullError-8 8.000 ± 0% geomean 6.214pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/observ cpu: Apple M3 │ parse_target.txt │ │ sec/op │ ParseTarget/HostName-8 38.00n ± ∞ ¹ ParseTarget/HostPort-8 51.33n ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 44.74n ± ∞ ¹ ParseTarget/IPv4WithPort-8 62.56n ± ∞ ¹ ParseTarget/IPv6Bare-8 94.89n ± ∞ ¹ ParseTarget/IPv6Bracket-8 93.78n ± ∞ ¹ ParseTarget/IPv6WithPort-8 57.57n ± ∞ ¹ ParseTarget/UnixSocket-8 8.329n ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 9.082n ± ∞ ¹ ParseTarget/Passthrough-8 58.06n ± ∞ ¹ geomean 40.64n ¹ need >= 6 samples for confidence interval at level 0.95 │ parse_target.txt │ │ B/op │ ParseTarget/HostName-8 48.00 ± ∞ ¹ ParseTarget/HostPort-8 48.00 ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 16.00 ± ∞ ¹ ParseTarget/IPv4WithPort-8 48.00 ± ∞ ¹ ParseTarget/IPv6Bare-8 16.00 ± ∞ ¹ ParseTarget/IPv6Bracket-8 16.00 ± ∞ ¹ ParseTarget/IPv6WithPort-8 48.00 ± ∞ ¹ ParseTarget/UnixSocket-8 0.000 ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 0.000 ± ∞ ¹ ParseTarget/Passthrough-8 48.00 ± ∞ ¹ geomean ² ¹ need >= 6 samples for confidence interval at level 0.95 ² summaries must be >0 to compute geomean │ parse_target.txt │ │ allocs/op │ ParseTarget/HostName-8 1.000 ± ∞ ¹ ParseTarget/HostPort-8 1.000 ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 1.000 ± ∞ ¹ ParseTarget/IPv4WithPort-8 1.000 ± ∞ ¹ ParseTarget/IPv6Bare-8 1.000 ± ∞ ¹ ParseTarget/IPv6Bracket-8 1.000 ± ∞ ¹ ParseTarget/IPv6WithPort-8 1.000 ± ∞ ¹ ParseTarget/UnixSocket-8 0.000 ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 0.000 ± ∞ ¹ ParseTarget/Passthrough-8 1.000 ± ∞ ¹ geomean ² ¹ need >= 6 samples for confidence interval at level 0.95 ² summaries must be >0 to compute geomean