Span kind for method instrumentation / Declarative configuration tooling #14014

zeitlinger · 2025-06-10T15:00:04Z

Brings parity to @WithSpan where you can configure the span kind.

Syntax example:

https://github.com/zeitlinger/opentelemetry-java-instrumentation/blob/3db95b4e48b527fa6f0c09be8c4cba8614951d99/instrumentation/methods/javaagent/build.gradle.kts#L18

-Dotel.instrumentation.methods.include=io.opentelemetry.javaagent.instrumentation.methods.MethodTest\$ConfigTracedCallable[call];io.opentelemetry.javaagent.instrumentation.methods.MethodTest\$ConfigTracedCompletableFuture[getResult=SERVER];javax.naming.directory.InitialDirContext[search=CLIENT]"

zeitlinger · 2025-06-10T16:02:04Z

@laurit I hope this makes sense 😄

trask · 2025-06-10T16:18:06Z

my preference would be for the method instrumentation to support declarative config, and only support this (and other) new features via declarative config instead of trying to encode more things into otel.instrumentation.methods.include

but I realize that is probably quite a bit more effort

zeitlinger · 2025-06-10T16:40:44Z

my preference would be for the method instrumentation to support declarative config, and only support this (and other) new features via declarative config instead of trying to encode more things into otel.instrumentation.methods.include

but I realize that is probably quite a bit more effort

Yeah - that's a good point. If declarative config was ready to go, then I'd be happy to add it there.

trask · 2025-06-10T18:01:20Z

If declarative config was ready to go

do you know what's missing that we still need? it feels like we're super close, maybe we could use this to motivate getting the final pieces in place?

zeitlinger · 2025-06-13T05:31:43Z

From SIG meeting:

Something like that for structured config:

zeitlinger · 2025-06-13T10:23:05Z

This is the error I get locally:

The deadlock should have been avoided using https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/muzzle/src/main/java/io/opentelemetry/javaagent/tooling/TransformSafeLogger.java

Found one Java-level deadlock:
=============================
"Test worker":
  waiting to lock monitor 0x00007461bcf707f0 (object 0x00000000e05509c8, a io.opentelemetry.javaagent.slf4j.simple.SimpleLoggerConfiguration),
  which is held by "otel-javaagent-transform-safe-logger"

"otel-javaagent-transform-safe-logger":
  waiting for ownable synchronizer 0x00000000e309eac0, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
  which is held by "Test worker"

Java stack information for the threads listed above:
===================================================
"Test worker":
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.write(SimpleLogger.java:261)
        - waiting to lock <0x00000000e05509c8> (a io.opentelemetry.javaagent.slf4j.simple.SimpleLoggerConfiguration)
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.innerHandleNormalizedLoggingCall(SimpleLogger.java:439)
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.handleNormalizedLoggingCall(SimpleLogger.java:377)
        at io.opentelemetry.javaagent.slf4j.helpers.AbstractLogger.handleArgArrayCall(AbstractLogger.java:401)
        at io.opentelemetry.javaagent.slf4j.helpers.AbstractLogger.debug(AbstractLogger.java:161)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.logViaPublicSLF4JLoggerAPI(DefaultLoggingEventBuilder.java:204)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.log(DefaultLoggingEventBuilder.java:167)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.log(DefaultLoggingEventBuilder.java:122)
        at io.opentelemetry.javaagent.logging.simple.Slf4jSimpleLogger.log(Slf4jSimpleLogger.java:32)
        at io.opentelemetry.javaagent.bootstrap.PatchLogger.log(PatchLogger.java:147)
        at io.opentelemetry.javaagent.bootstrap.ExceptionLogger.logSuppressedError(ExceptionLogger.java:26)
        at java.util.concurrent.ThreadPoolExecutor.execute([email protected]/ThreadPoolExecutor.java:1340)
        at org.gradle.internal.concurrent.AbstractManagedExecutor.execute(AbstractManagedExecutor.java:39)
        at org.gradle.internal.remote.internal.hub.MessageHub.addHandler(MessageHub.java:133)
        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection.addIncoming(MessageHubBackedObjectConnection.java:100)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.startReceivingTests(TestWorker.java:156)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:98)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63)
        at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:122)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:72)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
"otel-javaagent-transform-safe-logger":
        at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
        - parking to wait for  <0x00000000e309eac0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:221)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire([email protected]/AbstractQueuedSynchronizer.java:754)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire([email protected]/AbstractQueuedSynchronizer.java:990)
        at java.util.concurrent.locks.ReentrantLock$Sync.lock([email protected]/ReentrantLock.java:153)
        at java.util.concurrent.locks.ReentrantLock.lock([email protected]/ReentrantLock.java:322)
        at org.gradle.internal.remote.internal.hub.MessageHub$ChannelDispatch.dispatch(MessageHub.java:370)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
        at jdk.proxy3.$Proxy39.sendOutputEvent(jdk.proxy3/Unknown Source)
        at org.gradle.process.internal.worker.child.WorkerLogEventListener.onOutput(WorkerLogEventListener.java:58)
        at java.lang.invoke.LambdaForm$DMH/0x00007461482a8000.invokeInterface([email protected]/LambdaForm$DMH)
        at java.lang.invoke.LambdaForm$MH/0x00007461482a8c00.invoke([email protected]/LambdaForm$MH)
        at java.lang.invoke.Invokers$Holder.invokeExact_MT([email protected]/Invokers$Holder)
        at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl([email protected]/DirectMethodHandleAccessor.java:154)
        at jdk.internal.reflect.DirectMethodHandleAccessor.invoke([email protected]/DirectMethodHandleAccessor.java:103)
        at java.lang.reflect.Method.invoke([email protected]/Method.java:580)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:44)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:268)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:170)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:160)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
        at jdk.proxy3.$Proxy37.onOutput(jdk.proxy3/Unknown Source)
        at org.gradle.internal.logging.sink.OutputEventTransformer.invokeListener(OutputEventTransformer.java:113)
        - locked <0x00000000e3454b60> (a java.lang.Object)
        at org.gradle.internal.logging.sink.OutputEventTransformer.onOutput(OutputEventTransformer.java:106)
        at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:435)
        at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
        at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:58)
        at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:42)
        at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:173)
        at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:45)
        at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:97)
        at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:81)
        at java.io.OutputStream.write([email protected]/OutputStream.java:167)
        at java.io.PrintStream.implWrite([email protected]/PrintStream.java:643)
        at java.io.PrintStream.write([email protected]/PrintStream.java:623)
        at sun.nio.cs.StreamEncoder.writeBytes([email protected]/StreamEncoder.java:309)
        at sun.nio.cs.StreamEncoder.implFlushBuffer([email protected]/StreamEncoder.java:405)
        at sun.nio.cs.StreamEncoder.lockedFlushBuffer([email protected]/StreamEncoder.java:123)
        at sun.nio.cs.StreamEncoder.flushBuffer([email protected]/StreamEncoder.java:110)
        at java.io.OutputStreamWriter.flushBuffer([email protected]/OutputStreamWriter.java:192)
        at java.io.PrintStream.implWrite([email protected]/PrintStream.java:812)
        at java.io.PrintStream.write([email protected]/PrintStream.java:790)
        at java.io.PrintStream.print([email protected]/PrintStream.java:1002)
        at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:239)
        - locked <0x00000000e33c4678> (a org.gradle.internal.io.LinePerThreadBufferingOutputStream)
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.write(SimpleLogger.java:261)
        - locked <0x00000000e05509c8> (a io.opentelemetry.javaagent.slf4j.simple.SimpleLoggerConfiguration)
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.innerHandleNormalizedLoggingCall(SimpleLogger.java:439)
        at io.opentelemetry.javaagent.slf4j.simple.SimpleLogger.handleNormalizedLoggingCall(SimpleLogger.java:377)
        at io.opentelemetry.javaagent.slf4j.helpers.AbstractLogger.handleArgArrayCall(AbstractLogger.java:403)
        at io.opentelemetry.javaagent.slf4j.helpers.AbstractLogger.debug(AbstractLogger.java:161)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.logViaPublicSLF4JLoggerAPI(DefaultLoggingEventBuilder.java:204)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.log(DefaultLoggingEventBuilder.java:167)
        at io.opentelemetry.javaagent.slf4j.spi.DefaultLoggingEventBuilder.log(DefaultLoggingEventBuilder.java:122)
        at io.opentelemetry.javaagent.logging.simple.Slf4jSimpleLogger.log(Slf4jSimpleLogger.java:32)
        at io.opentelemetry.javaagent.bootstrap.PatchLogger.log(PatchLogger.java:147)
        at io.opentelemetry.javaagent.tooling.TransformSafeLogger$LogMessageQueueReader.run(TransformSafeLogger.java:94)
        at java.lang.Thread.runWith([email protected]/Thread.java:1596)
        at java.lang.Thread.run([email protected]/Thread.java:1583)

zeitlinger · 2025-06-13T16:49:45Z

@laurit can you help with fixing the test?

I get this deadlock (see above) locally - and in GH actions there's no output:

2025-06-13T11:30:36.5988266Z [otel.javaagent 2025-06-13 11:30:36:577 +0000] [main] DEBUG io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder - Global OpenTelemetry set to OpenTelemetrySdk{tracerProvider=SdkTracerProvider{clock=SystemClock{}, idGenerator=RandomIdGenerator{}, resource=Resource{schemaUrl=null, attributes={service.instance.id="1a77091e-238f-432a-8da0-357c0de53d75", service.name="unknown_service:java", telemetry.distro.name="opentelemetry-java-instrumentation", telemetry.distro.version="2.17.0-SNAPSHOT", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.51.0"}}, spanLimitsSupplier=SpanLimitsValue{maxNumberOfAttributes=128, maxNumberOfEvents=128, maxNumberOfLinks=128, maxNumberOfAttributesPerEvent=128, maxNumberOfAttributesPerLink=128, maxAttributeValueLength=2147483647}, sampler=ParentBased{root:AlwaysOnSampler,remoteParentSampled:AlwaysOnSampler,remoteParentNotSampled:AlwaysOffSampler,localParentSampled:AlwaysOnSampler,localParentNotSampled:AlwaysOffSampler}, spanProcessor=MultiSpanProcessor{spanProcessorsStart=[io.opentelemetry.contrib.baggage.processor.BaggageSpanProcessor@6fdbe764], spanProcessorsEnding=[], spanProcessorsEnd=[SimpleSpanProcessor{spanExporter=LoggingSpanExporter{}, exportUnsampledSpans=false}, io.opentelemetry.javaagent.testing.exporter.AgentTestingSpanProcessor@51c668e3], spanProcessorsAll=[SimpleSpanProcessor{spanExporter=LoggingSpanExporter{}, exportUnsampledSpans=false}, io.opentelemetry.contrib.baggage.processor.BaggageSpanProcessor@6fdbe764, io.opentelemetry.javaagent.testing.exporter.AgentTestingSpanProcessor@51c668e3]}, tracerConfigurator=ScopeConfiguratorImpl{conditions=[Condition{scopeMatcher=ScopeNameMatcher{nameMatcher=io.opentelemetry.sdk.internal.ScopeConfiguratorBuilder$$Lambda$79/243194708@537f60bf}, scopeConfig=TracerConfig{enabled=false}}]}}, meterProvider=SdkMeterProvider{clock=SystemClock{}, resource=Resource{schemaUrl=null, attributes={service.instance.id="1a77091e-238f-432a-8da0-357c0de53d75", service.name="unknown_service:java", telemetry.distro.name="opentelemetry-java-instrumentation", telemetry.distro.version="2.17.0-SNAPSHOT", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.51.0"}}, metricReaders=[PeriodicMetricReader{exporter=io.opentelemetry.javaagent.testing.exporter.OtlpInMemoryMetricExporter@18df8434, intervalNanos=9223372036854775807}], metricProducers=[], views=[], meterConfigurator=ScopeConfiguratorImpl{conditions=[Condition{scopeMatcher=ScopeNameMatcher{nameMatcher=io.opentelemetry.sdk.internal.ScopeConfiguratorBuilder$$Lambda$79/243194708@65c7a252}, scopeConfig=MeterConfig{enabled=false}}]}}, loggerProvider=SdkLoggerProvider{clock=SystemClock{}, resource=Resource{schemaUrl=null, attributes={service.instance.id="1a77091e-238f-432a-8da0-357c0de53d75", service.name="unknown_service:java", telemetry.distro.name="opentelemetry-java-instrumentation", telemetry.distro.version="2.17.0-SNAPSHOT", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.51.0"}}, logLimits=LogLimits{maxNumberOfAttributes=128, maxAttributeValueLength=2147483647}, logRecordProcessor=SimpleLogRecordProcessor{logRecordExporter=io.opentelemetry.javaagent.testing.exporter.OtlpInMemoryLogRecordExporter@4686afc2}, loggerConfigurator=ScopeConfiguratorImpl{conditions=[Condition{scopeMatcher=ScopeNameMatcher{nameMatcher=io.opentelemetry.sdk.internal.ScopeConfiguratorBuilder$$Lambda$79/243194708@1e0b4072}, scopeConfig=LoggerConfig{enabled=false}}]}}, propagators=DefaultContextPropagators{textMapPropagator=MultiTextMapPropagator{textMapPropagators=[W3CTraceContextPropagator, W3CBaggagePropagator]}}} by autoconfiguration

2025-06-13T11:30:39.8925429Z [otel.javaagent 2025-06-13 11:30:37:967 +0000] [main] DEBUG io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder - Autoconfiguring from configuration file: /home/runner/work/opentelemetry-java-instrumentation/opentelemetry-java-instrumentation/instrumentation/methods/javaagent/src/declarativeConfigTest/resources/declarative-config.yaml


2025-06-13T11:33:01.4948363Z > Task :instrumentation:methods:javaagent:declarativeConfigTest
2025-06-13T11:33:01.4951151Z     [otel.javaagent 2025-06-13 11:33:00:770 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - 
Transformed io.opentelemetry.context.internal.shaded.AbstractWeakConcurrentMap -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.4955859Z     [otel.javaagent 2025-06-13 11:33:00:787 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.context.internal.shaded.WeakConcurrentMap -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.4960309Z     [otel.javaagent 2025-06-13 11:33:00:806 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.context.StrictContextStorage$PendingScopes -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.4997130Z     [otel.javaagent 2025-06-13 11:33:00:818 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.context.ContextStorageWrappers -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.5001803Z     [otel.javaagent 2025-06-13 11:33:00:837 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.context.ArrayBasedContext -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.5005773Z     [otel.javaagent 2025-06-13 11:33:00:876 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.instrumentation.api.internal.HttpRouteState -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.5009998Z     [otel.javaagent 2025-06-13 11:33:00:908 +0000] [otel-javaagent-transform-safe-logger] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Transformed io.opentelemetry.instrumentation.testing.util.ContextStorageCloser$$Lambda$855 -- sun.misc.Launcher$AppClassLoader@73d16e93
2025-06-13T11:33:01.5012115Z 
2025-06-13T11:33:01.5012448Z MethodTest > bootLoaderMethodTraced() FAILED
2025-06-13T11:33:01.5013200Z     java.lang.AssertionError: Error waiting for 1 traces
2025-06-13T11:33:01.5014738Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.waitForTraces(InstrumentationTestRunner.java:81)
2025-06-13T11:33:01.5016916Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.doAssertTraces(InstrumentationTestRunner.java:134)
2025-06-13T11:33:01.5018941Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.lambda$waitAndAssertTraces$0(InstrumentationTestRunner.java:127)
2025-06-13T11:33:01.5020963Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.awaitUntilAsserted(InstrumentationTestRunner.java:302)
2025-06-13T11:33:01.5023017Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.awaitUntilAsserted(InstrumentationTestRunner.java:285)
2025-06-13T11:33:01.5025319Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.waitAndAssertTraces(InstrumentationTestRunner.java:127)
2025-06-13T11:33:01.5027383Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.waitAndAssertTraces(InstrumentationTestRunner.java:117)
2025-06-13T11:33:01.5029445Z         at io.opentelemetry.instrumentation.testing.InstrumentationTestRunner.waitAndAssertTraces(InstrumentationTestRunner.java:113)
2025-06-13T11:33:01.5031719Z         at io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension.waitAndAssertTraces(InstrumentationExtension.java:152)
2025-06-13T11:33:01.5033633Z         at io.opentelemetry.javaagent.instrumentation.methods.MethodTest.bootLoaderMethodTraced(MethodTest.java:65)
2025-06-13T11:33:01.5034680Z 
2025-06-13T11:33:01.5034850Z         Caused by:
2025-06-13T11:33:01.5035948Z         java.util.concurrent.TimeoutException: Timeout waiting for 1 completed trace(s), found 0 completed trace(s) and 0 total trace(s): []

zeitlinger · 2025-06-16T07:39:51Z

Root cause: in memory exporters are not injected with config file.

zeitlinger · 2025-06-16T11:40:57Z

@trask I was actually able to make it work - it only took a bit of wiring in the agent code to make it work

@laurit can you take a look?

zeitlinger · 2025-06-16T14:29:23Z

@jack-berg can you check if this integration makes sense?

jack-berg

Seems like the right approach to me!

instrumentation/methods/javaagent/src/declarativeConfigTest/resources/declarative-config.yaml

jack-berg · 2025-06-16T15:42:14Z

instrumentation/methods/javaagent/src/declarativeConfigTest/resources/declarative-config.yaml

+instrumentation/development:
+  java:
+    methods:
+      include:


jack-berg · 2025-06-16T15:45:44Z

...avaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodsConfig.java

+        .collect(Collectors.toList());
+  }
+
+  private static Stream<MethodInstrumentation> parseMethodInstrumentation(


👍 looks about right.

I wish there was an easier way to parse the config that didn't rely on jackson

It would help to define a schema for validation and auto-completion. Is there a way to do that (as a follow-up)?

@jaydeluca FYI

How / where would you propose that schema is defined? And what tooling would interpret that schema and parse config to that representation?

I have some ideas on this of course, but think / hope this can be driven by instrumentation.

BTW - added this to the SIG meeting agenda

How / where would you propose that schema is defined?

generated based on data from SDK and instrumentation

@jaydeluca is already working in extracting config options (and other metadat) from the java agent

And what tooling would interpret that schema and parse config to that representation?

yaml editors would consume this - as a special case, the spring boot editor already has a file format: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/spring/spring-boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

hope this can be driven by instrumentation

probably - since it's instrumentation specific

Created #14082

zeitlinger · 2025-06-17T11:53:53Z

instrumentation-api-incubator/build.gradle.kts

@@ -13,7 +13,7 @@ group = "io.opentelemetry.instrumentation"
 dependencies {
  api("io.opentelemetry.semconv:opentelemetry-semconv")
  api(project(":instrumentation-api"))
-  implementation("io.opentelemetry:opentelemetry-api-incubator")
+  api("io.opentelemetry:opentelemetry-api-incubator")


this has become necessary because of https://github.com/zeitlinger/opentelemetry-java-instrumentation/blob/132267f13ee6c3cf4e012a31adb2a83ba2dfc184/instrumentation-api-incubator/src/main/java/io/opentelemetry/instrumentation/api/incubator/config/internal/InstrumentationConfig.java#L124

laurit · 2025-06-18T09:15:25Z

This is the error I get locally:

From the stack trace I'd guess that there is an exception thrown from advice code that is applied to java.util.concurrent.ThreadPoolExecutor.execute. If whatever causes that exception is resolved then the deadlock should also go away. I think that the deadlock could also be resolved by reworking the logging by removing the separate TransformSafeLogger and moving its logic into a wrapper for InternalLogger so that it would apply to all logging from the agent.

...eclarativeConfigTest/java/io/opentelemetry/javaagent/instrumentation/methods/MethodTest.java

.../src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentation.java

...avaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodsConfig.java

...ain/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentationModule.java

laurit · 2025-06-18T09:50:47Z

@trask I think we can remove this from 2.17 milestone. As it is the declarative configuration isn't fully functional with the agent anyway.

zeitlinger · 2025-06-18T11:54:40Z

@trask I think we can remove this from 2.17 milestone. As it is the declarative configuration isn't fully functional with the agent anyway.

well - it should be fully functional with this PR - but there's no need to rush

zeitlinger · 2025-06-18T13:19:56Z

@laurit can you check again?

instrumentation/methods/javaagent/build.gradle.kts

.../src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentation.java

zeitlinger · 2025-06-20T14:35:03Z

@laurit can you check again?

laurit · 2025-06-20T17:59:31Z

...ain/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentationModule.java

+    List<TypeInstrumentation> list =
+        methods != null ? MethodsConfig.parseDeclarativeConfig(methods) : parseConfigProperties();
+    if (list.isEmpty()) {
+      return Collections.singletonList(new MethodInstrumentation(null, Collections.emptyMap()));


I think there should be a comment here elaborating whey we are doing it.
Also I thought maybe it would be nicer if instead of Map<String, SpanKind> we'd use Map<SpanKind, Collection<String>>. Tried it out in
zeitlinger#6 wdyt?

zeitlinger requested a review from a team as a code owner June 10, 2025 15:00

github-actions bot added the test native This label can be applied to PRs to trigger them to run native tests label Jun 13, 2025

jack-berg approved these changes Jun 16, 2025

View reviewed changes

zeitlinger force-pushed the span-kind-for-method-instrumentation branch from 24bdfa3 to 63bfcbf Compare June 16, 2025 16:07

zeitlinger mentioned this pull request Jun 16, 2025

Support declarative config for spring starter #14048

Open

zeitlinger added this to the v2.17.0 milestone Jun 17, 2025

zeitlinger commented Jun 17, 2025

View reviewed changes

zeitlinger force-pushed the span-kind-for-method-instrumentation branch from 63bfcbf to 02fb7b6 Compare June 17, 2025 12:19

laurit reviewed Jun 18, 2025

View reviewed changes

...eclarativeConfigTest/java/io/opentelemetry/javaagent/instrumentation/methods/MethodTest.java Show resolved Hide resolved

laurit reviewed Jun 18, 2025

View reviewed changes

.../src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentation.java Outdated Show resolved Hide resolved

laurit reviewed Jun 18, 2025

View reviewed changes

...avaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodsConfig.java Outdated Show resolved Hide resolved

laurit reviewed Jun 18, 2025

View reviewed changes

...ain/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentationModule.java Outdated Show resolved Hide resolved

zeitlinger removed this from the v2.17.0 milestone Jun 18, 2025

zeitlinger mentioned this pull request Jun 18, 2025

Spring starter declarative config #14062

Open

zeitlinger added 2 commits June 19, 2025 18:20

add span kind support for method instrumentation

9bd77a5

add span kind support for method instrumentation

6997acc

zeitlinger added 11 commits June 19, 2025 18:20

use declarative config

fbad8bb

format

66e8101

fix

6e3802b

fix

9296cc3

add test brigde for exporters

37256d6

add test bridge for exporters

2754cd5

separate test bridge for agent

c8b2613

reduce logging (gets it the way of tests)

2544ddd

simplify

cf806be

remove redundant imports

7b7de34

pr feedback

7a819f7

zeitlinger force-pushed the span-kind-for-method-instrumentation branch from a0f980d to 7a819f7 Compare June 19, 2025 16:55

laurit reviewed Jun 20, 2025

View reviewed changes

instrumentation/methods/javaagent/build.gradle.kts Outdated Show resolved Hide resolved

laurit reviewed Jun 20, 2025

View reviewed changes

.../src/main/java/io/opentelemetry/javaagent/instrumentation/methods/MethodInstrumentation.java Outdated Show resolved Hide resolved

zeitlinger added 3 commits June 20, 2025 10:54

cleanup

48ea785

simplify test

76351c5

simplify bytecode transform

58a2322

zeitlinger changed the title ~~Span kind for method instrumentation~~ Span kind for method instrumentation / Declarative configuration tooling Jun 20, 2025

zeitlinger added this to Declarative Configuration: Java Jun 20, 2025

zeitlinger moved this to In Progress in Declarative Configuration: Java Jun 20, 2025

zeitlinger self-assigned this Jun 20, 2025

This was referenced Jun 20, 2025

Vendor specific instrumentation config options handling #14016

Open

Schema for the instrumentation section of declarative config (maybe including vendor specific) #14082

Open

Matrix test for declarative config #14084

Open

revert change

9934fff

laurit reviewed Jun 20, 2025

View reviewed changes

Span kind for method instrumentation / Declarative configuration tooling #14014

Are you sure you want to change the base?

Span kind for method instrumentation / Declarative configuration tooling #14014

Uh oh!

Conversation

zeitlinger commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeitlinger commented Jun 10, 2025

Uh oh!

trask commented Jun 10, 2025

Uh oh!

zeitlinger commented Jun 10, 2025

Uh oh!

trask commented Jun 10, 2025

Uh oh!

zeitlinger commented Jun 13, 2025

Uh oh!

zeitlinger commented Jun 13, 2025

Uh oh!

zeitlinger commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeitlinger commented Jun 16, 2025

Uh oh!

zeitlinger commented Jun 16, 2025

Uh oh!

zeitlinger commented Jun 16, 2025

Uh oh!

jack-berg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laurit commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

laurit commented Jun 18, 2025

Uh oh!

zeitlinger commented Jun 18, 2025

Uh oh!

zeitlinger commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

zeitlinger commented Jun 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zeitlinger commented Jun 10, 2025 •

edited

Loading

zeitlinger commented Jun 13, 2025 •

edited

Loading