feat(instrumentation-openai): add instrumentation of openai SDK

[anuraaga]

Which problem is this PR solving?

• openai SDK usage should have gen ai conventions but there is no instrumentation yet

Short description of the changes

• Adds instrumentation targeting openai SDK 4 for chat and embeddings

Future PRs will add support for responses API and SDK 5

/cc @trentm if you can help review
/cc @lmolkova because I heard you are also interested in this instrumentation. Notably, I'd like this instrumentation to exist upstream so people can use it but need potential component owners I believe

This instrumentation is adapted from the one in https://github.com/elastic/elastic-otel-node/tree/main/packages/instrumentation-openai

[anuraaga]

Could use some help debugging this

[codecov]

Codecov Report

❌ Patch coverage is 87.96296% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.73%. Comparing base (9d90781) to head (c035ed2).

Files with missing lines	Patch %	Lines
...ages/instrumentation-openai/src/instrumentation.ts	90.90%	25 Missing ⚠️
packages/instrumentation-openai/src/utils.ts	53.33%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2941      +/-   ##
==========================================
- Coverage   89.79%   89.73%   -0.07%     
==========================================
  Files         188      191       +3     
  Lines        9222     9546     +324     
  Branches     1900     1977      +77     
==========================================
+ Hits         8281     8566     +285     
- Misses        941      980      +39     

Files with missing lines	Coverage Δ
packages/instrumentation-openai/src/semconv.ts	100.00% <100.00%> (ø)
packages/instrumentation-openai/src/utils.ts	53.33% <53.33%> (ø)
...ages/instrumentation-openai/src/instrumentation.ts	90.90% <90.90%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[trentm]

Thanks for starting this, Anuraag. This is just a quick start at looking. I haven't looked at src/ or test/ yet.

[seemk]

If one has set up a global logging provider, is it possible to opt out of the logs emitted from this instrumentation?

[trentm]

@seemk I think one could do so by setting a NoopLoggerProvider on the instrumentation via instr.setLoggerProvider(noopLoggerProvider).

But are you asking for something different than that?

I haven't been following the GenAI semconv discussions of late. I vaguely recall that there was discussion of making it user-optional whether GenAI instrumentations used log-events or span attributes. Is that what you are referring to?

[anuraaga]

Yeah - from what I understand, the conventions imply that when logs are enabled, logs are always emitted - this is how it currently works in the bedrock instrumentation of this repo, as well as genai in python and java. If it seems like there should be more knobs, we'd need to raise changes to the semconv but for now this implements the current practice.

[trentm]

Another round of review, looking at the implementation this time. Looks good! Thanks for all the type improvements.

I haven't yet looked at the tests.

[trentm]

Tests look good to me (with some comments below). I didn't comb through all 3000 lines; about half of them.

[trentm]

A couple more things: test-all-versions, and openai@5 support.

[trentm]

This LGTM.

Not sure if you were going to wait for feedback from the MS folks that might help and possibly be component owners.

Before or when being taken out of draft, this PR will need to add one or more component owners to ".github/component_owners.yml". I could be one if needed, at least for starters.

[trentm]

FWIW, I've started a PR to eventually add EVENT_* exports from the semantic-conventions package.
open-telemetry/opentelemetry-js#5832