Issue with OpenTelemetry Metrics Instrumentation in Apache Airflow

[SharookTailorNorstella]

Hi Airflow Team,

I hope you're doing well.
I'm reaching out to report an issue regarding the OpenTelemetry (OTel) metrics integration in Apache Airflow. As per the guidance from New Relic support, the OTel metrics exposed by Airflow are not maintained by New Relic, and they advised opening an issue here.

Issue Summary:
We are attempting to monitor Apache Airflow using OpenTelemetry and send metrics to New Relic, following the documentation here:
https://docs.newrelic.com/docs/infrastructure/other-infrastructure-integrations/monitoring-airflow-ot/

However, we have encountered issues with the instrumentation. We want step by step proper starting process for this.
We want to ensure proper observability of Airflow using OpenTelemetry and would appreciate any guidance or support on how to resolve this issue.

[potiuk]

Just to clarify - Airflow is a an open-source project run by volunteers. And you can use it for free - but also without any guarantees or promise of support. Your best way is to read the docs and if it is not enough, look at the sources and debug and solve (and then possibly contribute back documentation updates if during your investigation you will find step-by-step instructions and possibly even contribute back some code fixes. We do like here in the community of more than 3600 contributors that our users are becoming contributors.

Likely there are also paid consultancies you can seek for, if you expect "support". Other than that the discussion here might be commented by someone who will have some experience and will be able to help you, but your expectations should not be that you should "expect" someone magically finding and spending their time on solving your problem. They might volunteer their time - but this is on their own discretion - and usually, when you ask "I need your help in my problem" - you are really following this pattern https://dontasktoask.com/

So you are much more likely to get help if you explain all the efforts you've done, all the errors you encountered and ideally every time you have a problem you explain what it is, what you've already made to fix it, show some logs and explain your problem - that significantly increases your chances that someone will actually decide to spend their time outside of their day job and rather than spend it with their family will decide to spend some of their time on helping you.

Generally speaking also when you are actually "paying" New Relic (I assume you do) - this seems a bit outrageous from their side that they expect an open-source volunteers to help to solve a problem of customer that pays them for support, so I would rather advice you to come back to them and look at your agreement and SLA and see what kind of service you are paying for.

If you look at our licence - you will see that you can do whatever with our software without paying anything but it also means no guarantees and support.

I hope that puts it in the right perspective and I hope that you will get some help - after you follow those guidelines (because I am quite sure "want step-by-step procedure" will not have big chances to succeed - unless somoene actually did it and have such step-by-step handy.