Skip to content

Embedded OTel Collector as a subprocess POC #4130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Embedded OTel Collector as a subprocess POC #4130

wants to merge 1 commit into from

Conversation

thampiotr
Copy link
Contributor

@thampiotr thampiotr commented Aug 5, 2025 •
edited
Loading

POC WIP

example of start + sigint termination

thampiotr@MacWork ~/w/alloy (thampiotr/otel-subprocess-poc) [2]> make binaries && OTELCOL_CONFIG_PATH=./example-config-otel.yaml OTELCOL_BINARY_PATH=./build/otelcol ./build/alloy run ./example-config.alloy
GOEXPERIMENT= GOOS=darwin GOARCH=arm64 GOARM= CGO_ENABLED=1 go build  -ldflags "-X github.com/grafana/alloy/internal/build.Branch=thampiotr/otel-subprocess-poc -X github.com/grafana/alloy/internal/build.Version=v1.11.0-devel+dirty -X github.com/grafana/alloy/syntax/internal/stdlib.Version=v1.11.0-devel+dirty -X github.com/grafana/alloy/internal/build.Revision=5bbae9afb -X github.com/grafana/alloy/internal/[email protected] -X github.com/grafana/alloy/internal/build.BuildDate=2025-08-05T16:14:23Z" -tags "" -o build/alloy .
# github.com/grafana/alloy
ld: warning: ignoring duplicate libraries: '-lproc'
pushd otelcol && GOEXPERIMENT= GOOS=darwin GOARCH=arm64 GOARM= CGO_ENABLED=1 go build  -ldflags "-X github.com/grafana/alloy/internal/build.Branch=thampiotr/otel-subprocess-poc -X github.com/grafana/alloy/internal/build.Version=v1.11.0-devel+dirty -X github.com/grafana/alloy/syntax/internal/stdlib.Version=v1.11.0-devel+dirty -X github.com/grafana/alloy/internal/build.Revision=5bbae9afb -X github.com/grafana/alloy/internal/[email protected] -X github.com/grafana/alloy/internal/build.BuildDate=2025-08-05T16:14:23Z" -tags "" -o build/otelcol . && popd
~/workspace/alloy/otelcol ~/workspace/alloy
~/workspace/alloy
ts=2025-08-05T16:15:35.581257Z level=info msg="starting otelcol runtime" service=otelcol path=/Users/thampiotr/workspace/alloy/build/otelcol
ts=2025-08-05T16:15:35.581309Z level=info "boringcrypto enabled"=false
ts=2025-08-05T16:15:35.581313Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster
ts=2025-08-05T16:15:35.581316Z level=info msg="running usage stats reporter"
ts=2025-08-05T16:15:35.581318Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57
ts=2025-08-05T16:15:35.581331Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=tracing duration=1.584µs
ts=2025-08-05T16:15:35.581334Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=otel duration=500ns
ts=2025-08-05T16:15:35.581337Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=logging duration=149.334µs
ts=2025-08-05T16:15:35.581554Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=remotecfg duration=188.167µs
ts=2025-08-05T16:15:35.581568Z level=info msg="applying non-TLS config to HTTP server" service=http
ts=2025-08-05T16:15:35.581572Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=http duration=7.417µs
ts=2025-08-05T16:15:35.581577Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=cluster duration=458ns
ts=2025-08-05T16:15:35.581582Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=livedebugging duration=1.791µs
ts=2025-08-05T16:15:35.581586Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=ui duration=292ns
ts=2025-08-05T16:15:35.581641Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 node_id=labelstore duration=20.625µs
ts=2025-08-05T16:15:35.581645Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=c4c536d5b7c099de1588c4c2294c7d57 duration=600.458µs
ts=2025-08-05T16:15:35.581727Z level=info msg="scheduling loaded components and services"
ts=2025-08-05T16:15:35.582005Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s
ts=2025-08-05T16:15:35.582292Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=MacWork.local
ts=2025-08-05T16:15:35.582496Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345
2025-08-05T17:15:35.655+0100    info    [email protected]/service.go:199 Setting up own telemetry...     {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
2025-08-05T17:15:35.656+0100    info    builders/builders.go:26 Development component. May change in the future.        {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}
2025-08-05T17:15:35.657+0100    info    [email protected]/service.go:259 Starting otelcol...     {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}, "Version": "0.1", "NumCPU": 8}
2025-08-05T17:15:35.657+0100    info    extensions/extensions.go:41     Starting extensions...  {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
2025-08-05T17:15:35.657+0100    info    [email protected]/otlp.go:117       Starting GRPC server    {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}, "otelcol.component.id": "otlp", "otelcol.component.kind": "receiver", "endpoint": "localhost:4317"}
2025-08-05T17:15:35.658+0100    info    [email protected]/otlp.go:175       Starting HTTP server    {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}, "otelcol.component.id": "otlp", "otelcol.component.kind": "receiver", "endpoint": "localhost:4318"}
2025-08-05T17:15:35.658+0100    info    [email protected]/service.go:282 Everything is ready. Begin running and processing data. {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
^Cinterrupt received
2025-08-05T17:15:40.548+0100    info    [email protected]/collector.go:358       Received signal from OS {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}, "signal": "interrupt"}
ts=2025-08-05T16:15:40.548791Z level=info msg="sending interrupt signal to otelcol runtime" service=otelcol
ts=2025-08-05T16:15:40.548794Z level=error msg="failed to start reporter" err="context canceled"
2025-08-05T17:15:40.548+0100    info    [email protected]/service.go:324 Starting shutdown...    {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
ts=2025-08-05T16:15:40.549199Z level=info msg="node exited without error" node=otel
ts=2025-08-05T16:15:40.549213Z level=info msg="node exited without error" node=ui
ts=2025-08-05T16:15:40.54922Z level=info msg="node exited without error" node=labelstore
ts=2025-08-05T16:15:40.549339Z level=info msg="node exited without error" node=remotecfg
ts=2025-08-05T16:15:40.549285Z level=info msg="node exited without error" node=livedebugging
ts=2025-08-05T16:15:40.54936Z level=info msg="http server closed" service=http addr=memory err="http: Server closed"
ts=2025-08-05T16:15:40.549353Z level=info msg="http server closed" service=http addr=127.0.0.1:12345 err="http: Server closed"
ts=2025-08-05T16:15:40.549493Z level=info msg="node exited without error" node=http
ts=2025-08-05T16:15:40.549634Z level=info msg="node exited without error" node=cluster
2025-08-05T17:15:40.549+0100    info    extensions/extensions.go:69     Stopping extensions...  {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
2025-08-05T17:15:40.550+0100    info    [email protected]/service.go:338 Shutdown complete.      {"resource": {"service.instance.id": "16280aaf-3a65-45c1-b05d-2e01997e66f1", "service.name": "otelcol", "service.version": "0.1"}}
ts=2025-08-05T16:15:40.650079Z level=info msg="otelcol runtime exited" service=otelcol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@thampiotr