Zero-Code OpenTelemetry: Instrumenting a Real Distributed System

The fastest way to get traces, metrics, and logs out of a Python microservices app is to change none of its code. OpenTelemetry's zero-code auto-instrumentation does exactly that — and the seam only becomes visible when you apply it to a system that was genuinely built without observability in mind.

So we built one on purpose.

Instrument something that already works

Most instrumentation tutorials start with an app that was born instrumented, so you never see the join. We did it the other way. Fathom is a real distributed system — a customer-facing edge tier on one machine, a core-banking tier with a double-entry ledger on another, plus Postgres. It signed up hundreds of customers and passed a load test with its ledger proven consistent under concurrency before a single span existed.

That order matters. Telemetry from a system that was correct and load-tested first is trustworthy precisely because the thing being observed stands on its own. It is not a demo wired to look good in a dashboard.

The architecture

Fathom splits into two tiers on two hosts, so a transfer crosses the network — a real topology, not localhost.

A single transfer flows gateway → auth → payments → fraud → ledger, moving money atomically. That is the path we want telemetry to make visible.

Zero-code instrumentation, in three steps

No spans in the code. No providers to wire. No instrumentation module to import. Three moves, none of which touch a service's business logic.

1. Install the distro and bootstrap the instrumentors

opentelemetry-bootstrap inspects the environment and installs an instrumentor for every library it finds — Flask, requests, psycopg2, redis — so you never hand-pick them.

pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http
opentelemetry-bootstrap -a install

2. Run the app under the wrapper

Prefix the start command with opentelemetry-instrument. That is the only difference from running it uninstrumented.

opentelemetry-instrument gunicorn --chdir services/$SERVICE --bind 0.0.0.0:$PORT app:app

3. Point it at your backend with environment variables

OTEL_SERVICE_NAME=ledger-service
OTEL_RESOURCE_ATTRIBUTES=service.namespace=fathom
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingress.rocketgraph.app
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20$API_KEY
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true

That is the whole integration. Every service's code is unchanged — not even an import.

The gotcha: logs need a log level

Traces and metrics appeared immediately. Logs did not — and this is where most people conclude, wrongly, that they need a custom log bridge. They don't.

The wrapper already attaches an exporting handler to the root logger. The catch is subtler: Python's root logger defaults to WARNING, so INFO business logs are dropped before the handler ever sees them. One line fixes it, no bridge required:

import logging
logging.getLogger("fathom").setLevel(logging.INFO)   # once, in shared config

Scope it to your own logger namespace so you export your business events without dragging in noisy third-party INFO chatter. Once the level is right, the services just log normally and the records flow — each one already carrying the active trace_id.

log.info("transfer declined by fraud: user=%s amount_minor=%d score=%s", user_id, amount, score)

What lands in the backend

The same load test — hundreds of concurrent transfers — now produces a fully connected picture, three signals wide:

Click a declined transfer's trace and its logs are one query away — fraud declined … score=82 sitting exactly where the span shows the decision. That correlation is the entire point of shipping all three signals to one place: traces tell you where, logs tell you why.

Getting more metrics — still no code

Zero-code auto-instrumentation gives you only HTTP metrics out of the box — http.server.duration, http.client.duration, and http.server.active_requests. The reason is simple: opentelemetry-bootstrap installs instrumentors only for the libraries it detects (Flask, requests), and those are the metrics they emit.

Here is the part people miss: getting more is also no code — you add a package, not code. The opentelemetry-instrument wrapper auto-discovers any instrumentor installed in the environment (through entry points), so enriching your metrics is a one-line dependency change:

  opentelemetry-distro
  opentelemetry-exporter-otlp-proto-http
+ opentelemetry-instrumentation-system-metrics

That single line adds ~15–20 host and process metrics per service — CPU, memory, disk I/O, network, garbage collection, thread and file-descriptor counts — with no Meter, no create_counter, nothing touched in the application. Rebuild, and they flow.

So metrics fall into three buckets, and only one of them needs code:

Where Rocketgraph fits

Fathom is our reference application, and it exports over plain OpenTelemetry (OTLP/HTTP) — the same path any customer's own services use. Nothing about the instrumentation is Rocketgraph-specific; that is deliberate. You instrument once, with open standards, and point the endpoint wherever you like.

Rocketgraph is where that telemetry becomes cheap to keep and fast to search: OTLP in, object-storage economics underneath, usage-based pricing with no per-host fees, and AI agents that triage issues automatically. If you are weighing options, our cost comparison of Datadog alternatives is written to be useful even if you never send us a byte.