↓ Skip to main content
jdhao’s digital space
  1. Posts/

OpenTelemetry Tracing for Your FastAPI application (1)

·512 words·3 mins·
Python FastAPI OpenTelemetry
Table of Contents

With OpenTelemetry, you can easily trace a single request across multiple services in a distributed system. In this post, I would like to share how you can set up OpenTelemetry tracing for your FastAPI application.

Set up OpenTelemetry tracing for FastAPI application
#

OpenTelemetry(OTel) separates the packages into fine-grained levels. For beginners, it might be very daunting.

To set up tracing for a FastAPI application, we need to install the following packages (via uv):

uv add \
    "opentelemetry-api" \
    "opentelemetry-sdk" \
    "opentelemetry-exporter-otlp" \
    "opentelemetry-instrumentation-fastapi"

Package opentelemetry-api is a specification, and package opentelemetry-sdk implements that specification, see [this post][otel-api-vs-sdk] for an explanation of the differences.

The package [opentelemetry-exporter-otlp][exporter-otlp-pypi] implements how to export the OTel spans to a backend, using either HTTP or gRPC.

The package opentelemetry-instrumentation-fastapi is to add instrumentation for the FastAPI application: when a request is sent to an endpoint, a new span will be automatically generated, the trace id will be extracted from traceparent header.

Here is how to set up tracing for your FastAPI application.

from fastapi import FastAPI
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter


def configure_tracing(app: FastAPI):
    resource = Resource.create(
        attributes={
            "service.name": "my-demo-service",
        }
    )

    tracer_provider = TracerProvider(
        resource=resource,
    )

    # a tracer_provider can have multiple span processors
    otlp_span_processor = BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="http://localhost:4317",
            insecure=True,
        ),
    )
    console_span_processor = BatchSpanProcessor(ConsoleSpanExporter())

    tracer_provider.add_span_processor(otlp_span_processor)
    tracer_provider.add_span_processor(console_span_processor)

    trace.set_tracer_provider(tracer_provider)

    # configure fatapi instrumentation
    FastAPIInstrumentor.instrument_app(
        app=app,
        excluded_urls="health,metric"
    )

In OpenTelemetry, the tracer can have multiple different span processors. The span processor will produce spans for different destinations. For example, the OTLPSpanExporter will send the generated spans to the specified endpoint. The ConsoleSpanExporter will send the span info to the terminal/console.

Create custom spans in your code
#

Now that you have set up OTel tracing, you can create custom spans in your code to track the important steps.

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

@app.get("/my-endpoint")
def do_something():
    with tracer.start_as_current_span("step 1") as span:
        process_1()

    with tracer.start_as_current_span("step 2") as span:
        process_2()

    with tracer.start_as_current_span("step 3") as span:
        process_3()


def process_1():
    print("doing some work")
    with tracer.start_as_current_span("step 1.1") as span:
        process_11()

    with tracer.start_as_current_span("step 1.2") as span:
        process_12()

In the above code, we are using the context manager in python to create a span. This has very nice property that nested span gets the context from its parent span cleanly and automatically.

Check your traces in backend/UI
#

To check the span generated for a request and make sense of them, you need a back to store and visualize the spans. There are a lot of open-source and commercial tools for this. Among them:

We choose jaeger here, since it has a more feature rich UI. First, start the jaeger docker image locally:

# see https://opentelemetry.io/docs/languages/python/exporters/#jaeger
docker run --rm \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 9411:9411 \
  jaegertracing/jaeger:latest

Run your FastAPI application and send some requests to the endpoint. Open your browser and go to localhost:16686, you will be able to see the Jaeger UI and explore the generated traces.

Related