Generative AI instrumentation for Python
If your service uses Python LLM frameworks and you want observability of generative AI operations, then you can optionally install additional packages while installing the Python library.
Before you start
Verify your service environment and install the Python library. Before loading, install instrumentation based on your environment and the list below.
Additional LLM instrumentation packages
Install additional OpenTelemetry-based instrumentation packages for the LLM framework(s) used by your service to export signals from generative AI operations. For example, do pip install opentelemetry-instrumentation-genai-openai to instrument OpenAI operations. Note that these packages are still under rapid development and may introduce breaking changes in near-future releases. You can also install OpenLLMetry instrumentation packages which are compatible with the Python library.
| LLM Framework | OpenTelemetry instrumentation |
|---|---|
| Anthropic | opentelemetry-instrumentation-genai-anthropic |
| Google GenAI | opentelemetry-instrumentation-google-genai |
| Langchain | opentelemetry-instrumentation-genai-langchain |
| OpenAI | opentelemetry-instrumentation-genai-openai |
| OpenAI Agents | opentelemetry-instrumentation-genai-openai-agents |
How to enable input/output capture in telemetry
Full LLM conversations may include sensitive data and take up large amounts of storage. See precautions below for more information.
To capture the inputs, outputs, and tool definitions of your service’s LLM framework requests, you may opt into adding all of them as span attributes with the environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=span_only. This configuration is supported by all of the above LLM instrumentation packages. The opt-in writes the additional attributes gen_ai.input.messages and gen_ai.output.messages, and enhances gen_ai.tool.definitions with description and fields if applicable.
The value of OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is not case-sensitive. For example, span_only and SPAN_ONLY are both valid.
What type of data is captured
SolarWinds recommends setting OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT for development or staging environments, when telemetry volume is small and privacy is not a concern. Inputs, outputs, and tool definitions may include Personally Identifiable Information (PII) and other sensitive data such as usernames, emails, ticket numbers, business logic, API structures, logs with IP addresses, session IDs, and tokens. Full conversations also take up more space, especially if they are multi-turn or contain any images. See the OpenTelemetry Generative AI semantic conventions for guidelines and other options.
Anonymization to redact inputs in requests and span attributes
If your service uses anonymization to redact LLM inputs, then some span attribute values can also be redacted at the time of recording. Please verify your own service in a secure environment to test if the redaction will meet your compliance needs. The following open source frameworks can provide some guard rails:
-
LLM Guard (No longer in active development.)