Documentation forSolarWinds Observability

Configure the Python Library

See the following sections for information about the configuration options available for the Python Library and how to set them. The Service key is the only required configuration; all other configurations are optional.

How to set configuration options

Some configuration options to control the Python Library's behavior can be set in the JSON config file, some can be set in system environment variables, and some can be set in either the config file or in system environment variables. See the complete list of configuration options to determine where you can set the configuration option.

The location of the JSON config file is defined in the SW_APM_CONFIG_FILE environment variable. For an example of some configuration options defined in a config file, see Config File Example.

If the same configuration option is set in both a config file and an environment variable, the setting is applied with the following precedence:

  1. Environment variable
  2. Property in config file
  3. Default value

If an invalid configuration option is defined in one of these places, the next defined setting in the above order of precedence is used.

If a configuration option expects a Boolean value, the following inputs are accepted: true, false. Boolean values are case-insensitive.

Complete list of configuration options

The following is the complete list of instrumentation configuration options:

Service key

The Service key is used to identify your account and the service being instrumented. It is shown in the Add Data dialog when you add the new service.

It is a required configuration and should be in the form of YourApiToken:YourServiceName. Replace YourApiToken with the SolarWinds Observability API token (ingestion type) generated for this service, and replace YourServiceName with your chosen name for this service. See API Tokens.

The service name is also called the entity's service ID in SolarWinds Observability. The service name can also be seen in the Overview tab in the service entity's Entity Explorer details view. Editing the display name of the service entity in SolarWinds Observability does not affect the Service key.

YourServiceName in the Service key takes lower precedence than OTEL_SERVICE_NAME or OTEL_RESOURCE_ATTRIBUTES, if either is set.

System environment variable

SW_APM_SERVICE_KEY

Required

Yes

Default

None

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export SW_APM_SERVICE_KEY="ABC123abcABC123abcABC123abcABC123abcABC123abcABC123abcABC123abcABC123ab:my-service"

Collector

The collector is the ingestion endpoint the library connects to and exports data to. It should be defined using the format host:port. To determine the endpoint for your organization, see Data centers and endpoint URIs. Port is optional and defaults to 443.

System environment variable

SW_APM_COLLECTOR

Config file property

collector

Required

No

Default

apm.collector.cloud.solarwinds.com:443

History

Introduced with version 0.2.1 of the Python Instrumentation Library.

Example

  • System environment variable

    export SW_APM_COLLECTOR=YourHost:PortNumber

Host name alias

An optional descriptive host name can be used to easily identify the host. If configured, the host name alias is displayed and can be used as a filter in the Traces Explorer.

System environment variable

SW_APM_HOSTNAME_ALIAS

Config file property

hostnameAlias

Required

No

Default

None

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export SW_APM_HOSTNAME_ALIAS=apm-host-east

Trigger tracing mode

This enables or disables trigger tracing for the service. Setting it to disabled turns off support for trigger trace, which might affect DEM visibility into the service.

System environment variable

SW_APM_TRIGGER_TRACE

Config file property

triggerTrace

Required

No

Default

enabled

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Trusted certificate path

The library uses the trusted CA certificates configured for the underlying native extension library to verify the TLS connection to the collector. To override the default, define the trusted certificate path configuration option with an absolute path to a specific trusted certificate file in PEM format.

System environment variable

SW_APM_TRUSTEDPATH

Config file property

trustedpath

Required

No

Default

None

History

Introduced with version 0.2.1 of the Python Instrumentation Library.

Example

export SW_APM_TRUSTEDPATH=/path/to/cacert.crt

Logging level

By default, the Python Library and its underlying native extension library send debug output to stderr. For most environments, stderr is included in the web application's logs. To troubleshoot the Python Library, enable debug-level logging (4 or higher) and check the messages. Python Library logging can be disabled by setting the log level to -1.

System environment variable

SW_APM_DEBUG_LEVEL

Config file property

debugLevel

Valid Values

-1 disable
0 fatal
1 error
2 warning
3 info
4 debug low
5 debug medium
6 debug high

Required

No

Default

2 (warning)

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export SW_APM_DEBUG_LEVEL=-1

Logging output filepath

By default, the Python Library and its underlying native extension library send log output to stderr. To change output destination of the Python Library to a file instead, set the filepath of where the logs will be written. If the parent directory does not exist the library will attempt to create it.

System environment variable

SW_APM_LOG_FILEPATH

Config file property

logFilepath

Required

No

Default

None

History

Introduced with version 1.1.0 of the Python Instrumentation Library.

Example

export SW_APM_LOG_FILEPATH=/path/to/messages.log

EC2 Metadata Timeout

The Python Library is able to detect if it is running on an AWS EC2 or OpenStack instance by querying Instance Metadata. The timeout (in milliseconds) for retrieving the metadata can be set through this configuration. It is safe to set this value to 0 for apps that do not run on EC2/OpenStack.

System environment variable

SW_APM_EC2_METADATA_TIMEOUT

Config file property

ec2MetadataTimeout

Valid values

This field expects an integer between 0 and 3000

Required

No

Default

1000

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export SW_APM_EC2_METADATA_TIMEOUT=500

Use HTTP proxy

The agent can connect to the SolarWinds APM collector through a proxy. The config value should be either http://<proxyHost>:<proxyPort> for a proxy server that does not require authentication, or http://<username>:<password>@<proxyHost>:<proxyPort> for a proxy server that requires Basic authentication. HTTP is the only type of connection supported between the Python Library and the proxy. However, the library traffic to SolarWinds Observability is still encrypted using SSL/TLS.

SolarWinds recommends using the Python Library-specific configuration. However, if a Python Library-specific configuration is not set, its underlying network library uses a system-wide proxy defined in the environment variables grpc_proxy, https_proxy, or http_proxy. Refer to gRPC environment variables for more information.

System environment variable

SW_APM_PROXY

Config file property

proxy

Valid Values

Any string starting with http://

This field is case-sensitive.

Required

No

Default

None

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Examples

export SW_APM_PROXY="http://my-proxy:3306"

Tracing mode

Tracing can be disabled for the service. When disabled, requests are no longer monitored, and traces and metrics are not sent. The trace context is still injected in the HTTP request and response headers.

To completely disable the library, remove the library. See Uninstall the Python Library.

System environment variable

SW_APM_TRACING_MODE

Config file property

tracingMode

Required

No

Possible values

disabled Does not continue existing traces or start new ones, and does not report metrics for requests.
enabled (Default) Create or continue traces (subject to sampling and rate limits), and always report metrics for requests.

Default

enabled

History

Introduced with version 0.10.0 of the Python Instrumentation Library.

Example

export SW_APM_TRACING_MODE=disabled

Tracing mode per transaction (Enable or disable tracing for specific requests)

Defines the tracing mode for requests at a granular level, based on whether the requests match the specified criteria. Tracing is enabled or disabled for requests based on the tracing property of the matching criteria, superseding tracing mode settings applied at the service level. Requests with "tracing" : "disabled" are not monitored, and do not create traces or send metrics. Requests with "tracing" : "enabled" define the requests that are monitored even when tracing is disabled at the service level.

Config file property

transactionSettings

Possible values

Config value should be an array of JSON entries where each entry must contain a matcher and the desired tracing mode:

  • regex (required): The value should be a string representation of a Python regular expression (see Python regular expression module). Because the regular expression is defined in a JSON string, use double backslashes to escape your Python regular expressions. The first backslash ensures the second backslash is retained as a Python escape character in the regular expression and still escapes whatever follows in the regex pattern.

  • tracing (required): The value should either be enabled or disabled.

Entries are applied in the order defined in the JSON array. To find a match, each entry is compared against the Span kind and name (concatenated with a colon, for example CLIENT:SELECT). The first matching entry for a request is taken. Support for HTTP URL-based filtering is coming soon.

Required

No

Default

None. No transaction-specific tracing mode.

History

Introduced with version 0.10.0 of the Python Instrumentation Library.

Example

  • Config file property

    "transactionSettings": [
    {
        "regex": "CLIENT:GET",
        "tracing": "disabled"
    },
    {
        "regex": "INTERNAL:Task\\.run",
        "tracing": "disabled"
    },
    {
        "regex": "SERVER:my/[a-zA-Z0-9-<>:]*/path/",
        "tracing": "disabled"
    }
]

Transaction naming

The Python Library provides the option to assign a custom transaction name to all traces. This can be used to, for example, target specific instrumented lambda functions. To configure a name, set the SW_APM_TRANSACTION_NAME environment variable as described below.

Configuration of a custom transaction name is superseded by any use of the SDK to set_transaction_name, described in Python set_transaction_name.

System environment variable

SW_APM_TRANSACTION_NAME

Config file property

transactionName

Valid values

Any string representing a valid transaction name.

This field is case-sensitive.

Required

No

Default

None

History

Introduced with version 0.18.0 of the Python Instrumentation Library.

Example

export SW_APM_TRANSACTION_NAME=my-cool-request

Disable the library

The Python Library provides the option to completely disable the instrumentation without explicitly removing the library from your application. To disable the library, set the SW_APM_AGENT_ENABLED environment variable as described below.

System environment variable

SW_APM_AGENT_ENABLED

Config file property

agentEnabled

Valid values

This field is case-insensitive.

false

Do not instrument the application (that is, do not generate any trace data)
true Normal tracing operation

Required

No

Default

true

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export SW_APM_AGENT_ENABLED=false

Disable specific instrumentation libraries

The Python Library provides the option to disable the instrumentation of specific Python libraries, while still instrumenting other libraries in your application. To disable specific libraries, set the OTEL_PYTHON_DISABLED_INSTRUMENTATIONS environment variable as described below.

System environment variable

OTEL_PYTHON_DISABLED_INSTRUMENTATIONS

Required

No

Default

None

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=urllib3,asyncpg

Customize OpenTelemetry propagators

The Python Library sets the OpenTelemetry propagators necessary for distributed tracing by default, so no changes to the propagators are required for the library to work. If you want additional propagators, you can install and use them. For example:

pip install my-custom-propagator

After installing the propagators, set the OTEL_PROPAGATORS environment variable as described below.

System environment variable

OTEL_PROPAGATORS

Valid values

A string representing a list of valid propagator entry point names, delimited by commas.

Both tracecontext and solarwinds_propagator must be included, and tracecontext must come before solarwinds_propagator.

The optional baggage propagator is included by default. If baggage is specified explicitly, it must come before solarwinds_propagator.

  • Introduced with version 0.15.0 of the Python Instrumentation Library.

This field is case-sensitive.

Required

No

Default

tracecontext,baggage,solarwinds_propagator

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export OTEL_PROPAGATORS="my_custom_propagator,tracecontext,solarwinds_propagator"

Customize OpenTelemetry exporters

The Python Library sets an OpenTelemetry exporter for SolarWinds Observability platform by default, so no changes to the exporters are necessary for the library to work. If you want additional exporters, you can install and use them. For example:

pip install opentelemetry-exporter-otlp-proto-grpc

After installing the exporters, set the OTEL_TRACES_EXPORTER environment variable as described below.

System environment variable

OTEL_TRACES_EXPORTER

Valid values

A string representing a list of valid exporter entry point names, delimited by commas.

solarwinds_exporter must be included. The order of the exporters does not matter.

This field is case-sensitive.

Required

No

Default

solarwinds_exporter

History

Introduced with version 0.1.0 of the Python Instrumentation Library.

Example

export OTEL_TRACES_EXPORTER="solarwinds_exporter,otlp_proto_grpc"

Customize OpenTelemetry service name

The Python Library provides the option to pass a service name to OpenTelemetry environment variable. This is a convenience variable that takes precedence over service.name in OTEL_RESOURCE_ATTRIBUTES and YourServiceName of the Service key.

System environment variable

OTEL_SERVICE_NAME

Valid values

A string representing a service name.

This field is case-sensitive.

Required

No

Default

None

History

Introduced with version 0.5.0 of the Python Instrumentation Library.

Example

export OTEL_SERVICE_NAME="my-service"

Customize OpenTelemetry resource attributes

The Python Library provides the option to pass resource information to OpenTelemetry by environment variable.

If service.name key-value is set, it will be passed to OpenTelemetry with precedence over YourServiceName of the Service key, but with a lower precedence than OTEL_SERVICE_NAME, if set.

System environment variable

OTEL_RESOURCE_ATTRIBUTES

Valid values

A string representing a list of attribute keys and their values, delimited by commas.

This field is case-sensitive.

Required

No

Default

None

History

Introduced with version 0.5.0 of the Python Instrumentation Library.

Example

export OTEL_RESOURCE_ATTRIBUTES="service.name=my-service,another-resource=some-value"

Config file location

Specify the location of the Python Library config file

System environment variable

SW_APM_CONFIG_FILE

Required

No

Default

./solarwinds-apm-config.json

History

Introduced with version 0.9.0 of the Python Instrumentation Library.

Example

export SW_APM_CONFIG_FILE=/path/to/my_config.json

Trace context in logs

Trace context as a string identifier can be added to logs to correlate log entries for a single transaction. See Python trace context in logs for information about how this feature is supported in the Python Library.

Application logs for your service entity are not included in the data sent by APM libraries. Configure your server or application to send application logs to SolarWinds Observability. See Add logs from services.

System environment variable

OTEL_PYTHON_LOG_CORRELATION

Required

No

Possible scope values

false No action for the scope.
true Trace context is inserted in logs to correlate log entries to transactions.

Default

false

History

Introduced with version 0.11.0 of the Python Instrumentation Library.

Example

export OTEL_PYTHON_LOG_CORRELATION=true

Custom logging format

Define a custom logging format when trace context in logs is set with OTEL_PYTHON_LOG_CORRELATION. The Python Library custom logging format is based on the OTel custom logging format, but the default value for the SolarWinds Observability Python Library is different from the standard OTel custom logging format.

While the LoggingInstrumentor function is available for the standard OTel agent, it cannot be used to define the custom logging format for the SolarWinds Python Library.

System environment variable

OTEL_PYTHON_LOG_FORMAT

Valid values

Any string representing a valid logging format.

Required

No

Default

%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s trace_flags=%(otelTraceSampled)02d resource.service.name=%(otelServiceName)s] - %(message)s

Example

export OTEL_PYTHON_LOG_FORMAT="[%(otelServiceName)s][00-%(otelTraceID)s-%(otelSpanID)s-%(otelTraceSampled)02d] - %(message)s"

Trace context in queries

Trace context as a string identifier can be added to queries to correlate queries for a single transaction. See Python trace context in queries for information about how this feature is supported in the Python Library.

System environment variable

OTEL_SQLCOMMENTER_ENABLED

Required

No

Possible scope values

false No action for the scope.
true Trace context is inserted in queries to correlate queries to transactions.

Default

false

History

Introduced with version 0.13.0 of the Python Instrumentation Library.

Example

export OTEL_SQLCOMMENTER_ENABLED=true

Customize trace context in queries

Trace context and other values added as comments in queries can be customized. See Python trace context in queries for information about how this feature is supported in the Python Library.

Customization of values is applied only if OTEL_SQLCOMMENTER_ENABLED is true.

System environment variable

OTEL_SQLCOMMENTER_OPTIONS

Required

No

Valid values

A string representing a list of commenter options keys and their values, delimited by commas.

This field is case-sensitive.

The sqlcommenter options available are defined by some OpenTelemetry instrumentation libraries. All values are Boolean. They are true by default if OTEL_SQLCOMMENTER_ENABLED is true. For more information, see Flask instrumentation, SqlAlchemy instrumentation, and psycopg2 instrumentation.

Default

None

History

Introduced with version 0.15.0 of the Python Instrumentation Library.

Example

export OTEL_SQLCOMMENTER_OPTIONS=opentelemetry_values=true,db_driver=true,controller=false,route=false

Config File Example

{
    "agentEnabled": true,
    "tracingMode": "enabled",
    "collector": "apm.collector.cloud.solarwinds.com:443",
    "debugLevel": 3,
    "hostnameAlias": "apm-host-east",
    "trustedpath": "/path/to/cacert.crt",
    "ec2MetadataTimeout": 1000,
    "proxy": "http://my-proxy:3306",
    "transactionSettings": [
        {
            "regex": "CLIENT:GET",
            "tracing": "disabled"
        },
        {
            "regex": "INTERNAL:Task\\.run",
            "tracing": "disabled"
        },
        {
            "regex": "SERVER:my/[a-zA-Z0-9-<>:]*/path/",
            "tracing": "disabled"
        }
    ]
}

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.