Documentation forSolarWinds Observability SaaS

Ingesting logs from Python (Papertrail)

You can ship Python application logs directly to SolarWinds Papertrail using either of the following methods:

  1. Syslog (TLS)

  2. HTTPS bulk

To get a destination and token in Papertrail, add a new log destination. The ingestion token is generated along with the destination. Use the collector endpoints for your account's region:

  • Syslog (TLS): syslog.collector.<REGION>.cloud.solarwinds.com:6514

  • HTTPS: https://logs.collector.<REGION>.cloud.solarwinds.com/v1/logs/bulk

Replace <REGION> with the data center your organization uses: na-01, eu-01, or ap-01.

Ship logs with Syslog (TLS)

Severity and host/program are carried per log record, the ingestion token is sent as structured data.

To ship Python application logs directly with Syslog (TLS), install the following package:

pip install rfc5424-logging-handler

This also installs the pytz and tzlocal as transitive dependencies.

Now configure your Python application.

import logging
import socket
import os
from rfc5424logging import Rfc5424SysLogHandler

handler = Rfc5424SysLogHandler(
    address         = ("syslog.collector.<REGION>.cloud.solarwinds.com", 6514),
    appname         = "my-service",
    # hostname defaults to socket.gethostname() if omitted
    socktype        = socket.SOCK_STREAM,
    tls_enable      = True,
    utc_timestamp   = True,
    structured_data = {"solarwinds@41058": {"token": os.environ["SWO_TOKEN"]}},
)

logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(handler)

The structured_data dict injects [solarwinds@41058 token="..."] as an RFC 5424 structured data element into every outgoing syslog message.

TLS certificates

No manual trust store configuration is required. The Papertrail collector certificate (*.cloud.solarwinds.com) is issued by Amazon RSA 2048 M04, which is trusted by Python's ssl.create_default_context() on all major platforms (it uses the system CA store on Linux/macOS and the Windows certificate store on Windows). No certifi or custom CA bundle is needed.

Additional info

  • Severity mapping (according to RFC 5424): DEBUG → 7, INFO → 6, WARNING → 4, ERROR → 3, CRITICAL → 2.

  • Exception stack traces are included in the log message.

  • The handler holds a single persistent TCP+TLS connection and reconnects automatically on write error.

  • There are no external libraries beyond rfc5424-logging-handler.

Ship logs with HTTPS bulk

When shipping logs with HTTPS bulk, the host/program is found in the X-Otel-Resource-Attr header, and the ingestion token is in the Authorization: Bearer header. The severity can be set as a field in the payload and arrives as a searchable json.* field only.

To ship Python application logs direct with HTTPS bulk, install the following package:

pip install python-json-logger

Then copy the following file into your project: 

import logging
import urllib.request


class SwoHttpHandler(logging.Handler):
    """Sends each log record as JSON to the SWO HTTPS bulk endpoint. Uses stdlib urllib only."""

    def __init__(self, url, token, hostname, appname):
        super().__init__()
        self.url      = url
        self.token    = token
        self.hostname = hostname
        self.appname  = appname

    def emit(self, record):
        try:
            body = self.format(record).encode("utf-8")
            req  = urllib.request.Request(
                self.url,
                data    = body,
                method  = "POST",
                headers = {
                    "Authorization":        f"Bearer {self.token}",
                    "X-Otel-Resource-Attr": f"host.name={self.hostname},service.name={self.appname}",
                    "Content-Type":         "application/x-ndjson",
                },
            )
            urllib.request.urlopen(req, timeout=10)
        except Exception:
            self.handleError(record)

Now configure the HTTPS log handler.

import logging
from pythonjsonlogger.json import JsonFormatter
from swo_http_handler import SwoHttpHandler

handler = SwoHttpHandler(
    url      = "https://logs.collector.<REGION>.cloud.solarwinds.com/v1/logs/bulk",
    token    = "<SWO_TOKEN>",
    hostname = "my-host",
    appname  = "my-service",
)
handler.setFormatter(
    JsonFormatter("%(message)s %(levelname)s", rename_fields={"levelname": "severity"})
)
logging.getLogger().setLevel(logging.DEBUG)
logging.getLogger().addHandler(handler)

The JsonFormatter from python-json-logger produces a JSON object per record. All JSON fields are searchable in Logs Explorer as json.*.

Additional info

  • SwoHttpHandler sends one HTTP request per log event. For high-throughput apps, wrap it in a logging.handlers.QueueHandler / QueueListener pair to avoid blocking the calling thread.

  • host.name and service.name are set from the X-Otel-Resource-Attr header and apply to the whole request batch. They are not per-record.

  • Exception stack traces land in json.exc_info as a multiline string, fully readable in Logs Explorer.

Fallback: Forward via a syslog daemon

If your application cannot connect directly to the Papertrail collector — for example, a strict outbound firewall blocks ports 6514 and 443 — write logs to a local file and use the host's syslog daemon to forward them to Papertrail over TLS. SolarWinds Observability supports rsyslog on Linux and NXlog on Windows for this purpose. See Send Logs using syslog for configuration instructions.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.