Ingesting logs from Python (Papertrail)
You can ship Python application logs directly to SolarWinds Papertrail using either of the following methods:
-
Syslog (TLS)
-
HTTPS bulk
To get a destination and token in Papertrail, add a new log destination. The ingestion token is generated along with the destination. Use the collector endpoints for your account's region:
-
Syslog (TLS):
syslog.collector.<REGION>.cloud.solarwinds.com:6514 -
HTTPS:
https://logs.collector.<REGION>.cloud.solarwinds.com/v1/logs/bulk
Replace <REGION> with the data center your organization uses: na-01, eu-01, or ap-01.
Determining your method
| Syslog (TLS) | HTTPS bulk | |
|---|---|---|
| Log record carries per-record severity | Yes | No (always "info"; actual level in json.severity) |
| Carries system and program | Yes | Yes, from request header (for whole batch) |
| Exception traces | Included in the log message | Full multiline in json.exc_info |
| Setup effort | pip install rfc5424-logging-handler
|
One extra Python file + pip install python-json-logger |
| Extra dependencies | rfc5424-logging-handler (+ pytz, tzlocal) |
python-json-logger
|
| Outbound port | 6514 (TLS) | 443 (HTTPS) |
How Papertrail fills system and program
The Papertrail Logs Explorer is syslog-shaped and fills two key columns:
-
System (host/source)
-
Program (application)
How these columns are populated depends on the transport. Over syslog, the source is the RFC 5424 HOSTNAME and APP-NAME fields. Each record carries its own values. Over HTTPS bulk, the host.name and service.name attributes come from an HTTP header. The header applies to the whole request batch, so they are not read from the log body.
Ship logs with Syslog (TLS)
To ship Python application logs directly with Syslog (TLS), install the following package:
pip install rfc5424-logging-handler
This also installs the pytz and tzlocal as transitive dependencies.
Now configure your Python application.
import logging
import socket
import os
from rfc5424logging import Rfc5424SysLogHandler
handler = Rfc5424SysLogHandler(
address = ("syslog.collector.<REGION>.cloud.solarwinds.com", 6514),
appname = "my-service",
# hostname defaults to socket.gethostname() if omitted
socktype = socket.SOCK_STREAM,
tls_enable = True,
utc_timestamp = True,
structured_data = {"solarwinds@41058": {"token": os.environ["SWO_TOKEN"]}},
)
logging.getLogger().setLevel(logging.INFO)
logging.getLogger().addHandler(handler)
The structured_data dict injects [solarwinds@41058 token="..."] as an RFC 5424 structured data element into every outgoing syslog message.
TLS certificates
No manual trust store configuration is required. The Papertrail collector certificate (*.cloud.solarwinds.com) is issued by Amazon RSA 2048 M04, which is trusted by Python's ssl.create_default_context() on all major platforms (it uses the system CA store on Linux/macOS and the Windows certificate store on Windows). No certifi or custom CA bundle is needed.
Additional info
-
Severity mapping (according to RFC 5424): DEBUG → 7, INFO → 6, WARNING → 4, ERROR → 3, CRITICAL → 2.
-
Exception stack traces are included in the log message.
-
The handler holds a single persistent TCP+TLS connection and reconnects automatically on write error.
-
There are no external libraries beyond
rfc5424-logging-handler.
Ship logs with HTTPS bulk
When shipping logs with HTTPS bulk, the host/program is found in the X-Otel-Resource-Attr header, and the ingestion token is in the Authorization: Bearer header. The severity can be set as a field in the payload and arrives as a searchable json.* field only.
To ship Python application logs direct with HTTPS bulk, install the following package:
pip install python-json-logger
Then copy the following file into your project:
import logging
import urllib.request
class SwoHttpHandler(logging.Handler):
"""Sends each log record as JSON to the SWO HTTPS bulk endpoint. Uses stdlib urllib only."""
def __init__(self, url, token, hostname, appname):
super().__init__()
self.url = url
self.token = token
self.hostname = hostname
self.appname = appname
def emit(self, record):
try:
body = self.format(record).encode("utf-8")
req = urllib.request.Request(
self.url,
data = body,
method = "POST",
headers = {
"Authorization": f"Bearer {self.token}",
"X-Otel-Resource-Attr": f"host.name={self.hostname},service.name={self.appname}",
"Content-Type": "application/x-ndjson",
},
)
urllib.request.urlopen(req, timeout=10)
except Exception:
self.handleError(record)
Now configure the HTTPS log handler.
import logging
from pythonjsonlogger.json import JsonFormatter
from swo_http_handler import SwoHttpHandler
handler = SwoHttpHandler(
url = "https://logs.collector.<REGION>.cloud.solarwinds.com/v1/logs/bulk",
token = "<SWO_TOKEN>",
hostname = "my-host",
appname = "my-service",
)
handler.setFormatter(
JsonFormatter("%(message)s %(levelname)s", rename_fields={"levelname": "severity"})
)
logging.getLogger().setLevel(logging.DEBUG)
logging.getLogger().addHandler(handler)
The JsonFormatter from python-json-logger produces a JSON object per record. All JSON fields are searchable in Logs Explorer as json.*.
Additional info
-
SwoHttpHandlersends one HTTP request per log event. For high-throughput apps, wrap it in alogging.handlers.QueueHandler/QueueListenerpair to avoid blocking the calling thread. -
host.nameandservice.nameare set from theX-Otel-Resource-Attrheader and apply to the whole request batch. They are not per-record. -
Exception stack traces land in
json.exc_infoas a multiline string, fully readable in Logs Explorer.
Fallback: Forward via a syslog daemon
If your application cannot connect directly to the Papertrail collector — for example, a strict outbound firewall blocks ports 6514 and 443 — write logs to a local file and use the host's syslog daemon to forward them to Papertrail over TLS. SolarWinds Observability supports rsyslog on Linux and NXlog on Windows for this purpose. See Send Logs using syslog for configuration instructions.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.