Documentation forPapertrail

systemd

To send system logs from Linux distributions using systemd (including newer releases of CentOS and CoreOS), follow the steps shown below to forward output from journalctl to Papertrail.

For applications running in a Docker container, the options described on the Docker configuration page are recommended, but also see Docker and systemd below.

Forwarding system logs

Create a new unit/service that pipes output from journalctl into ncat or socat by following the steps below:

  • Create a new unit file named papertrail.service under /etc/systemd/system (root permissions required).
  • Paste the directives shown below, and replacing logsN and XXXXX with the values that appear under log destinations.
Copy
[Unit]
Description=Papertrail
After=systemd-journald.service
Requires=systemd-journald.service
[Service]
ExecStart=/bin/sh -c "journalctl -f | ncat --ssl logsN.papertrailapp.com XXXXX"
TimeoutStartSec=0
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
  • Run $ sudo systemctl enable /etc/systemd/system/papertrail.service to enable the unit.
  • Then run $ sudo systemctl start papertrail.service to start it.
  • Finally, use $ journalctl -f -u papertrail.service to view the local logs and ensure the new unit has started successfully

Using socat

If ncat is not available, use socat instead by:

Copy
ExecStart=/bin/sh -c "journalctl -f | socat - SSL:logsN.papertrailapp.com:XXXXX,cafile=/etc/papertrail-bundle.pem"

Again, replace logsN and XXXXX with the values that appear under log destinations.

Docker and systemd

When using systemd to send logs from Docker containers, as on CoreOS, the solutions above will result in all Docker logs coming in under the program name docker rather than the service name. If sending logs from containers via systemd is desirable, try the solution created by Papertrail customer C2FO:

Copy
journalctl -o json -f | jq -r --unbuffered '"\(.__REALTIME_TIMESTAMP | tonumber / 1000000) \(._HOSTNAME) \(._SYSTEMD_UNIT // .UNIT // .SYSLOG_IDENTIFIER): \(.msgtype = (.MESSAGE | type) | if .msgtype == "array" then (.MESSAGE | implode) else .MESSAGE end)"' | awk '{ $1=strftime("%b %d %T", $1); print $0; fflush(); }'

piped to ncat or socat. This is easiest to place in a separate script, but it can also be placed directly in an ExecStart directive. It makes use of the JSON format for journalctl logs, transformed to a syslog string using jq and awk. There’s also an option to set up coproc to better detect sending failures.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.