Documentation forSolarWinds Observability

Send logs using syslog

Using a syslog daemon such as rsyslog or NXlog, you can collect application, operating system, and other log files on your device and forward those logs to SolarWinds Observability. Linux devices typically come with rsyslog built in. NXLog can be installed on Windows devices to enable syslog functionality.

From a Linux machine

  1. In SolarWinds Observability, click Add Data at the top.

  2. In the Add Data dialog, click Logs.

  3. Click Manual Configuration.

  4. Create or select an API ingestion token to use when sending your logs by doing either of the following:

    • Select Generate New Token and enter an Ingestion Token Name, and then click Next.
    • Select Use Existing Token and select an ingestion token from the list, and then click Next.

  5. Select Syslog from the Method drop-down and select your Linux distribution from the Operating System drop-down.

  6. Click Easy to follow a simple, script-based method or click Manual to install certificates and configure rsyslog.

  7. For the Easy, script-based method, see Easy method below. For the full instructions to install the certificate and configure rsyslog, see Manual method.

    • Do either of the following:

      • Click Copy to Clipboard under the Run the script step and run the copied command in the terminal.

      • Run the following command in the terminal, replacing:

        • YourApiToken with your API ingestion token (found in the Token field).
        • YourSyslogEndpoint with your organization's syslog collector endpoint. The syslog collector endpoint is in the form of syslog.collector.xx-yy.cloud.solarwinds.com, where xx-yy represents the data center your organization uses. See Data centers and endpoint URIs.
        SWO_SCRIPT_URL=https://agent-binaries.cloud.solarwinds.com/syslog &&
curl -sSO $SWO_SCRIPT_URL/solarwinds-syslog-setup.sh &&
curl -sS $SWO_SCRIPT_URL/solarwinds-syslog-setup.sh.sha256 | sha256sum -c &&
SWO_HOSTNAME=YourSyslogEndpoint \
SWO_TOKEN=YourApiToken \
sudo -E sh solarwinds-syslog-setup.sh

      1. If Certificate Authority (CA) certificates are not already installed on your machine, install them now using the command:

        Debian/Ubuntu
        apt-get install ca-certificates rsyslog-gnutls
        RHEL
        yum install ca-certificates rsyslog-gnutls

      2. Create a new file named 99-solarwinds.conf and save it in /etc/rsyslog.d/. To determine the contents of the new file, do either of the following:

        • Press Copy to Clipboard under the Setup Config step and paste the contents into the 99-solarwinds.conf you just created.

        • Put the following configuration text into the 99-solarwinds.conf you just created, replacing:

          • YourApiToken with your API ingestion token (found in the Token field).
          • YourGeneralEndpoint with your organization's general collector endpoint. The general collector endpoint is in the form of collector.xx-yy.cloud.solarwinds.com, where xx-yy represents the data center your organization uses. See Data centers and endpoint URIs.
          • YourSyslogEndpoint with your organization's syslog collector endpoint. The syslog collector endpoint is in the form of syslog.collector.xx-yy.cloud.solarwinds.com, where xx-yy represents the data center your organization uses. See Data centers and endpoint URIs.
          Debian/Ubuntu
          $DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.YourGeneralEndpoint

$template SWOFormat,"<%pri%>1 %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [YourApiToken@41058]%msg:::sp-if-no-1st-sp%%msg%"
  
*.* @@YourSyslogEndpoint:6514;SWOFormat
          RHEL
          $DefaultNetstreamDriverCAFile /etc/pki/tls/certs/ca-bundle.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.YourGeneralEndpoint

$template SWOFormat,"<%pri%>1 %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [YourApiToken@41058]%msg:::sp-if-no-1st-sp%%msg%"
  
*.* @@YourSyslogEndpoint:6514;SWOFormat

      3. Restart rsyslog using the command:

        sudo service rsyslog restart

  8. Click View Logs to open the Logs Explorer and view the logs forwarded with syslog.

From a Windows machine

NXLog is required to send logs using syslog on a Windows machine. If it is not already installed, download and install the latest version of NXLog Community Edition with the default options.

  1. In SolarWinds Observability, click Add Data at the top.

  2. In the Add Data dialog, click Logs.

  3. Click Manual Configuration.

  4. Create or select an API ingestion token to use when sending your logs by doing either of the following:

    • Select Generate New Token and enter an Ingestion Token Name, and then click Next.
    • Select Use Existing Token and select an ingestion token from the list, and then click Next.

  5. Select Syslog and select Windows from the Operating System drop-down.

  6. Download the certificate to the cert folder in the NXLog installation directory (typically C:\Program Files\nxlog\ on 64-bit systems).

  7. Open nxlog.conf in the conf folder in the NXLog installation directory. To determine the contents of the new file, do either of the following:

    • Press Copy to Clipboard under the Setup Config step and paste the contents into the nxlog.conf you just created.

    • Put the following configuration text into the nxlog.conf you just created, replacing:

      • YourApiToken with your API ingestion token (found in the Token field).

      • C:\Program Files (x86)\nxlog with your installation directory.

      • YourSyslogEndpoint with your organization's syslog collector endpoint. The syslog collector endpoint is in the form of syslog.collector.xx-yy.cloud.solarwinds.com, where xx-yy represents the data center your organization uses. See Data centers and endpoint URIs.

      ## Please set the ROOT to your nxlog installation directory

define TOKEN YourApiToken
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
  Module xm_syslog
</Extension>

# Monitor application log files
<Input watchfile>
  Module im_file
  # File 'C:\\path\\to\\*.log'
  Exec $Message = $raw_event;
  Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1;
  SavePos TRUE
  Recursive FALSE
</Input>

# Monitor Windows event logs
<Input eventlog>
  # Uncomment for Windows Vista/2008 or later
  Module im_msvistalog

  # Uncomment for Windows 2000 or later
  # Module im_mseventlog
</Input>

<Output syslogout>
  Module om_ssl
  Host YourSyslogEndpoint
  Port 6514
  Exec $Hostname = hostname(); to_syslog_ietf();
  Exec $raw_event =~ s/^(?:[^ ]* ){6}\K(?:-|\[(?:\\.|[^]\\])*])/[%TOKEN%@41058]/;
  OutputType Syslog_TLS
  CAFile %CERTDIR%\cacert.pem
  AllowUntrusted FALSE
</Output>

<Route 1>
  Path eventlog, watchfile => syslogout
</Route>

  8. Review the nxlog.conf file to identify any lines that you need to uncomment or change for your system configuration.

    NXLog v3.0 - 3.1.x has a known issue where the program name will not appear as expected, instead the program name shows as a hyphen. Upgrade to NXLog 3.2 or newer, or update your config file replace this line: 

    Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1;

    with: 

    Exec $SourceName = 'programName';

  9. To optionally send the contents of a local log file, uncomment and modify the following line to point to your log file(s)

    File 'C:\path\to\*.log'

  10. Restart the nxlog service using the commands:

    net stop nxlog
net start nxlog

  11. Click View Logs to open the Logs Explorer and view the logs forwarded by NXlog.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.