Documentation forPapertrail

ESXi

VMware best practices dictate that ESXi virtualization hosts should have their logs stored remotely. ESXi supports sending log data to a remote log collector via the syslog protocol, which allows Papertrail to ingest it.

Prepare firewall

Connecting to Papertrail requires opening a new outgoing firewall rule, which can only be performed by SSHing to the ESXi host itself. Once the SSH service has been started via the "Security Policy" section of the VMware client, log in and run the following command to generate a new firewall rule:

Copy
$ cat <<EOF > /etc/vmware/firewall/papertrail.xml
<ConfigRoot>
  <service id='1000'>
    <id>Papertrail</id>
    <rule>
      <direction>outbound</direction>
      <protocol>tcp</protocol>
      <porttype>dst</porttype>
      <port>XXXXX</port>
    </rule>
    <enabled>true</enabled>
    <required>false</required>
  </service>
</ConfigRoot>
EOF

where XXXXX is the port number shown under log destinations.

Once that's done, refresh the firewall configuration using:

Copy
$ esxcli network firewall refresh

Note that these new rules will not persist across a reboot unless they are applied via a vSphere Installation Bundle (VIB). A custom VIB can be created by following the instructions in this VMware Knowledge Base article.

Enable syslog

Once the firewall is ready, configure remote syslog using:

Copy
$ esxcli system syslog config set --loghost='ssl://logsN.papertrailapp.com:XXXXX'
$ esxcli system syslog reload

where logsN and XXXXX are the name and port number shown under log destinations.

Certificate verification

To secure the syslog traffic against man-in-the-middle attacks, enable certificate verification.

Starting with ESXi 6.0 Update 3, certificate verification is enforced regardless of the --check-ssl-certs setting. If the following steps are skipped, /var/log/.vmsyslogd.err reports SSL certificate verification failed, and no logs are forwarded to Papertrail.

To install Papertrail's root certificate bundle, download papertrail-bundle.pem, and append the contents to /etc/vmware/ssl/castore.pem.

If the ESXi host is managed by vCenter, trusted root certificates are controlled by the Platform Services Controller. After downloading papertrail-bundle.pem, follow these instructions instead:

Finally, after using either technique to install Papertrail's root certificate bundle, enable certificate verification:

Copy
$ esxcli system syslog config set --check-ssl-certs=true
$ esxcli system syslog reload

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.