Logstash Logs

Loggly provides the infrastructure to aggregate and normalize log events so they are available to explore interactively, build visualizations, or create threshold-based alerting. In general, any method to send logs from a system or application to an external source can be adapted to send logs to Loggly. The following instructions provide one scenario for sending logs to Loggly.

Logstash logs can easily be sent to Loggly over HTTP. We will automatically parse the logs sent by Logstash in JSON format. You can send logs from any of Logstash’s inputs, but we offer an example showing a standard Logstash input. These instructions were tested with latest version 7.4.2. You may try to apply the below instructions for latest Logstash versions. For alternatives, please see the Advanced Options section.

Logstash Logging Setup

1. Configure Logstash To Output To Loggly

Download the logstash tar.gz file from here.
Unzip and Untar the file

sudo tar -xzvf logstash-7.4.2.tar.gz

Move the folder to /opt/

sudo mv logstash-7.4.2 /opt/

Go to the folder and install the logstash-output-loggly plugin

cd logstash-7.4.2
sudo bin/logstash-plugin install logstash-output-loggly

Create a logstash-loggly.conf file and add it to the root folder of the Logstash directory. We included a source field for logstash to make them easier to find in Loggly.

vim logstash-loggly.conf

If you are using version 1.x, then add the following content in the file

input{
  file{
    path => "FILENAME"
    start_position => beginning
  }
}
filter{
  mutate{
    add_field => ["timestamp", "%{@timestamp}"]
  }
}
output{
  loggly{
    key => "TOKEN"
    host => "logs-01.loggly.com"
    proto => "https"
  }
}

If you are using version 2.x to version 7.x, then add the following content in the file

input{
  file{
    path => "FILENAME"
    start_position => beginning
  }
}
filter{
  mutate{
    add_field => ["timestamp", "%{@timestamp}"]
  }
}
output{
  loggly{
    key => "TOKEN"
    tag => "logstash"
    host => "logs-01.loggly.com"
    proto => "https"
  }
}

Put this file in the root folder of the logstash directory

Replace:

FILENAME: your file name with absolute path whose logs are to be monitored
TOKEN: your customer token from the source setup page

2. Sending Logstash Logs to Loggly

Run Logstash to send the files to Loggly. This command will run it in the background. Please run it inside the root folder for Logstash.

bin/logstash -f logstash-loggly.conf &

3. Verify Logstash Events

Search Loggly for events with the Logstash in json.source field over the past 20 minutes. It may take a few minutes to index the event. If it doesn’t work, see the troubleshooting section below.

tag:logstash

Click on one of the logs to show a list of JSON fields (see screenshot below). If you don’t see them, please check that you are using one of our automatically parsed formats.
Logstash Logs Example

Advanced Logstash Logging Options

Contrib-plugin – Extended contrib plug-ins for Logstash

Troubleshooting Logstash Logs

If you don’t see any data show up in the verification step, then check for these common Logstash logging errors.

How to check it:

Wait a few minutes in case indexing needs to catch up
Check to see if the logstash-loggly.conf is properly created and placed in the proper location. It should be in the root of logstash folder downloaded from the web
Check if you are running commands in the proper location. You should be inside the root of the logstash folder downloaded from the web
Check if the file path provided in the logstash-loggly.conf file is correct
You can’t send JSON data to the current Loggly output plugin because it escapes the data before inserting it into the message. You can fork and modify it as needed.
Run "sudo tcpdump dst logs-01.loggly.com and port 80″ to verify HTTP events are being sent to Loggly

Still Not Working?

Search or post your own Logstash question in the community forum.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.