Navigation Notice: When the APM Integrated Experience is enabled, Loggly shares a common navigation and enhanced feature set with other integrated experience products. How you navigate Loggly and access its features may vary from these instructions.
Use these tips to troubleshoot problems with Rsyslog. You can use our automated test, check the configuration, send sample data, and check transmission. Additionally, you can read the Rsyslog manual, try their support forum (they offer professional Rsyslog support), or check out our Rsyslog manual configuration docs.
Wait a few minutes after sending an event to give it time to index and appear in the search results. It normally happens within seconds, but sometimes it can take longer.
If Loggly isn’t seeing data check our status page to make sure we are indexing data and search is running. You should see green dots and "All Systems Operational".
Our configure-syslog script can send a test event to Loggly, and then verify if it’s received using the Loggly search API. You can overwrite your existing loggly configuration to make sure there are no errors and verify it again. It may take a few minutes to run.
curl -O https://www.loggly.com/install/configure-linux.sh sudo bash configure-linux.sh -a SUBDOMAIN -u USERNAME
- SUBDOMAIN: your account subdomain that you created when you signed up for Loggly
- USERNAME: your Loggly username
Make sure you restarted rsyslog so your changes take effect
sudo service rsyslog restart
Make sure rsyslog is running. If this command returns nothing than it’s not running.
ps -A | grep rsyslog
Check the rsyslog configuration. If there are no errors listed, then it’s ok.
Make sure you have Rsyslog version 5.8 or higher
Check the Linux system log for rsyslog errors. You should see an event that it started and no errors. Some logs may also be in /var/log/syslog.
sudo cat /var/log/messages | grep rsyslog
Make sure Loggly is configured in your rsyslog configuration. There should be an endpoint for logs-01.loggly.com either in your main rsyslog.conf file or an include to the 22-loggly.conf file.
sudo vim /etc/rsyslog.d/22-loggly.conf
Check the permissions of rsyslog and the file you want to monitor to be sure it can read that file. You may need to use alter the privilege in the rsyslog.conf file:
Verify rsyslog is sending data to Loggly by making a test event. Then search for that event in Loggly by searching for "TroubleshootingTest" in the last hour.
Check the Linux system log to see if Rsyslog recorded the test event
sudo cat /var/log/messages | grep TroubleshootingTest
If you are sending repeated test messages, you should turn off repeated message reduction in the rsyslog configuration.
If you are filtering events out with a lower priority, you should send test events with a high enough priority.
logger -p local0.error "TroubleshootingTest"
Use netstat to verify Rsyslog has an established connection to Loggly. Specifically, check that Loggly can make a connection through your firewall on the proper port. It’s 514 for syslog, 6514 for TLS syslog, 80 for HTTP, and 643 for HTTPS.
sudo netstat -taupn | grep syslog
Use telnet to verify we can make an outbound connection to Loggly. For syslog we use port 514. If you can’t connect it might be a network or firewall issue.
telnet logs-01.loggly.com 514
Use tcpdump to verify data is being sent to Loggly. If you send your events in cleartext while tcpdump is running, you should be able to see them in the left hand column.
sudo tcpdump -A dst logs-01.loggly.com
If your application logs syslog to rsyslog, you can also test to see if messages making it to rsyslog over UDP to localhost.
sudo tcpdump -i lo -A udp and port 514
Some older version of rsyslog may have trouble resuming after a log is rotated. If you have log rotation setup, follow these instructions to force rsyslog to pick up the new file.
- Rsyslog-users – Mailing list for rsyslog describing many common support issues
Please search our community forum for more Rsyslog configuration answers or post your own question.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.