Python Logging Over HTTP/S

Loggly provides the infrastructure to aggregate and normalize log events so they are available to explore interactively, build visualizations, or create threshold-based alerting. In general, any method to send logs from a system or application to an external source can be adapted to send logs to Loggly. The following instructions provide one scenario for sending logs to Loggly.

You can send Python logs to Loggly using the Loggly’s Python logging handler package. This is an asynchronous and non-blocking handler that uses requests-futures to send logs. It also supports JSON and multiline events. For alternatives, please see the Advanced Options section.

Python Logging Setup

1. Install loggly-python-handler using pip

Install the loggly-python-handler library from the Python Package Index using pip or from GitHub. This handler is better to use if you are hosted in a PaaS and don’t have access to the system syslog daemon, or if you are logging from a client-side script or app.

sudo pip install loggly-python-handler

2. Add HTTPSHandler to the Configuration file

Add a configuration file python.conf with the following contents in it.

[handlers]
keys=HTTPSHandler

[handler_HTTPSHandler]
class=loggly.handlers.HTTPSHandler
formatter=jsonFormat
args=('https://logs-01.loggly.com/inputs/TOKEN/tag/python','POST')

[formatters]
keys=jsonFormat

[loggers]
keys=root

[logger_root]
handlers=HTTPSHandler
level=INFO

[formatter_jsonFormat]
format={ "loggerName":"%(name)s", "timestamp":"%(asctime)s", "fileName":"%(filename)s", "logRecordCreationTime":"%(created)f", "functionName":"%(funcName)s", "levelNo":"%(levelno)s", "lineNo":"%(lineno)d", "time":"%(msecs)d", "levelName":"%(levelname)s", "message":"%(message)s"}

Replace:

TOKEN: your customer token from the source setup page

3. Use Configuration in python file

Write the following code inside your Python source file to send logs to Loggly over HTTPS

import logging
import logging.config
import time
import loggly.handlers

logging.config.fileConfig('python.conf')
logging.Formatter.converter = time.gmtime
logger = logging.getLogger('myLogger')

logger.info('Test log')

It assumes that python.conf and your source file are placed in the same directory. Run your application and send some logs.

4. Django Logging

To setup Django logging use the following configuration in your settings.py file

LOGGING = {
  'version': 1,
  'disable_existing_loggers': False,
  'formatters': {
    'verbose': {
      'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
    },
    'simple': {
      'format': '%(levelname)s %(message)s'
    },
    'json': {
      'format': '{ "loggerName":"%(name)s", "timestamp":"%(asctime)s", "fileName":"%(filename)s", "logRecordCreationTime":"%(created)f", "functionName":"%(funcName)s", "levelNo":"%(levelno)s", "lineNo":"%(lineno)d", "time":"%(msecs)d", "levelName":"%(levelname)s", "message":"%(message)s"}',
    },
  },
  'handlers': {
    'console': {
      'class': 'logging.StreamHandler',
      'level': 'DEBUG',
      'formatter': 'verbose',
    },
    'loggly': {
      'class': 'loggly.handlers.HTTPSHandler',
      'level': 'INFO',
      'formatter': 'json',
      'url': 'https://logs-01.loggly.com/inputs/TOKEN/tag/python',
    },
  },
  'loggers': {
    'django': {
      'handlers': ['console', ],
      'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
    },
    'APPNAME': {
      'handlers': ['console', 'loggly'],
      'level': 'INFO',
    },
  },
}

Replace:

TOKEN: your customer token from the source setup page
APPNAME: your Django application name

5. Verify Events

Search Loggly for events with the tag as python over the past 20 minutes. It may take few minutes to index the event. If it doesn’t work, see the troubleshooting section below.

tag:python

Python Logging Over HTTP/S Example

Advanced Python Logging Options

Python Syslog Handler – You can use Syslog Handler to send logs to syslog.
Django Logging – Step by step guide for setting up Django logging
LogglyAppEngine – If you use Google App Engine use this and set the endpoint to logs-01.loggly.com
Urllib2 – If you would like to send requests directly to Loggly over HTTP

Troubleshooting Python Logs

If you don’t see any data show up in the verification step, then check for these common problems.

Wait a few minutes in case indexing needs to catch up
See our HTTP Troubleshooting Guide to verify HTTP events are being sent to Loggly.
Check out our Guide to Python Logging
Search or post your own Python logging configuration questions, or questions how to import logging to Loggly, logging levels, logging modules, and more in the community forum.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.

Search SolarWinds Support