Loggly Event Retrieval APIs

Loggly offers two different APIs that you can use for event retrieval.  There is the single-block event retrieval API and the Paginating event retrieval API.  Here’s how they differ:

	Single-block event retrieval API	Paginating event retrieval API
Operation	Asynchronous (non blocking)	Synchronous (blocking)
Events Per Retrieval	5000	All Events
Use case	Require less than 5,000 events	Require more than 5,000 events

Please note:  The Single-block event retrieval API is deprecated and will be End of Life (EOL) on January 1, 2018. We recommend you begin planning to migrate to the Paginating event retrieval API. We encourage you to try the Paginating API and send any feedback to Support.

Paginating Event Retrieval API

For performance reasons you might want to paginate the response output. This is because returning the entire data set might be feasible for some queries but prohibitive for others that return a very large amount of data. We now provide a new events Iterator Endpoint to allow paginated output. It is completely synchronous/blocking and does not need a polling / RSID response querying pattern.

Events Iterator Endpoint

The base endpoint for the Events Iterator Endpoint is:

https://<subdomain>.loggly.com/apiv2/events/iterate 

Replace <subdomain> with your organization's subdomain.

Usage:

First, you call the endpoint with query parameters.  The API will respond with the first page (from 50 to 1000 events), if any are found.  If additional pages are available, the response will include a "next" URL that is the last JSON object returned.  Use this URL to retrieve subsequent pages.

Note:

• The results of the initial search are cached.  The cache expires if 10 minutes elapses between calls.  
• The query parameters cannot be altered or added to while calling the "next" URL.
• If the count of events is less than the requested page size, then the "next" URL will not be provided on the page.

First call – Create the query:

A set of query parameters must be sent on the initial call to /events/iterate endpoint. These parameters are stored and cached and are automatically applied when the "next" URL is called.

Events Iterator Endpoint Request Parameters
q	optional	query string, check out the Search Query help. Defaults to "All" events.
from	optional	Start time (in UTC) for the search. Defaults to "-24h". (See valid time parameters.)
until	optional	End time (in UTC) for the search. Defaults to "now". (See valid time parameters.)
size	optional	Events returned on each page. Defaults to 50. Maximum size is 1000.
order	optional	Direction of results returned, either "asc" or "desc". Defaults to "desc".

Events Iterator Endpoint Response Parameters
events:	Each item in the events array will have the following fields:
	tags:	An array of any tags associated with the event
	timestamp:	See timestamps to understand how a reference timestamp is derived.
	logmsg:	The message portion of the log event. (Any headers aren’t included.)
	event:	Any parsed fields are included.
	logtypes:	An array of log types that were detected.
	raw:	The raw event.
	unparsed:	The portion of the event which was not parsed by our parsers. It will be null if it is not applicable.
	id:	Loggly’s event ID.
next:	URL of the next set of events.

Events Iterator Endpoint Usage Example

To see events in pages, start with a request like the examples below. Replace <token> with your API token and <subdomain> with your organization's subdomain.

curl -H 'Authorization: bearer <token>' -XGET 'https://<subdomain>.loggly.com/apiv2/events/iterate?q=*&from=-10m&until=now&size=2' 

Output Example:

curl -H 'Authorization: bearer <token>' -XGET 'https://<subdomain>.loggly.com/apiv2/events/iterate?q=*&from=-10m&until=now&size=1' 
{
    "events": [
       {
           "raw": "{\"function\": \"handle\", \"publish_success\": 83, \"timestamp\": \"17-02-01 00:58:04,852149\", \"start_time\": 1485910682.351175, \"args\": \"\", \"duration\": 2501, \"message\": \"end publish run\", \"publish_fail\": 0, \"level\": \"INFO\", \"source_duration\": {\"duration\": 1, \"end_time\": 1485910682.351801}, \"feature_duration\": {\"duration\": 1427, \"end_time\": 1485910683.786751}, \"pathname\": \"/opt/loggly/web/app/alert/management/commands/alerterd.py\", \"lineno\": 329, \"cache_duration\": {\"duration\": 1, \"end_time\": 1485910684.821093}, \"action\": \"alerterd\", \"update_duration\": {\"duration\": 8, \"end_time\": 1485910684.852121}, \"end_time\": 1485910684}",
           "logtypes": [
              "json",
              "syslog"
           ],
           "timestamp": 1485910684852,
           "unparsed": null,
           "logmsg": "{\"function\": \"handle\", \"publish_success\": 83, \"timestamp\": \"17-02-01 00:58:04,852149\", \"start_time\": 1485910682.351175, \"args\": \"\", \"duration\": 2501, \"message\": \"end publish run\", \"publish_fail\": 0, \"level\": \"INFO\", \"source_duration\": {\"duration\": 1, \"end_time\": 1485910682.351801}, \"feature_duration\": {\"duration\": 1427, \"end_time\": 1485910683.786751}, \"pathname\": \"/opt/loggly/web/app/alert/management/commands/alerterd.py\", \"lineno\": 329, \"cache_duration\": {\"duration\": 1, \"end_time\": 1485910684.821093}, \"action\": \"alerterd\", \"update_duration\": {\"duration\": 8, \"end_time\": 1485910684.852121}, \"end_time\": 1485910684}",
           "id": "7ce48bda-e819-11e6-808b-12a1c1f6d2c3",
           "tags": [],
           "event": {
               "syslog": {
                   "severity": "Informational",
                   "appName": "msg",
                   "timestamp": "2017-02-01T00:58:04.852+00:00",
                   "facility": "local use 1",
                   "priority": "142",
                   "host": "127.0.0.1"
               },
               "json": {
                   "function": "handle",
                   "publish_success": 83,
                   "level": "INFO",
                   "timestamp": "17-02-01 00:58:04,852149",
                   "start_time": 1485910682.351175,
                   "cache_duration": {
                       "duration": 1,
                       "end_time": 1485910684.821093
               },
               "update_duration": {
                   "duration": 8,
                   "end_time": 1485910684.852121
               },
               "duration": 2501,
               "pathname": "/opt/loggly/web/app/alert/management/commands/alerterd.py",
               "end_time": 1485910684,
               "source_duration": {
                   "duration": 1,
                   "end_time": 1485910682.351801
               },
               "action": "alerterd",
               "message": "end publish run",
               "publish_fail": 0,
               "feature_duration": {
                   "duration": 1427,
                   "end_time": 1485910683.786751
               },
               "lineno": 329
               }
           }
       }
    ],
    "next": "https://<subdomain>.loggly.com/apiv2/events/iterate?next=eea25ee6-0e48-4428-a544-36d6441d132c"
}

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.