Dynamic Field Explorer

With Loggly Dynamic Field Explorer^TM, troubleshooting never begins with a blank search page. Instead, you see a structured summary of all your parsed logs, showing inherent structure, identified field names, and frequency of individual values. The structured summary displays like a map of your data, showing you both the most common events and the anomalies, and it provides a quick and precise way to hone in on specific logs and filter out the noise.

As you continue to refine what you are looking for, Dynamic Field Explorer is always updating to provide relevant insights into the data you are viewing at the moment. This makes your process much faster than starting with a series of trial-and-error searches or having to learn a new search language before you gain any insight.

Common Scenarios Where Field Explorer Can Help

• You are not sure where to start searching and would like a guided search experience.

• You don’t know the name or syntax of a specific field you want to search on.

• You would like an instant summary of all the logs seen during a specific time or a subset of hosts/users, instead of manually checking one by one.

• You want to know how frequently a certain event or value shows up in your logs, and be able to filter out the noise quickly and precisely.

• You want to quickly browse your log data to see if any anomalies stand out.

Using Dynamic Field Explorer

Dynamic Field Explorer^TM is shown by default in any new tab you create. It gives you an instant summary of all the field categories that have been identified in the past 10 minutes.

After you select a field category, a list of all the known fields for that category displays along with a tremendous amount of information and analytics about your log data to help you get to insights quickly.

Dynamic Field Explorer really shines in helping you get to insights or find the root of a problem, even if you don’t know what you're looking for. Many customers find they are able to gain the necessary insight without typing a single search query. This is all possible due to the real-time navigable summaries created and updated with each and every click or change in your search context.

Dynamic Field Explorer has limit of 500 fields that can be displayed in the Loggly UI.

Anatomy of Dynamic Field Explorer

Search fields/values

If you are looking for a specific field name or value, you can use the quick search option by clicking anywhere near the search icon at the top of the Field Explorer.

A search bar displays where you can type what you are looking for. The search is performed across all field categories and the current selected value list.

Field actions

Provides access to analytical actions you take on the field, such as plotting a trend graph. See below for more details on field actions.

Field List Panel

Fields are listed in alphabetical order in hierarchical views (for example, JSON.context.debug would be a tree of 3 levels).  Tree views are collapsed by default. Numeric values are marked with a # to distinguish them from categorical fields.

Value List & Quick Stats Panel

All the values for the specific field are shown in order of the number of times it appears in the current search context.  Exact count appears next to each event.

The header has the same name as the selected field, the total number of unique values identified, and the total number of events the field is found in.

Can be used to quickly and precisely filter with a click. Allows you to click more than one.

Slider to customize size of Panel

Click and hold to drag each panel to the preferred size.  Panels have a minimum size. If you reduce beyond that minimum the panel automatically snaps to a collapsed state.

If you click instead of dragging, the panel collapses fully.

Quick Recall bar for categories

Hovering your mouse quickly brings back the cover screen for Field Explorer, listing all the field categories available.  Use this to quickly switch between categories

Numeric Stats

Dynamic Field Explorer also provides common statistics on numeric field values. These include the min, max, mean, standard deviation, and variance of the data. They are visible in the right pane after clicking on a numeric field.

Field Actions

A variety of actions can be performed with a single click based on the specific field. There are actions for both categorical and numeric fields.

Categorical Field Actions

• Pie chart. A pie chart of the top values so you can see their relative proportions.
• Bar chart. A horizontal bar chart of the top values.
• Timeline Chart. A timeline representation of the values. Lets you see a count over time of the top values.
• Copy Values. Copies the values pane as a table to your clipboard so you can easily paste it into a word processor or spreadsheet.

Numeric Field Actions

• Value of. Provides a visual graph of all the values for that numeric field shown over time so you can quickly get insight about its distribution, outliers, etc.
• Statistics. A timeline representation of the values. Lets you do some quick aggregate calculations (sum, average, max/min) over time.
• Single Value. Similar to statistics but instead of measuring an aggregate over time, it gives you the calculation over the entire search context (for example, if you want to know the MAX value over the entire period).
• Quick Filtering option. Allows you to quickly filter the logs based on a numeric range of values.

FAQ

• What type of logs are automatically parsed?
  Loggly's Dynamic Fields™ technology automatically parses the most common log types sent by customers (for example, Apache, Nginx, Java, Rails, JSON, etc.).  For the full and most current list, see Automated Parsing Log Types.
• Can I still filter based on my own custom tags?
  Absolutely.  If you have set up custom tags for your data, you’ll find them (along with other special attributes) under Other along with the logtype field.
• What happens with events that are not parsed?
  Dynamic Field Explorer only provides summary data for events with parsed fields.  For other events, you can continue to leverage the other robust search, navigation, and graphing capabilities to quickly find the data you are looking for.
• Can I define rules to custom parse parts of my data Loggly may not be automatically parsing?
  The best way to define rules to custom parse is to use readily available 3rd-party tools to translate custom log formats into JSON and then send them to Loggly (find out more about Custom Parsing). SolarWinds is working to streamline this definition within the Loggly console. To provide feedback, go to the bottom of this page under Was this page helpful?, click No, and then leave comments.
• I’ve sent Numeric fields in my custom JSON but don’t see them. How can I search based on those fields?
  Numeric fields are not exposed through the Dynamic Field Explorer interface. You need to search across numeric fields and ranges using the search bar.
• Why do I only see a subset of values for a certain field, even though I know more values exist?
  Dynamic Field Explorer is an automatically generated summary of the events that exist in your current search context.  The most likely reasons are that the values are the only ones seen during the time range or subset of data. The easiest way to double check is to do a blank search over a wider time range. You should then see all the fields and values that have been detected.
• Why are some fields not showing up in the Field Explorer?
  Double check that your field does not have numeric values. If you are not seeing fields you expect to see, contact Loggly support so SolarWinds can investigate.