Configure SEM to store original log messages (nDepth log retention)

SEM can store raw (unnormalized) log messages for retention and search purposes. To enable this feature, configure the SEM Manager and the applicable connectors accordingly.

nDepth log retention refers to storing raw data (that is, original log messages) in a separate database. Other than the name, nDepth log retention is separate from the nDepth search engine that is available in the SEM console under Explore > nDepth.

About nDepth log retention

This section describes nDepth log retention.

Why use a separate nDepth VM?

A separate nDepth appliance provides additional capacity to store and retrieve raw log messages. If long-term storage of original log messages is a priority, then consider a separate nDepth VM. Otherwise, a separate instance is probably unnecessary. For more information contact your SolarWinds sales representative or SolarWinds Technical Support.

  • Rules do not fire on raw (unnormalized) log data. Rules can only fire on normalized data.
  • Raw (unnormalized) log messages do not appear in Monitor view in the Console.
  • If you enable original log storage (raw database storage), and you enable connectors to send data to both databases, SEM storage requirements may double for the same retention period, and extra resource reservations of at least two additional CPUs and 8-16GB of RAM may be required.

Install a separate nDepth appliance or VM

In this configuration, each SEM Manager has its own dedicated nDepth appliance or VM that stores the original log files from each host (network device) and source (application or connector) that the SEM Manager monitors. You still access and explore this information using the SEM console's nDepth view even though it resides in a separate appliance or VM.

  • To use a separate nDepth appliance or VM, you must install it before you begin using nDepth. Contact SolarWinds Technical Support for instructions on installing a separate appliance.

  • If you are not using a separate appliance, this procedure is not required, because short-term log messages are stored directly on SEM.

Configure network connectors for use with nDepth

Each data-gathering connector (or sensor connector) must be configured for use with nDepth log retention. First, decide which network devices, applications, and connectors monitored by the Manager should send raw log messages to nDepth. Next, configure each of these connectors for use with nDepth. You can route connector log messages directly to SEM, directly to nDepth, or to both.

See Configure connectors to send original log data to SEM for more information.

SolarWinds recommends configuring each connector so it routes its log messages to both nDepth and SEM. This allows you to receive events on these connectors, and to search log messages stored on the separate nDepth instance.

Configure SEM Manager to store original log files in their own database

The following procedure must be completed prior to configuring any connector to send log messages to your SEM appliance.

  1. Open the CMC command line. See Log in to the SEM CMC command line interface for steps.
  2. At the cmc> prompt, enter manager.

  3. At the cmc::manager> prompt, enter configurendepth, and then follow the prompts to configure your SEM Manager to use an nDepth server.
    1. Enter y at the Enable nDepth? prompt.

    2. If you are prompted with Run nDepth locally? (Recommended), enter y. This will configure a separate database on your SEM appliance to store original log files.

    3. If your SEM impsementation consists of several appliances, follow the prompts to complete the process for your dedicated database or nDepth appliance. For additional information about this process, contact Support.

  4. At the cmc::manager> prompt, enter exit to return to the previous prompt.

  5. At the cmc> prompt, enter ndepth.

  6. At the cmc::nDepth# prompt, enter start. This command will start the Log Message search/storage service.

  7. To return to the previous prompt, enter exit.

  8. To log out of your SEM appliance, enter exit.

Configure connectors to send original log data to SEM

  1. Open the connector for editing in the Connector Configuration window for the SEM Manager or SEM Agent, as applicable.
    • If the connector has already been configured, stop the connector by clicking > Stop, and then click > Edit.

    • If the connector has not been configured, create a new instance of the connector by clicking > New next to the connector you want to configure.

  2. In the Connector Details pane, change the Output value to Alert, nDepth. Leave the nDepth Host and nDepth Port values alone unless otherwise instructed by Support.

    The Output values are defined as:

    • Alert: Sending data to the alert database

    • nDepth: Sending data to the RAW (original log) database

    For help, see The Connector Configuration form fields for data-gathering (sensor) connectors

  3. If you are finished configuring the connector, click Save.

  4. To start the connector, click , and then select Start.

  5. To close the Connector Configuration window, click Close.

  6. Repeat these steps for each connector you want to send original log data to your SEM appliance.

View and search your original log messages

See Search raw log messages using nDepth search in SEM for details.