Documentation forSecurity Event Manager

Troubleshoot alerts in the SEM Console

This section describes how to troubleshoot unmatched data or internal new connector data alerts that may appear in your SEM console.

Typically, unmatched data and internal new connector data alerts indicate that one or more of the connectors on the SEM VM or appliance cannot properly normalize the associated log data.

  1. Ensure that your syslog devices are sending logs to a syslog facility on your SEM appliance.
  2. Determine which devices are logging to each facility, and whether those devices conflict with each another.
  3. Ensure that your SEM Agent connectors, such as Windows-based and database connectors are running correctly.
  4. Apply the latest connector update package.
  5. Generate a syslog sample from the SEM appliance, and then open a ticket with SolarWinds Technical Support for further assistance.

Task 1: Troubleshoot syslog devices

Complete the following troubleshooting procedures for devices that send logs to a syslog facility on your SEM appliance.

  1. Verify that the connector and device are pointed at the same local facility.

  2. Check the configuration on your device to determine what local facility it is logging to on your SEM appliance. In some cases, you cannot modify this setting.

    For additional information, search for your device in the SolarWinds Success Center. Except for CheckPoint firewall, the SEM receives UDP syslog data on port 514.

  3. Verify that the connector is pointed to the same logging facility as the device.

    1. On the SEM Console, navigate to Configure > Manager Connectors.

    2. Under Configured connectors, locate the connector in the list.

    3. Select the configured connector, and then click Edit.

    4. View its details, and verify the Log File value matches the output value in the device configuration.
  4. If the device and connector configurations do not match, point the connector to the appropriate location.

    1. Select the configured connector, and then click Stop.

    2. Click Edit, and then change the Log File value so it matches your device.

    3. Click Save, and then click Start.

Task 2: Troubleshoot device logging

Certain devices (including Cisco devices) have similar logging formats that cause connector conflicts when logging to the same facility on your SEM appliance. Use the following procedure and table to determine what devices are logging to each facility, and whether those devices conflict with one another.

  1. Ope the CMC command line.

    See Log in to the SEM CMC command line interface for directions.

  2. At the cmc> prompt, type appliance.

  3. At the cmc::appliance> prompt, type checklogs.

  4. Enter an item number to select and view a local facility.

  5. To view the device sending the event, open the log facility.

    The EPOCH timestamp (1427722392000) starts each event, which is the date and time in Unix numeric format. The device sending the event follows. You will typically see ProviderSID (ASA-1-106021), which is similar to an Event ID.

  6. If two or more devices are logging to the same facility, see Troubleshoot conflicting devices" below to determine whether those devices conflict with each other.

Troubleshoot conflicting devices

Different firewall types should log to different facilities. For example, Cisco firewalls and Palo Alto should log to different facilities.

However, both devices should log to their own facilities. Ensure that the devices in each of these groups are logging to distinct local facilities on your SEM VM. For example, if a device in Group 1 is logging to local1, make sure a device in Group 2 is not also logging to that facility.

SolarWinds recommends splitting the devices and vendors to different facilities. Having all devices pointed at one facility with multiple connectors reading that facility will impact your SEM performance.

Group Devices
Group 1 Cisco ASA
Cisco IOS
Cisco PIX
Group 2 Cisco Catalyst (CatOS)
Group 3 Cisco Wireless LAN Controller (WLC)
Group 4 Cisco Nexus
Group 5 Cisco VPN
Group 6 Dell PowerConnect

Task 3: Troubleshoot Agent devices and connectors

Complete the following steps to troubleshoot SEM Agent connectors, such as Windows-based and database connectors.

  1. Verify the connector is pointing to the appropriate folder or event log.

  2. Check the configuration on the host computer to determine which folder or event log it is logging in to.

    In some cases, you cannot modify this setting. For additional information, search the SolarWinds Success Center for your device.

  3. Verify that the connector is pointed to the same folder or event log as the device:

    1. On the SEM Console, navigate to Configure > Nodes.

    1. Under Refine Results, expand the Type group, and then select the Agent check box.

    2. Select the SEM Agent for the host computer, and then click Manage node connectors.

    3. Locate the configured connector in the list.

    4. Select the configured connector, and then click edit.

    5. View its details, and ensure the Log File value matches the output value in the host computer configuration.
  4. If the host computer and connector configurations do not match, point the connector to the appropriate location:

    1. Select the configured connector, and then click Stop.

    2. Click Edit, and then change the Log File value so it matches your device.

    3. Click Save, and then click Start.

Task 4: Apply the latest connector update package

If you completed the procedures in this section and you still see the unmatched data or internal new connector data alerts, apply the latest connector package before you contact Technical Support.

See Apply a SEM connector update package to learn how.

Task 5: Contact SolarWinds Technical Support

If you are unable to resolve your issue using this article, open a ticket with SolarWinds Technical Support for further assistance. Be prepared to provide the following information to a support technician:

  • A copy of the SEM report (in Crystal Reports format) entitled Tool Maintenance by Alias for the last 24 hours or the period during that the unmatched data was detected.
  • (Syslog devices only). A sample of the logs currently sent to SEM for the affected connector. For more information, see Export log files using the CMC exportsyslog command.
  • (Windows connectors only). A copy of the entire event log in English and EVTX formats.
  • (Database connectors only). A sample of the event table containing the unread events and the details about these events.
  • (Database connectors only). The database schema (if available).

Generate a syslog sample from the SEM appliance

  1. Open the CMC command line.

    See Log in to the SEM CMC command line interface for directions.

  2. At the cmc> prompt, type appliance.

  3. At the cmc::appliance> prompt, type exportsyslog.

  4. Enter an item number to select a local facility to export.

  5. Repeat the previous step to specify more than one facility.

  6. Enter q to proceed.

  7. Follow the on-screen instructions to complete the export.