Documentation forSecurity Event Manager

Troubleshoot SEM rules and email responses

This section provides some troubleshooting steps to try if your SEM rules that are not firing as expected or if your rules are not sending the expected notifications.

The rule does not fire and expected alerts do not display

Problem statement: You cannot see the expected InternalRuleFired alerts in the default SolarWinds Alerts or Rule Activity filters on the SEM Console or the alerts needed to fire your rule anywhere in your SEM console.

To determine if the requisite alerts are in your SEM console, create a filter or nDepth search that matches the correlations in your rule.

If the alerts are not present, complete the following procedure:

  1. Review the network devices sending syslog data to the SEM and validate the configurations on that network device to send data. Verify that one of your devices is logging the events you want to capture.

    For example:

    • Remote logging devices, such as firewalls and web filters, should be logging your web traffic events
    • Domain controllers and end-user computers should be logging domain-level and local authentication and change management events

      If you have multiple domain controllers, they will not all replicate every domain event. Each server only logs the events they execute.

    • Other servers, such as database servers and web servers, should be logging events associated with their particular functions.
  2. Verify that the SEM is receiving data.

    Verify that the SEM icons display a syslog or Agent connection. Syslog device IPs display with the icon in the Manage > Nodes grid. Agent host names and IP addresses appear in the Manage > Nodes list with the icon.

    Next, verify that the syslog facility or Agent is receiving data. If a network syslog device is sending syslog data to the SEM, you can view the SEM syslog files for that data.

    1. Open the CMC command line. See Log in to the SEM CMC command line interface for steps.
    2. Type appliance, and enter the checklogs command.

      You can also open a PuTTY session on port 32022 as a cmc user.

    3. View the syslog that was chosen by the network device. All the data received in this area is UDP traffic received on port 514.

  3. If your device is not in the Nodes list, configure your computers by installing a SEM Agent or configure other devices (such as firewalls) to log to your SEM VM or appliance. After your device is in the list, continue to the next step.

  4. If your device is in the Nodes list, configure the appropriate connectors:

    1. Open the SEM legacy Flash console. See Log in to the SEM web console for steps.

    2. On the SEM menu bar, navigate to Manage > Appliances.
    3. Next to the Agent or SEM Manager, click , and then select Connectors.

      Use the Search box at the top of the Refine Results pane to locate the appropriate connectors.

    4. Configure the syslog connector according to your needs.

    5. On the SEM menu bar, navigate to Manage > Nodes.

    6. Next to the Agent, click .

    7. Configure the Agent connector as required.

View and modify the time on your SEM appliance

  1. Open the CMC command line. See Log in to the SEM CMC command line interface for steps.

  2. At the cmc> prompt, enter appliance.

  3. At the cmc::appliance> prompt, enter dateconfig.

  4. Press Enter through all the prompts to view the current date and time settings on your SEM appliance.

    By default, SEM receives a time synchronization from the VM host computer. Without the synchronization, the SEM time is not correct and the rules may not trigger when required.

  5. Disable the time sync on the VM host computer and enable SEM to receive time information from an NTP server.
    1. At the cmc::appliance> prompt, enter ntpconfig and press Enter.

    2. Press Enter to start the configuration script.

    3. Enter the IP addresses of your NTP servers separated by spaces.

    4. Enter y and press Enter to verify your entry.

  6. To leave the CMC interface, enter exit, and then press Enter twice.

The rule is not triggered when it should be

Check your rule logic and timestamps. The SEM VM host layer may need to be configured for NTP. By default, rules will not fire when incoming data drifts more than five minutes from the SEM VM clock.

  1. Open the CMC command line. See Log in to the SEM CMC command line interface for steps.
  2. To enter the appliance menu, type appliance.

  3. Enter the dateconfig command, and confirm the date and time. You can change the time with this command, but when the vSphere/Hyper-V time sync pushes the time to SEM, this will change.