Patch vulnerable nodes with Ansible in SolarWinds Observability Self-Hosted Advanced

Remediate OS vulnerabilities without leaving the SolarWinds Platform Web Console. SolarWinds Observability Self-Hosted Advanced integrates with an Ansible server that you manage, so you can start an OS patch playbook for a vulnerable node directly from the security pages and watch the playbook output as it runs.

To use Ansible patch remediation:

1. Add an SSH connection to your Ansible server. The connection uses SSH key-based authentication and deploys the patch playbooks to the Ansible server.

2. Patch a node from the list of vulnerable nodes.

See the following sections:

Requirements

• SolarWinds Observability Self-Hosted Advanced license.

• An existing Ansible server with Ansible installed, including the modules required for the target operating systems (for example, the ansible.windows collection for Windows nodes).

• SSH access from the SolarWinds Platform server to the Ansible server. The default SSH port is 22.

• A user account on the Ansible server that can log in with a username and password. SolarWinds Observability Self-Hosted uses these credentials to deploy an SSH public key to the account.

• The SolarWinds Ansible service must be running on the SolarWinds Platform server.

• The Administrator security role. See View firmware vulnerability and risk information.

Supported operating systems

You can patch nodes running the following operating systems:

• Linux: Debian, Ubuntu, Linux Mint, Kali, Red Hat Enterprise Linux (RHEL), CentOS, Rocky Linux, and AlmaLinux

• Windows

Network devices are not supported.

Add an SSH connection to an Ansible server

Before you can patch nodes, add at least one SSH connection profile for your Ansible server. When you add a connection, SolarWinds Observability Self-Hosted generates an SSH key pair, deploys the public key to the Ansible server, verifies key-based authentication, and copies the patch playbooks to the Ansible server.

1. In the SolarWinds Platform Web Console, click Settings > All Settings.

2. Under Credentials, click Manage Ansible SSH Keys.

3. Click New.

4. On the Add SSH Connection page, enter the connection details:

   • Profile Name: A name that identifies the Ansible server when you select it for a patch.

   • Host Address: The IP address or host name of the Ansible server.

   • Port: The SSH port. The default is 22.

   • Username, Password, and Confirm Password: Credentials for the account on the Ansible server.

5. Click Check Capabilities.

   SolarWinds Observability Self-Hosted connects to the Ansible server over SSH with the username and password and checks the server capabilities.

6. Click Generate & Deploy Public Key.

   A new SSH key pair is generated. The public key is added to the authorized_keys file for the account on the Ansible server. The private key is encrypted and stored in the SolarWinds Platform credential store.

7. Click Verify Connection.

   SolarWinds Observability Self-Hosted reconnects to the Ansible server using only the new SSH key to confirm that key-based authentication works.

8. Click Save SSH Connection.

The connection profile is saved, and the patch playbooks are deployed to the Ansible server, by default to /opt/ansible/orion-scripts. The Manage Ansible SSH Keys page opens and lists the new connection.

Manage SSH connections

The Manage Ansible SSH Keys page lists all SSH connection profiles, with their profile name, host address, port, and username.

You cannot edit an existing connection. To change a connection, delete it and add it again.

Delete SSH connections

1. In the SolarWinds Platform Web Console, click Settings > All Settings.

2. Under Credentials, click Manage Ansible SSH Keys.

3. Select one or more connections, and click Delete.

4. Confirm the deletion.

Patch a vulnerable node

Start an Ansible patch playbook for a Linux or Windows node that has vulnerabilities.

1. Click My Dashboards > Security > Vulnerability and Risk Dashboard, and open the list of nodes.

2. Locate the node you want to patch, and click Patch in the Actions column.

   The Patch Node dialog opens and displays the node IP address, OS type, and the time of the last patch.

3. Select the Ansible server to run the playbook. The list includes only SSH connections that are compatible with the node OS type.

   If the node OS type is not supported, you cannot run the patch, and a message lists the supported vendor families.

4. Select the patch type:

   • Security: Install security updates only. This is the default.

   • Critical Updates: Install critical updates.

   • All: Install all available updates.

5. Optionally, select whether to reboot the node after the patch, and whether to use the credentials stored for the node to connect to it.

6. Click Run Script.

The Ansible output is displayed in the dialog as the playbook runs. When the playbook finishes, the status bar shows whether the patch completed, failed, timed out, or was canceled. To copy the full output, click the copy button.

To stop a running patch, click Cancel.

How patch jobs run

• Patches run in the background. You can close the dialog while a patch runs. While the patch is running, the Patch button for the node displays Processing.

• To see the current status and the output collected so far, click Patch for the same node again.

• You can run only one patch at a time. If a patch is running on another node, the dialog displays a warning and you cannot start a new patch until the running patch finishes or is canceled.

Troubleshoot Ansible patch remediation

If you cannot add an SSH connection or a patch fails, review the Ansible output in the Patch Node dialog, and check the following log files on the SolarWinds Platform server:

• C:\ProgramData\SolarWinds\Logs\Ansible\Ansible.BusinessLayer.log

• C:\ProgramData\SolarWinds\Logs\Orion\BusinessLayerHost-SOLARWINDS.ANSIBLE.BUSINESSLAYER.DLL.log

• C:\ProgramData\SolarWinds\Logs\Orion\BusinessLayerHost.log

• C:\ProgramData\SolarWinds\Logs\Orion\ApolloWebApi.log