Troubleshoot other Agent issues
Connection issues
If the Agent status is Disconnected, it might be caused by a connection issue.
-
Check the Agent logs. Search for issues related to accessing endpoints or blocked ports. See Collect Agent logs from target hosts.
On target hosts, log files are stored in the following default locations:
- Windows:
C:\ProgramData\SolarWinds\UAMSClient\log\uamsclient.log - Linux:
/var/log/solarwinds/uamsclient/uamsclient.log
- Windows:
-
Make sure that the firewall on the host allows access to allSolarWinds Observability SaaS endpoints. See Data centers and endpoint URIs.
Collect Agent logs from target hosts
-
In SolarWinds Observability SaaS, click Settings > Agents.
-
Click the vertical ellipsis at the end of a SolarWinds Observability Agent row and select Collect Logs. Agent logs available on the target host start to be downloaded to SolarWinds Observability SaaS.
-
When the download is complete, you will see a download logs button next to the agent. Click the button to download the logs to your computer.
Agent logs are downloaded as a ZIP archive and are available for download in SolarWinds Observability SaaS for 24 hours. To get a fresh set of logs, repeat the steps above.
Multiple items for one Agent on Agents page
When you see multiple items for one SolarWinds Observability Agent on the Agents page, configure your deployment to override the Client ID.
By default, Agent ID is tied to the external MAC address of the target host. Once the host reboots, a new Client ID is created and a new Agent item based on the ID is displayed in Agents.
To resolve the issue, override the client ID using the uamsclient-ctl tool or an environment variable.
Use uamsclient-ctl tool to override the client ID
To use a randomly generated client ID, run the following code:
Windows PowerShell for a random client ID
C:\'Program Files'\SolarWinds\UAMSClient\uamsclient-ctl.exe bind-new-identity
Linux for a random client ID
sudo -u swagent /opt/solarwinds/uamsclient/sbin/uamsclient-ctl bind-new-identity
To use a specific client ID, replace <your UUID> with the ID and run the code:
Windows PowerShell for a specific client ID
C:\'Program Files'\SolarWinds\UAMSClient\uamsclient-ctl.exe bind-identity -id <your UUID>
Linux for a specific client ID
sudo -u swagent /opt/solarwinds/uamsclient/sbin/uamsclient-ctl bind-identity -id <your UUID>
When you set the new identity, the Agent will not be replicated on the Agents page for each reboot. However, if you set the client ID value in the environment variable, the variable value will overwrite the value set by the uamsclient-ctl tool.
Set the UAMS_CLIENT_ID_OVERRIDE [string] variable
Set the client ID value in the UAMS_CLIENT_ID_OVERRIDE [string] variable. If the variable is defined during the client start, the value will be used. This value has priority over the client-id-override value in the dynamic configuration file.
When you set the UAMS_CLIENT_ID_OVERRIDE [string] variable, the defined value will always be used and the Agent will not be replicated on the Agents page for each reboot. The UAMS_CLIENT_ID_OVERRIDE variable must be in the Universally Unique Identifier (UUID) format
Potentially duplicated Agents
The Potentially duplicated Agent detected Spotlight message displayed on the Agent list indicates that multiple Agents across different environments are using the same Agent ID. This can cause several types of inconsistencies and misconfigurations, for example:
- Agents from different environments can appear as a single Agent on the Agents page.
- Monitoring data can appear inconsistent or merged.
- Changing an Agent's configuration can affect multiple environments unexpectedly.
Resolving this is slightly different depending on the Agent deployment method, but all options require you to go to all your Agent deployments and verify the value of the Agent ID override using the management interface. See Manage SolarWinds Observability Agents for more information.
Perform changes only on confirmed duplicate hosts. Keep the identity of the primary host unchanged.
Find duplicated Agents
There are two different methods available for finding duplicated Agents.
Using the Spotlight widget
-
Open the Data collection page and select the Agents tab.
-
Review the Spotlight section of the Overview widgets for the Potentially duplicated agents message.
-
Select the number of the item to filter the list to the affected Agents.
Using the filter available on the Agents tab
-
Open the Data collection page and select the Agents tab.
-
Choose the Duplicated value for the Agent status filter.
-
Review the Agent status column for the warning icon and the Duplicated label.
Understanding potential duplicates
The Potential duplicates tab on the Agent inspector panel helps you determine which host should retain the existing identity and which hosts require new identities.
-
Select an affected Agent to open the Agent inspector panel.
-
Open the Potential duplicates tab.
| Field | Details |
|---|---|
| Running longer |
Shows the host instance that has been running the longest. In most cases, the host that has been running the longest is the original or primary deployment. It should retain the existing identity. |
| Potential duplicates |
Lists other host instances that registered with the same Agent ID. These hosts typically require new identities. |
Fix duplicated Agents
The Fix button, located next to the Potentially duplicated Agents message in the Spotlight section helps you find and resolve duplicated Agents. Clicking the Fix button allows you to see the following:
- What happened.
- The list of potentially duplicated Agents.
- Steps to resolve the platforms hosting the duplicated Agents.
To fix potentially duplicated Agents, take the following steps:
-
Identify the primary host.
- In the Potential duplicates tab, compare the host details for Running longer and Potential duplicates.
- Confirm the systems with your infrastructure inventory.
- Designate the original or correct host as the Primary host.
- Designate all other instances as Duplicate hosts.
-
Assign a new identity to duplicate hosts.
Perform changes only on confirmed duplicate hosts. Running the provided identity command on the primary host changes the Agent identity you intended to preserve.
-
Windows:
-
Open PowerShell as an administrator.
-
Run the following command:
C:\'Program Files'\SolarWinds\UAMSClient\uamsclient-ctl.exe bind-new-identity
-
Restart the Agent service:
Restart-Service swiuamsclientd
-
-
Linux:
-
Open a terminal on the duplicate host.
-
Run the following:
sudo -u swagent /opt/solarwinds/uamsclient/sbin/uamsclient-ctl bind-new-identity
-
Restart the Agent service:
sudo systemctl restart uamsclient.service
-
-
Docker containers:
-
Stop the duplicate container.
-
Re-run the container using a unique
UAMS_CLIENT_ID_OVERRIDEfor each instance. See Install SolarWinds Observability Agent as a Docker image -
Confirm the value is not hardcoded in a shared image or manifest.
-
For images, run
docker image inspect <image> --format '{{json .Config.Env}}'and confirmUAMS_CLIENT_ID_OVERRIDEis not baked into the image. -
For Docker compose or manifests, check
docker-compose.yaml,.env, Kubernetes manifests, ConfigMaps, Secrets, or Helm values withgrep -rnUAMS_CLIENT_ID_OVERRIDE. Ensure no static literal ID is reused across containers or pods. -
Ensure containers do not share the same persistent Agent work directory, such as
/uamsclient/workdiror/opt/solarwinds/uamsclient/etc.
-
-
Start the container again.
-
-
Verify duplicate resolution
Allow up to one hour for Agent registrations and historical duplicate indicators to reconcile.
To verify duplicate Agents have been successfully resolved, return to the Data collection page and select the Agents tab.
-
Confirm that each host appears as a separate Agent entry.
-
Verify that the Potentially duplicated Agents message is no longer shown in the Spotlight section.
-
Open the primary Agent's Inspector panel. Confirm the Agent status no longer contains the Duplicated label and that the Potential duplicates tab is no longer visible.
-
Check that all relevant Agents report a healthy status and current telemetry.
If the Potential duplicates tab remains visible but only a single Agent is detected with no duplicates present, restart the Agent to verify proper functionality and ensure the status is updated correctly.
Once the duplicate agent issue has been resolved, the Potential duplicates tab should disappear from the Inspector panel. If the tab remains visible, but only a single Agent is detected with no duplicates present, restart the Agent to verify proper functionality and ensure the status is updated correctly.
Best practices to prevent duplicated Agents
- Always use a unique
UAMS_CLIENT_ID_OVERRIDEorUAMS_BIND_IDENTITYper environment. - Avoid cloning VMs with pre-installed and configured Agents.
- Treat restored system images as new hosts unless the original host is permanently retired.
- Automate the generation of unique Agent IDs in deployment scripts or infrastructure-as-code templates.
- See Manage Agent on virtual machines for information on preventing the duplication of Agent IDs when running the Agent on a virtual machine.