Troubleshooting (legacy agent)

The following content pertains to troubleshooting for the legacy AppOptics Ruby Agent.

AppOptics agents are no long receiving updates. The new SolarWinds Observability libraries can send APM data in AppOptics and are regularly updated with new features and improvements. If you are still relying on the AppOptics agents and your components are supported by the new libraries, consider transitioning to the SolarWinds Observability libraries for your APM needs. For more information about the benefits of migrating to the SolarWinds Observability libraries. Alternatively, you can use SolarWinds Observability as your primary APM solution.

If you have already transitioned to the new SolarWinds Observability Ruby Library, see the SolarWinds Ruby Library documentation for troubleshooting information.

SolarWinds Observability libraries are not compatible with AppOptics agents. Do not use a mix of SolarWinds Observability libraries and AppOptics agents to instrument applications that are part of a distributed trace.

If you are not receiving traces and metrics in your AppOptics dashboard these are some options to troubleshoot.

Tracing disabled

The message Missing AppOpticsAPM libraries. Tracing disabled. means that the appoptics_apm gem couldn’t load the core c-extension that is built on gem install. This c extension is only available for Linux platforms (see supported_platforms).

Another cause could be that gem install appoptics_apm or bundle install was run as root and the application is run by a non-root user.

Despite this message, your application will run normally, but no metrics or traces will be collected.

Nothing showing up in my.appoptics.com

A likely cause is, that your service key is not configured correctly. There should be a message in the log saying APPOPTICS_SERVICE_KEY not set. Cannot submit data. or Server responded: Invalid API key

You can find the API Token for your Organization at https://my.appoptics.com/organization/tokens. The service key is a combination of your token and the name you would like to see on the dashboard for your service: <token>:<service name>.

If that is not the problem, please take a look at the response headers in your app.

If they don't contain a X-Trace header, then the instrumentation is in noop mode. Ensure that gem 'appoptics_apm' is the last entry in your Gemfile, and if not running a Rails app, appoptics_apm is required after any gems that need instrumentation.

If there is an X-Trace header, then it's value should be similar to 2B9000C789406646F4EFE825D9240D46C46B95C29CBEBD615BA2951E2F01. If this header ends with 1, the request is traced, if it ends with a 0 only metrics are sent. If it always ends with a 0, then there may be a problem with the sampling rate, either the local one or the one reflected by the collector.

Running behind a proxy or firewall

The agent currently can't send data using a proxy. A workaround is to create a rule in your router or firewall to allow connections to collector.appoptics.com. See the FAQ: My application is behind a firewall. What IP address(es) do I need to whitelist?.

Where are the agent logs?

For Rails and Padrino, the agent uses the respective applications logger, otherwise the appoptics_apm gem logs to STDERR. This behavior can be changed by assigning the AppOpticsAPM.logger to a different logger, e.g, AppOpticsAPM.logger = MyApp.logger during initialization.

More verbose logs

Use export APPOPTICS_DEBUG_LEVEL=6 to set the logging to maximum verbosity level. The levels are:

6 debug high
5 debug medium
4 debug low
3 info (default)
2 warn
1 error
0 fatal
-1 disabled

Simple test of setup

If everything seems setup correctly the following sequence can be tried:

bundle exec irb
require 'appoptics_apm'
target = {}
AppOpticsAPM::SDK.start_trace_with_target(:test, nil, target) {}
target

It should return something like {"X-Trace"=>"2B3669083F409E053615860A12EE0D5BAA4B8F8975D3015AEAD723FC9B01"}. If the string ends with ‘01’, the trace will show up in the dashboard. If it ends with ‘00’, it means that it wasn’t sampled.

Use the newest version

The appoptics_apm gem is under development; new features are added and bugs squashed. Use the newest version. See Release Notes for changes.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.

Search SolarWinds Support