Documentation forAppOptics

Troubleshooting

If you are not receiving traces and metrics in your AppOptics dashboard these are some options to troubleshoot.

Is your platform supported?

  • Please check your application's host environment against our supported platforms. The agent may work on an unsupported platform, but it may also be the cause of crashes or other unexpected behavior.
  • If your platform is not on the list, let us know!

Are requests being sent to the application?

  • Restart the web server or PHP process and ensure that your application is receiving requests. In order for traces and metric data to be sent to your AppOptics dashboard your application must be receiving requests.

Do you have another apm extension installed?

  • It is not recommended to run other APM agent extensions alongside our PHP Agent. Please disable other APM agents from your PHP installation before starting the web server or PHP process with our agent.

How to check the agent extension is enabled?

  • Determine the Server API (SAPI) your PHP is using, then do one of the following to get information about the agent:
    • For the CLI SAPI, run the php -i command
    • For other SAPIs, use the phpinfo() function in a script browseable under your web server
  • The PHP Agent extension name is appoptics, if it is enabled you should see the location of appoptics.ini along with the version and configuration options in effect for the agent.
  • Note that by default the agent is disabled for CLI, it can be enabled via the configuration option appoptics.enable_cli.

Is the agent installed for the correct SAPI?

  • If you used the Install script to install the agent and didn’t specify the parameters --extension-dir, --ini-dir, and --thread-safety the script will attempt to detect these values automatically for your SAPI.

  • In case your system uses a non-thread safe version of CLI PHP and a thread-safe version of Apache/FPM PHP then automatic detection won't work for the thread-safe version. Please expicitly specify the parameters --extension-dir, --ini-dir, and --thread-safety to match your SAPI.

  • For example Amazon Linux 2 uses a non-thread-safe version of CLI PHP and a thread-safe version of Apache PHP. The call to appoptics-php.sh would look like this to install the agent for Apache PHP:

    Copy
    ./appoptics-php.sh --service-key=<SERVICE_KEY> --extension-dir=/usr/lib64/php-zts/modules --ini-dir=/etc/php-zts.d --thread-safety=TS
  • For more information see Installing via wrapper script.

Has the agent been initialized?

  • One way to check if the agent has been initialized, is to look at the response headers returned. The agent will add the header X-Trace to the response. If the response doesn’t contain the header X-Trace, then the agent probably wasn’t loaded correctly.
  • Review the other troubleshooting options to try to resolve the issue.

Is the agent connected to the AppOptics server?

Are requests being sampled for tracing?

  • Requests may not be sampled due to a low sample rate or connection issues. The X-Trace response header ends in 00 if a request isn’t traced and ends in 01 if the request is being traced.
  • Some requests should have a X-Trace header returned that ends in 01. If all X-Trace headers being returned end in 00 the server may not be able to connect to the AppOptics collector.
  • Review the web server logs for any connection issues logged by the agent.

Is the service key configured correctly?

  • The service key is added to the appoptics.ini file under your PHP configuration .ini files directory. The first part of the service key (before the colon :) should match one of the AppOptic API tokens listed in the AppOptics dashboard under Organization Settings > API Tokens.

Where can agent logs be found?

How to increase log verbosity?

  • Logging level can be set via the config option appoptics.debug_level or the environment variable APPOPTICS_DEBUG_LEVEL, which takes the following values: -1 disable, 0 fatal, 1 error, 2 warning (the default), 3 info, 4 debug low, 5 debug medium, 6 debug high. Increase the logging verbosity to one of the debug levels to get more detailed information added to the agent logs.

  • If using the environment variable and running under Apache mod_php, please make this environment variable available at Apache startup. With the default apache2 install on a Debian/Ubuntu system you can set this in the /etc/apache2/envvars file, and with the default httpd install on RHEL/CentOS/Amazon Linux you can set this in the /etc/sysconfig/httpd file:

    Copy
    export APPOPTICS_DEBUG_LEVEL=6
  • Restart the web server or PHP process to have the new level take effect.

  • Log messages from our PHP agent are sent to stderr and have the string [AppOptics-...] at the end of the message.

Are there warnings reported by the agent?

  • Search the web server log files for [AppOptics-...-error or [AppOptics-...-warn for errors and warnings from the agent.

Is your application supported?

  • Out of the box, the agent instruments a wide range of PHP versions and components, please refer to the Support Matrix for details.
  • If your component is not on the list, let us know! And consider using our SDK for custom instrumentation.

Is there a limit set on memory usage?

  • The agent increases the memory usage of the application. Trace data needs to be buffered before it can get sent off to our servers for analysis. If the connection is slow this will cause more buffering. We do have a limit on the buffers but that might be more than your memory limit is set in PHP. If your app is already close to the limit without AppOptics, then running it with AppOptics might just tip over the memory usage above the limit.
  • You can try to
    • increase the PHP memory limit. PHP memory limits can be set with the memory_limit configuration option, either in the INI file (memory_limit = 512M), or in the PHP script itself (ini_set( 'memory_limit', '512M' ))
    • disable backtrace collection. By default the agent generates backtrace data for each request. This can add a great amount of extra data which needs to be buffered. You can disable backtrace by setting this config option in appoptics.ini: appoptics.enable_backtrace = 0
    • reduce number of max transactions. Each transaction within a 30 second period is stored with a histogram object which takes up a good amount of memory. The default of 50 per process can be reduced by changing this config option to a lower value: appoptics.max_transactions = 10
    • reduce buffer sizes. Collected data is internally stored in buffers before sending them to the AppOptics server. These buffers can hold 1,000 messages (status and metrics) and 10 x 1,000 messages (events) by default. The default size of 1,000 can be reduced by setting the environment variable APPOPTICS_BUFSIZE (e.g. a value of 100 will reduce the buffer size for status and metrics to 100 and for events to 1,000).

How to enable/disable tracing for a particular CLI invocation?

Can I use pcntl_fork() in my PHP application?

  • The pcntl_fork() function is currently only supported if a trace is started manually after pcntl_fork() has been invoked.

  • Example of how to start/stop traces in a forked environment:

    Copy
    <?php
      $pid = pcntl_fork();
      
      if ($pid == -1) {
      
        die('could not fork');
      } else if ($pid) {
      
        // in parent process
        appoptics_start_trace('parent');
        echo "this is the parent\n";
        appoptics_end_trace();
        pcntl_wait($status);
        
      } else {
      
        // in child process
        appoptics_start_trace('child');
        echo "this is the child\n";
        appoptics_end_trace();
        
      }
    ?>

Why is there an error about GLIBC?

  • If you encounter this error (or similar):

    Copy
    PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/appoptics.so' - /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /usr/lib64/php/modules/appoptics.so) in Unknown on line 0
  • This can happen if you run AppOptics on an unsupported (old) platform. Please confirm that your platform is supported.

What to do in case of a crash?

  • Please confirm that your platform is supported and that there isn’t another APM extension installed.

  • We do our best to keep the agent extension as bug-free as possible. However, there might be some combination of PHP extensions and PHP frameworks/workflows that escape our testing. If you encounter a crash that only occurs when our agent extension is enabled, please let us know so we can fix it as soon as possible.

  • In order for us to identify the problem we would ask you to

    • create a core dump (see here)

    • run our packcore script on that core dump to collect all relevant binaries (executable + shared libraries) from the system:

      Copy
      wget https://files.appoptics.com/appoptics-php-packcore.sh
      sh appoptics-php-packcore.sh <COREFILE>
  • This will result in a .tar.xz archive which you can send to support@appoptics.com for analysis. If possible, please also provide a PHP script that helps us to reproduce the error.

When the APM Integrated Experience is enabled, AppOptics shares a common navigation and settings with the other integrated experiences' products. How you navigate AppOptics and access its features may vary from these instructions. For more information, go to the APM Integrated Experience documentation.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.