Documentation forServer & Application Monitor
Monitoring your applications and environment is a key capability of SolarWinds Observability Self-Hosted (formerly Hybrid Cloud Observability) and is available in the Essentials edition. Server & Application Monitor (SAM) is also available in a standalone module.

Linux Memory Monitoring Perl

This SAM application monitor template assesses the memory performance of a of Linux servers through the use of Perl scripts.

This template supports the SolarWinds Platform Agent for Linux.

Prerequisites

SSH and Perl are installed on the target server.

If Perl is installed in a location different from /usr/bin/perl, edit paths in the first line of the field ”script body” (#!/usr/bin/perl) for component monitors, or create a symbolic link to Perl with ln command.

To determine where Perl is installed, use the following command: which perl

Operating Systems

The following OS versions may cause issues with certain counters in this template:

  • Red Hat Enterprise Linux 3, 4
  • SUSE Linux Enterprise Server 9

Credentials

Root credentials on the target server.

Component monitors

Components without predetermined threshold values have guidance such as "Use the lowest threshold possible" or "use the highest threshold possible" to help you find a threshold appropriate for your application. To learn more, see Manage thresholds in SAM.

Total memory (kB)

The amount of total usable RAM in kB.

Used memory (kB)

The amount of used memory in kB.

This should be as low as possible.

Free memory (kB)

The amount of available memory in kB.

This should be more than 100000 kB at all times or paging will occur.

Total swap (kB)

The amount of total swap space in kB.

Used swap (kB)

The amount of used swap space in kB. Paging is one of the memory-management schemes by which a computer can store and retrieve data from secondary storage for use in main memory. Paging is an important part of virtual memory implementation in most contemporary general-purpose operating systems, allowing them to use disk storage for data that does not fit into physical random-access memory (RAM).

This should be as low as possible. Ideally it should be near zero. If the value is large, it may indicate that there is no free memory left.

Free swap (kB)

The amount of available swap space in kB.

It should be near the “total swap” value. If paging occurs in the system, the value should be as high as possible.

Buffers (kB)

RAM that is allocated to disk write operations.

Cache (kB)

RAM that is allocated to disk read operations.

Dirty Pages (kB)

The total amount of memory, in kilobytes, waiting to be written back to the disk.

This should be as low as possible.

You should monitor this counter for some time and then set thresholds appropriately for your environment.

Anonymous Pages (kB)

Anonymous memory is memory that is managed by segvn but is not really directly associated with a file. It is used for things like process stacks, heap, or COW (copy on write) pages. A good example of this is if you fork a process. All the addresses in the second process actually map back to the same bits of physical memory (the same pages). However if your child process was then to do something different with the memory (for example the child went off and manipulated an array in memory), the VM subsystem would copy those pages and change the mappings in the child process to point to the new pages. This new memory would be anonymous memory, and the child process would merrily make the changes to the array, unaware it now had new "physical" memory it was addressing.

This should be as low as possible. You should monitor this counter for some time and then set thresholds appropriately for your environment.

Amount of zombie processes

The number of “zombie” processes. A “zombie” or defunct process is a process that has completed execution, but still has an entry in the process table. This entry is still needed to allow the process that started the (now zombie) process to read its exit status. When a process finishes execution, it will have an exit status to report to its parent process. Because of this last little bit of information, the process will remain in the operating system’s process table as a zombie process, indicating that it is not to be scheduled for further execution, but that it cannot be completely removed (and its process ID cannot be reused) until it has been determined that the exit status is no longer needed.

This should always be zero (0). If it is not zero, you should manually kill zombie processes. Use the following command to show these zombie processes (and look for a Z in the STAT column): ps aux

To kill zombie processes:

  • The first option is to wait. It is possible that the parent process is intentionally leaving the process in a zombie state to ensure that future children it may create will not receive the same pid. Or perhaps the parent is occupied, and will reap the child process momentarily.
  • The second option is to send a SIGCHLD signal to the parent (kill -s SIGCHLD <ppid>). This will cause well-behaving parents to reap their zombie children.
  • The third option is to kill the parent process of the zombie. At that point, all of the parent’s children will be adopted by the init process (pid 1), which periodically runs wait() to reap any zombie children.