Documentation forAppOptics

Kubernetes

Overview

This plugin collects runtime metrics from Kubernetes masters and nodes within its cluster. It gathers information about resource usage and performance characteristics.

This plugin is currently available for x86_64 Linux and Windows platforms.

Setup

The kubernetes plugin is included with the SolarWinds Snap Agent by default, please follow the directions below to enable it on a given host (each Kubernetes Master).

Prerequisites

This plugin requires that the agent user solarwinds has access to a valid kubeconfig and kube-apiserver.

A kubeconfig file is used to configure access to Kubernetes clusters and is commonly found in ~/.kube/config. kubeconfig is a generic way of referring to the Kubernetes configuration files.

There are different ways to give the agent access to Kubernetes clusters (to the Kubernetes API). One method is to copy your ~/.kube folder to /opt/SolarWinds/Snap/etc:

Copy
$ cd /opt/SolarWinds/Snap/etc
$ sudo cp -r ~/.kube .
$ sudo chown -R solarwinds:solarwinds .kube

To check that the agent has access to the Kubernetes API, you can run the following command as the solarwinds user, which retrieves the list of all pods in the current namespace:

Copy
$ sudo -u solarwinds kubectl get pods --kubeconfig /opt/SolarWinds/Snap/etc/.kube/config

If the command does not execute successfully, ensure:

  1. both the symlinks and source files are owned by the solarwinds group and user.
  2. the configured kubeconfigpath is valid.

Configuration

The agent provides an example configuration file to help you get started quickly. It defines the plugin and task file to be loaded by the agent, but requires you to provide the correct settings for your Kubernetes deployment. To enable the plugin:

  1. Make a copy of the kubernetes example configuration file /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml.example, renaming it to /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml:

    Copy
    $ sudo cp /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml.example /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml
  2. Update the /opt/SolarWinds/Snap/etc/plugins.d/kubernetes.yaml configuration file with settings specific to your Kubernetes deployment, for example:

    Copy
    collector:
      kubernetes:
        all:
          incluster: false
          kubeconfigpath: "/opt/SolarWinds/Snap/etc/.kube/config"
          interval: "60s"
          events: |
            # Embedded YAML (as a multiline string literal)
            filters:
            - namespace: default
              type: warning
              count: 2
            - namespace: kube-system
              type: warning
            file: /var/log/SolarWinds/Snap/events.log
    • incluster has to be set to false unless when running as a pod or daemonset.

    • kubeconfigpath a valid path to Kubernetes config files (e.g. ~/.kube/config).

    • interval sets time interval between subsequent metrics submissions (optional, defaults to 60s).

    • events enables Kubernetes Cluster Events collection (together with filters subfield). It’s an embedded YAML dictionary as string literal. It consists of:

      • filters list of dictionaries that allows for events filtering based on equality check on following fields:

        • action
        • count
        • host
        • kind
        • namespace
        • reason
        • reportingController
        • reportingInstance
        • type
      • file specifies filepath where the events will be written to, defaults to /var/log/SolarWinds/Snap/events.log.

  3. Restart the agent:

    Copy
    $ sudo service swisnapd restart
  4. Enable the Kubernetes plugin in the AppOptics UI

    On the Integrations Page you will see the Kubernetes available if the previous steps were successful. If you do not see the plugin, see Troubleshooting Linux.

    Select the Kubernetes plugin to open the configuration menu in the UI, and enable the plugin.

    You should soon see the kubernetes metrics reported to your dashboard.

Metrics and Tags

All the metrics are fetched via the Kubernetes API. The tables below outline the default set of metrics collected by the kubernetes plugin along with the optional metrics available.

Pod Metrics

Namespace Description
kubernetes.pod.status.condition.ready specifies if the pod is ready to serve requests
kubernetes.pod.status.condition.scheduled status of the scheduling process for the pod
kubernetes.pod.status.phase.Pending This includes time before being bound to a node, as well as time spent pulling images onto the host
kubernetes.pod.status.phase.Running The pod has been bound to a node and all of the containers have been started.
kubernetes.pod.status.phase.Succeeded All containers in the pod have voluntarily terminated with a container exit code of 0, and the system is not going to restart any of these containers
kubernetes.pod.status.phase.Failed All containers in the pod have terminated, and at least one container has terminated in a failure
kubernetes.pod.status.phase.Unknown For some reason the state of the pod could not be obtained, typically due to an error in communicating with the host of the pod

Pod Metrics Tags

Tag Name Description
hostname Name of the host. Instead of using this tag we recommend using the @host alias
Pod Name of container group (Pod)
Node Kubernetes node name
Namespace Kubernetes namespace

Container Metrics

Namespace Description
kubernetes.container.requested.cpu.cores The limit on cpu cores to be used by a container.
kubernetes.container.limits.memory.bytes The limit on memory to be used by a container in bytes.
kubernetes.container.requested.cpu.cores The number of requested cpu cores by a container.
kubernetes.container.requested.memory.bytes The number of requested memory bytes by a container.
kubernetes.container.status.ready specifies whether the container has passed its readiness probe
kubernetes.container.status.restarts number of times the container has been restarted
kubernetes.container.status.running value 1 if container is running else value 0
kubernetes.container.status.terminated value 1 if container is terminated else value 0
kubernetes.container.status.waiting value 1 if container is waiting else value 0

Container Metric Tags

Tag Name Description
hostname Name of the host. Instead of using this tag we recommend using the @host alias
pod Name of container group (Pod)
node Kubernetes node name
namespace Kubernetes namespace
container Kubernetes container name

Node Metrics

Namespace Description
kubernetes.node.spec.unschedulable Whether a node can schedule new pods.
kubernetes.node.status.allocatable.cpu.cores The CPU resources of a node that are available for scheduling.
kubernetes.node.status.allocatable.memory.bytes The memory resources of a node that are available for scheduling.
kubernetes.node.status.allocatable.pods The pod resources of a node that are available for scheduling.
kubernetes.node.status.capacity.cpu.cores The total CPU resources of the node.
kubernetes.node.status.capacity.memory.bytes The total memory resources of the node.
kubernetes.node.status.capacity.pods The total pod resources of the node.
kubernetes.node.status.outofdisk True if there is insufficient free space on the node for adding new pods, otherwise False

Node Metric Tags

Tag Name Description
hostname Name of the host. Instead of using this tag we recommend using the @host alias
Node Kubernetes node name

Deployment Metrics

Namespace Description
kubernetes.deployment.metadata.generation The desired generation sequence number for deployment. If a deployment succeeds should be the same as the observed generation.
kubernetes.deployment.status.observedgeneration The generation sequence number after deployment.
kubernetes.deployment.status.targetedreplicas Total number of non-terminated pods targeted by this deployment (their labels match the selector).
kubernetes.deployment.status.unavailablereplicas Total number of unavailable pods targeted by this deployment.
kubernetes.deployment.status.availablereplicas Total number of available pods (ready for at least minReadySeconds) targeted by this deployment.
kubernetes.deployment.spec.desiredreplicas Number of desired pods.
kubernetes.deployment.status.deploynotfinished If desired and observed generation are not the same, then either an ongoing deploy or a failed deploy
kubernetes.deployment.spec.paused An optional boolean field for pausing and resuming a Deployment.
kubernetes.deployment.status.updatedreplicas Total number of non-terminated pods targeted by this deployment that have the desired template spec

Deployment Metric Tags

Tag Name Description
hostname Name of the host. Instead of using this tag we recommend using the @host alias
namespace Kubernetes namespace
deployment Kubernetes deployment name

Events Metrics

Namespace Description
kubernetes.events.count Total number of collected Kubernetes cluster events.

When the APM Integrated Experience is enabled, AppOptics shares a common navigation and settings with the other integrated experiences' products. How you navigate AppOptics and access its features may vary from these instructions. For more information, go to the APM Integrated Experience documentation.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.