Kubernetes

Overview

This plugin collects runtime metrics from Kubernetes master and worker nodes within its cluster. It gathers information about resource usage and performance characteristics.

Setup

The Kubernetes integration is included as part of a SWI collector. The SWI collector bundles several collectors into one integrated binary.

The kubernetes plugin is included with the SolarWinds Snap Agent by default, please follow the directions below to enable it on a given host (each Kubernetes Master).

With Snap Agent version 4.5.0 and later, this plugin was upgraded to use the v2 plugins framework. If you are running v1 of the collector, see AppOptics Snap Agent 4.4.0 documentation.

To migrate to the v2 plugins framework, update Snap Agent to the latest version and see Migrate from v1 to v2 appoptics publisher plugins.

Prerequisites

This plugin requires that the agent user solarwinds has access to a valid kubeconfig and kube-apiserver.

A kubeconfig file is used to configure access to Kubernetes clusters and is commonly found in ~/.kube/config. kubeconfig is a generic way of referring to the Kubernetes configuration files.

There are different ways to give the agent access to Kubernetes clusters (to the Kubernetes API). One method is to copy your ~/.kube folder to /opt/SolarWinds/Snap/etc:

cd /opt/SolarWinds/Snap/etc
sudo cp -p -r ~/.kube .
sudo chown -R solarwinds:solarwinds .kube

To check that the agent has access to the Kubernetes API, you can run the following command as the solarwinds user, which retrieves the list of all pods in the current namespace:

sudo -u solarwinds kubectl get pods --kubeconfig /opt/SolarWinds/Snap/etc/.kube/config

If the command does not execute successfully, ensure:

Both the symlinks and source files are owned by the solarwinds group and user.
The configured kubeconfigpath is valid.

Configuration

The SolarWinds Snap Agent provides an example task file with basic Kubernetes metrics to help you get started quickly. Modify the example task file with the correct settings for your Kubernetes setup. To enable the plugin:

Make a copy of the Kubernetes example task file task-swi-kubernetes.yaml.example, renaming it to task-swi-kubernetes.yaml.

On Windows, run the following in PowerShell or the command line:

copy "C:\ProgramData\SolarWinds\Snap\tasks-autoload.d\task-swi-kubernetes.yaml.example" "C:\ProgramData\SolarWinds\Snap\tasks-autoload.d\task-swi-kubernetes.yaml"

On Linux, run the following in the command line:

sudo cp -p /opt/SolarWinds/Snap/etc/tasks-autoload.d/task-swi-kubernetes.yaml.example /opt/SolarWinds/Snap/etc/tasks-autoload.d/task-swi-kubernetes.yaml

Update the task-swi-kubernetes.yaml task file with settings specific to your Kubernetes setup. For example:

if your Kubernetes deployment is not running as a pod or daemonset, incluster should be set to false
kubeconfigpath should be set to a valid path to the Kubernetes config files (for example, /home/username/.kube/config)

---
version: 2

schedule:
  type: simple
  interval: "60s"

plugins:
  - plugin_name: swi

    config:
      kubernetes:
        incluster: false
        kubeconfigpath: ""

    publish:
      - plugin_name: publisher-appoptics

    metrics:
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/phase/Pending
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/phase/Running
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/phase/Succeeded
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/phase/Failed
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/phase/Unknown
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/condition/ready
      - /kubernetes/pod/[namespace]/[node]/[pod]/status/condition/scheduled
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/status/restarts
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/status/ready
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/status/waiting
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/status/running
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/status/terminated
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/requested/cpu/cores
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/requested/memory/bytes
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/limits/cpu/cores
      - /kubernetes/container/[namespace]/[node]/[pod]/[container]/limits/memory/bytes
      - /kubernetes/node/[node]/spec/unschedulable
      - /kubernetes/node/[node]/status/outofdisk
      - /kubernetes/node/[node]/status/capacity/cpu/cores
      - /kubernetes/node/[node]/status/capacity/memory/bytes
      - /kubernetes/node/[node]/status/capacity/pods
      - /kubernetes/node/[node]/status/allocatable/cpu/cores
      - /kubernetes/node/[node]/status/allocatable/memory/bytes
      - /kubernetes/node/[node]/status/allocatable/pods
      - /kubernetes/deployment/[namespace]/[deployment]/metadata/generation
      - /kubernetes/deployment/[namespace]/[deployment]/status/observedgeneration
      - /kubernetes/deployment/[namespace]/[deployment]/status/targetedreplicas
      - /kubernetes/deployment/[namespace]/[deployment]/status/availablereplicas
      - /kubernetes/deployment/[namespace]/[deployment]/status/unavailablereplicas
      - /kubernetes/deployment/[namespace]/[deployment]/status/updatedreplicas
      - /kubernetes/deployment/[namespace]/[deployment]/spec/desiredreplicas
      - /kubernetes/deployment/[namespace]/[deployment]/spec/paused
      - /kubernetes/deployment/[namespace]/[deployment]/status/deploynotfinished

Restart the agent.

On Windows, run the following in the command line:
```
net stop swisnapd
net start swisnapd
```
On Linux, run the following in the command line:
```
sudo service swisnapd restart
```
Enable the Kubernetes plugin in AppOptics.

If you completed these steps successfully, after a few minutes the Kubernetes plugin will be listed on the Integrations page. If the plugin is not included on the page, see Troubleshooting Linux.

Select the Kubernetes plugin to open the configuration menu in the user interface, and then enable the plugin. When completed, the kubernetes metrics are available on the dashboard.

To retrieve logs from the swisnap agent pod, use one of the following commands:

kubernetes logs -n kube-system swisnap-agent-<name>

docker logs <docker-container-name>

Metrics and Tags

All metrics are fetched via the Kubernetes API. The tables below outline the default set of metrics collected by the kubernetes plugin, along with the optional metrics available.

Pod

Pod Metrics

Namespace	Description
kubernetes.pod.status.condition.ready	specifies if the pod is ready to serve requests
kubernetes.pod.status.condition.scheduled	status of the scheduling process for the pod
kubernetes.pod.status.phase.Pending	This includes time before being bound to a node, as well as time spent pulling images onto the host
kubernetes.pod.status.phase.Running	The pod has been bound to a node and all of the containers have been started.
kubernetes.pod.status.phase.Succeeded	All containers in the pod have voluntarily terminated with a container exit code of 0, and the system is not going to restart any of these containers
kubernetes.pod.status.phase.Failed	All containers in the pod have terminated, and at least one container has terminated in a failure
kubernetes.pod.status.phase.Unknown	For some reason the state of the pod could not be obtained, typically due to an error in communicating with the host of the pod

Pod Tags

Tag Name	Description
hostname	Name of the host. Instead of using this tag we recommend using the `@host` alias
Pod	Name of container group (Pod)
Node	Kubernetes node name
Namespace	Kubernetes namespace

Container

Container Metrics

Namespace	Description
kubernetes.container.requested.cpu.cores	The limit on cpu cores to be used by a container.
kubernetes.container.limits.memory.bytes	The limit on memory to be used by a container in bytes.
kubernetes.container.requested.cpu.cores	The number of requested cpu cores by a container.
kubernetes.container.requested.memory.bytes	The number of requested memory bytes by a container.
kubernetes.container.status.ready	specifies whether the container has passed its readiness probe
kubernetes.container.status.restarts	number of times the container has been restarted
kubernetes.container.status.running	value 1 if container is running else value 0
kubernetes.container.status.terminated	value 1 if container is terminated else value 0
kubernetes.container.status.waiting	value 1 if container is waiting else value 0

Container Tags

Tag Name	Description
hostname	Name of the host. Instead of using this tag we recommend using the `@host` alias
pod	Name of container group (Pod)
node	Kubernetes node name
namespace	Kubernetes namespace
container	Kubernetes container name

Node

Node Metrics

Namespace	Description
kubernetes.node.spec.unschedulable	Whether a node can schedule new pods.
kubernetes.node.status.allocatable.cpu.cores	The CPU resources of a node that are available for scheduling.
kubernetes.node.status.allocatable.memory.bytes	The memory resources of a node that are available for scheduling.
kubernetes.node.status.allocatable.pods	The pod resources of a node that are available for scheduling.
kubernetes.node.status.capacity.cpu.cores	The total CPU resources of the node.
kubernetes.node.status.capacity.memory.bytes	The total memory resources of the node.
kubernetes.node.status.capacity.pods	The total pod resources of the node.
kubernetes.node.status.outofdisk	True if there is insufficient free space on the node for adding new pods, otherwise False

Node Tags

Tag Name	Description
hostname	Name of the host. Instead of using this tag we recommend using the `@host` alias
Node	Kubernetes node name

Deployment

Deployment Metrics

Namespace	Description
kubernetes.deployment.metadata.generation	The desired generation sequence number for deployment. If a deployment succeeds should be the same as the observed generation.
kubernetes.deployment.status.observedgeneration	The generation sequence number after deployment.
kubernetes.deployment.status.targetedreplicas	Total number of non-terminated pods targeted by this deployment (their labels match the selector).
kubernetes.deployment.status.unavailablereplicas	Total number of unavailable pods targeted by this deployment.
kubernetes.deployment.status.availablereplicas	Total number of available pods (ready for at least minReadySeconds) targeted by this deployment.
kubernetes.deployment.spec.desiredreplicas	Number of desired pods.
kubernetes.deployment.status.deploynotfinished	If desired and observed generation are not the same, then either an ongoing deploy or a failed deploy
kubernetes.deployment.spec.paused	An optional boolean field for pausing and resuming a Deployment.
kubernetes.deployment.status.updatedreplicas	Total number of non-terminated pods targeted by this deployment that have the desired template spec

Deployment Tags

Tag Name	Description
hostname	Name of the host. Instead of using this tag we recommend using the `@host` alias
namespace	Kubernetes namespace
deployment	Kubernetes deployment name

Navigation Notice: When the APM Integrated Experience is enabled, AppOptics shares a common navigation and enhanced feature set with other integrated experience products. How you navigate AppOptics and access its features may vary from these instructions.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.

Search SolarWinds Support