Poll an orchestrator and its devices using the same polling engine
SolarWinds recommends that you assign the orchestrator and all its devices to be polled using the same polling engine. Review the reasons:
- Fewer API requests thanks to the shared cache
- A single job to poll data about devices on a specific orchestrator, regardless of the orchestrator infrastructure size
How does polling work?
Orchestrator data is polled using the vendor API. The polled data belong to different areas, such as SD-WAN, interfaces, topology, wireless, and so on. For each area, a separate worker process is created. The worker process is isolated and its task is to execute polling jobs to retrieve data associated with the area.
For example, the SD-WAN worker process polls data related to SD-WAN from orchestrator nodes. Common API requests are cached, and the cache can be re-used for polling other orchestrator. The cache is bound to the worker process. If you poll a part of orchestrator nodes using one polling engine and another part using a different polling engine, each polling engine has its own worker process to poll the area-specific data and as a result, the worker processes cannot share the cache.
When all orchestrator nodes are polled by the same polling engine, requests can be executed once, cached, and then be taken from the cache.
When orchestrator nodes are polled by different polling engines, polling needs to be executed repeatedly for each polling engine. Repeated polling can affect API polling and lead to exceeding the supported rates.
Orchestrator-centric polling - a single polling job for polling devices on one orchestrator
A polling job is scheduled for each orchestrator device. A worker process executes each polling job.
If you are monitoring 1,000 orchestrator nodes, polling node details requires scheduling 1,000 separate polling jobs.
When the limit of 500 polling jobs per worker process at the same time is exceeded, an additional worker process is created.
Creating additional worker processes leads to duplicating some API calls, because the cache is bound to a specific worker process and is not accessible for the other worker processes. If you are monitoring many orchestrator nodes, some API calls might be sent repeatedly due to the 500 polling jobs per worker process at the same time limitation.
To work around the limitation, the orchestrator-centric polling is used for the following technologies: Topology, Node Details, Node Status, and Response Time.
Orchestrator-centric approach schedules a single job to poll data about all devices belonging to a specific orchestrator. The polled data is then processed separately for each monitored device. This way, all data is polled in scope of a single job, regardless the orchestrator infrastructure size, and resending requests is avoided. However, this approach requires that the orchestrator and monitored orchestrator nodes are polled by the same polling engine because the polling job is scheduled per orchestrator node and only has access to the devices on the same polling engine. Orchestrator nodes monitored by another polling engine are ignored.