Here is an overview of how you can operate Promitor.

Health

Resource Discovery Support Badge Scraper Support Badge

Promitor provides a basic health endpoint that indicates the state of the scraper.

Health endpoints can be useful for monitoring the scraper, running sanity tests after deployments or use it for sending liveness / health probes.

Consuming the health endpoint

You can check the status with a simple GET:

❯ curl -i -X GET "http://<uri>/api/v1/health"

Health is currently indicated via the HTTP response status:

The endpoint provides more details on integration with following dependencies:

Discovery

Resource Discovery Support Badge Scraper Support Badge

Promitor Resource Discovery provides a way to discover the resources for our Scraper agent to dynamically scrape resources.

Next to that, it provides a variety of system metrics that provides information concerning your Azure landscape.

Subscription

Our promitor_azure_landscape_subscription_info metrics provides an overview of all the Azure subscriptions that Promitor is able to discover in your Azure Landscape.

It provides the following tags with more information:

# HELP promitor_azure_landscape_subscription_info Provides information concerning the Azure subscriptions in the landscape that Promitor has access to.
# TYPE promitor_azure_landscape_subscription_info gauge
promitor_azure_landscape_subscription_info{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_name="Windows Azure MSDN - Visual Studio Ultimate",subscription_id="0329dd2a-59dc-4493-aa54-cb01cb027dc2",state="Enabled",spending_limit="On",quota_id="MSDN_2014-09-01",authorization="RoleBased"} 1 1628779903451
promitor_azure_landscape_subscription_info{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_name="Visual Studio Enterprise",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",state="Enabled",spending_limit="Off",quota_id="Sponsored_2016-01-01",authorization="RoleBased"} 1 1628779903451

Resource Groups

Our promitor_azure_landscape_resource_group_info metrics provides an overview of all the Azure resource groups that Promitor is able to discover in your Azure Landscape across all your subscriptions.

It provides the following tags with more information:

# HELP promitor_azure_landscape_resource_group_info Provides information concerning the Azure resource groups in the landscape that Promitor has access to.
# TYPE promitor_azure_landscape_resource_group_info gauge
promitor_azure_landscape_resource_group_info{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",resource_group_name="NetworkWatcherRG",region="westeurope",provisioning_state="Succeeded",managed_by="n/a"} 1 1628779903423
promitor_azure_landscape_resource_group_info{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",resource_group_name="promitor-testing-resource-discovery-eu",region="westeurope",provisioning_state="Succeeded",managed_by="n/a"} 1 1628779903423
promitor_azure_landscape_resource_group_info{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",resource_group_name="MC_keda-demos_keda-demos_westeurope",region="westeurope",provisioning_state="Succeeded",managed_by="/subscriptions/0f9d7fea-99e8-4768-8672-06a28514f77e/resourcegroups/keda-demos/providers/Microsoft.ContainerService/managedClusters/keda-demos"} 1 1628779903423

Performance

Resource Discovery Support Badge Scraper Support Badge

You can easily monitor the performance of Promitor through the following Prometheus metrics:

# HELP promitor_runtime_http_request_duration_seconds duration histogram of http responses labeled with: status_code, method, path
# TYPE promitor_runtime_http_request_duration_seconds histogram
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.005"} 30
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.01"} 31
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.025"} 31
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.05"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.075"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.1"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.25"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="0.75"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="1"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="2.5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="7.5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="10"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/scrape",le="+Inf"} 34
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/scrape"} 0.27116070000000003
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/scrape"} 34

Scraping Prometheus endpoint

Every Promitor agent supports exposing Prometheus metrics:

Scraping Azure Monitor

You can easily monitor the performance of Promitor Scraper agent integrating with Azure Monitor through the following Prometheus metrics:

# HELP promitor_scrape_error Provides an indication that the scraping of the resource has failed
# TYPE promitor_scrape_error gauge
promitor_scrape_error{metric_name="promitor_demo_app_insights_dependency_duration_200_OK",resource_group="docker-hub-metrics",resource_name="Microsoft.Insights/Components/docker-hub-metrics",resource_type="Generic",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27"} 1 1623691623231
# HELP promitor_scrape_success Provides an indication that the scraping of the resource was successful
# TYPE promitor_scrape_success gauge
promitor_scrape_success{metric_name="promitor_demo_automation_update_deployment_machine_runs",resource_group="promitor-sources",resource_name="promitor-sandbox",resource_type="AutomationAccount",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27"} 1 1623691626335

System

Resource Discovery Support Badge Scraper Support Badge

Promitor provides a basic system endpoint that provides information about itself such as its version.

Consuming the System endpoint

You can check the status with a simple GET:

❯ curl -i -X GET "http://<uri>/api/v1/system"

Exploring our REST APIs

We provide API documentation to make it easier for you to consume our REST APIs them:

Integrations

Azure Resource Manager API - Consumption & Throttling

Resource Discovery Support Badge Scraper Support Badge

Promitor exposes runtime metrics to provide insights on the API consumption of Azure Resource Manager API:

# HELP promitor_ratelimit_arm Indication how many calls are still available before Azure Resource Manager (ARM) is going to throttle us.
# TYPE promitor_ratelimit_arm gauge
promitor_ratelimit_arm{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0329dd2a-59dc-4493-aa54-cb01cb027dc2",app_id="ceb249a3-44ce-4c90-8863-6776336f5b7e"} 11995 1629719527020
promitor_ratelimit_arm{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",app_id="ceb249a3-44ce-4c90-8863-6776336f5b7e"} 11989 1629719532626
# HELP promitor_ratelimit_arm_throttled Indication concerning Azure Resource Manager are being throttled. (1 = yes, 0 = no).
# TYPE promitor_ratelimit_arm_throttled gauge
promitor_ratelimit_arm_throttled{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0329dd2a-59dc-4493-aa54-cb01cb027dc2",app_id="ceb249a3-44ce-4c90-8863-6776336f5b7e"} 0 1629719527020
promitor_ratelimit_arm_throttled{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",subscription_id="0f9d7fea-99e8-4768-8672-06a28514f77e",app_id="ceb249a3-44ce-4c90-8863-6776336f5b7e"} 0 1629719532626

You can read more about the Azure Resource Manager limitations on docs.microsoft.com.

Azure Resource Graph

Resource Discovery Support Badge Scraper Support Badge

Promitor exposes runtime metrics to provide insights on the API consumption of Azure Resource Graph:

# HELP promitor_ratelimit_resource_graph_remaining Indication how many calls are still available before Azure Resource Graph is going to throttle us.
# TYPE promitor_ratelimit_resource_graph_remaining gauge
promitor_ratelimit_resource_graph_remaining{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",cloud="Global",auth_mode="ServicePrincipal",app_id="67882a00-21d3-4ee7-b32a-430ea0768cd3"} 9 1629719863738
# HELP promitor_ratelimit_resource_graph_throttled Indication concerning Azure Resource Graph are being throttled. (1 = yes, 0 = no).
# TYPE promitor_ratelimit_resource_graph_throttled gauge
promitor_ratelimit_resource_graph_throttled{tenant_id="c8819874-9e56-4e3f-b1a8-1c0325138f27",cloud="Global",auth_mode="ServicePrincipal",app_id="67882a00-21d3-4ee7-b32a-430ea0768cd3"} 0 1629719863738

You can read more about the Azure Resource Graph throttling on docs.microsoft.com.

Azure Monitor

Promitor interacts with Azure Monitor API to scrape all the required metrics.

During troubleshooting it can be interesting to gain insights on what the API returns, for which you can opt-in.

You can opt-in for it by configuring the runtime telemetry.

← back