Azure monitor logs

To alert on VM‐level errors and push them into PagerDuty, you’ll use Azure Monitor’s Log Alerts (formerly “Log Search Alerts”) against a Log Analytics workspace, plus an Action Group wired to PagerDuty via a webhook. In broad strokes the steps are:

1. Ingest your VM logs into a Log Analytics workspace.
2. Create an Action Group that calls PagerDuty’s Events API v2 webhook.
3. Define a log‐query alert rule that fires when “Error” events appear and attaches the PagerDuty Action Group.

---

1. Send VM logs to Log Analytics

Ensure your VM is reporting its event logs (Windows) or syslog (Linux) to a Log Analytics workspace:

• In the Azure portal, go to your VM → Diagnostic settings → Add diagnostic setting.
• Check Send to Log Analytics and select (or create) your workspace.
• Under Log categories include Syslog (for Linux) or Windows Event Logs (Application, System, Security).
• Save.

Once configured, you’ll see data flow into the workspace’s Logs blade .

---

2. Create a PagerDuty–backed Action Group

An Action Group bundles one or more notification “receivers.” To route Azure alerts into PagerDuty:

1. In the Azure portal search for Monitor → Alerts → Action groups → + Create.
2. Fill in name/resource group/short name.
3. Under Actions click Add action:

• Action Type: Webhook
• Name: e.g. PagerDutyWebhook
• URI: your PagerDuty Events API v2 endpoint, e.g. https://events.pagerduty.com/v2/enqueue?integration_key=<YOUR_KEY> (learn.microsoft.com)
  1. Review + Create.

Tip: You obtain that Integration Key by adding a “Custom Events API v2” integration to a PagerDuty Service in the PagerDuty UI (pagerduty.com).

---

3. Define a Log-query alert rule

You can do this in the portal or via CLI/ARM/Terraform. We’ll show both:

---

A. Portal approach

1. In Monitor → Alerts → + New alert rule.
2. Scope: click Select resource, choose your Log Analytics workspace.
3. Condition: click Add condition → under Log, pick Custom log search.
4. In the query editor enter a Kusto query that filters for errors. For example:

   union
    (Syslog | where SeverityLevel >= 3),
    (Event | where EventLevelName == "Error")
   | where TimeGenerated > ago(5m)
   

5. Set Alert logic: “When the number of results is Greater than 0.”
6. Action group: click Select action groups, pick the one you created with the PagerDuty webhook.
7. Define Alert rule details (severity, name, description) and Review + Create (learn.microsoft.com).

---

B. Azure CLI example

# 1. Variables
rg="MyResourceGroup"
ws="MyLogAnalyticsWorkspace"
ag="PagerDutyAG"
key="<YOUR_PAGERDUTY_KEY>"

# 2. Create action group with webhook
az monitor action-group create \
  --resource-group $rg \
  --name $ag \
  --short-name pd \
  --action webhook PagerDutyWebhook \
    https://events.pagerduty.com/v2/enqueue?integration_key=$key \
  --verbose

# 3. Get workspace resource ID
ws_id=$(az monitor log-analytics workspace show \
  --resource-group $rg \
  --workspace-name $ws \
  --query id --output tsv)

# 4. Create scheduled (log) query alert
az monitor scheduled-query create \
  --resource-group $rg \
  --name "VMErrorAlert" \
  --scopes $ws_id \
  --description "Alert on VM error logs" \
  --severity 2 \
  --evaluation-frequency 5m \
  --window-size 5m \
  --condition "count 'ErrorQuery' > 0" \
  --condition-query 'union (Syslog | where SeverityLevel>=3),(Event | where EventLevelName=="Error") | where TimeGenerated>ago(5m)' \
  --action "/subscriptions/<subId>/resourceGroups/$rg/providers/microsoft.insights/actionGroups/$ag" \
  --verbose

In this CLI snippet, --condition references a placeholder name ('ErrorQuery') defined by --condition-query, so Azure knows when the alert should fire (learn.microsoft.com).

---

Validation & Next Steps

• Test by generating an error on the VM (e.g. an event in Windows Event Viewer or a logged logger "something error" on Linux). Within 5 minutes you should see an incident in PagerDuty.
• Tune your Kusto query to filter only the logs you care about (by computer name, log source, text pattern, etc.).
• Manage via IaC: if you use Terraform, you can use azurerm_monitor_action_group and azurerm_monitor_scheduled_query_rules_alert.

That’s it – you now have VM‐level log monitoring in Azure, with real‐time error incidents routed straight into your PagerDuty on-call rotation.