Rules
This content is not available in your language yet.
Rules are inserted at first start, then belong to you: DumbMonit never rewrites a rule you changed, it only recreates missing ones. Built-in rules can be edited and disabled, not deleted.
Built-in rules
Section titled “Built-in rules”Severities are shown with the UI word; the API value is in parentheses. A rule whose metric no monitor produces has no series and never fires, so every rule below is shipped enabled on every instance.
Every monitor
Section titled “Every monitor”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Monitor unreachable | No measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d])) | > 180 s | 1 min | Warning (critical) | 6 h |
| High CPU | CPU load sustained above 90%, averaged per monitor across SNMP, Proxmox and agent sources; clears below 85 %. | > 90 % | 10 min | Advisory (warning), escalates after 1 h | 6 h |
| Unusual CPU | CPU load noticeably different from the usual at this time and day of the week (seasonal baseline). | score > 3.5 | 15 min | Info (info) | 6 h |
| Disk almost full | Filesystem 90% full or more, per mount point (SNMP, Proxmox storages and root filesystems); clears below 88 %. | ≥ 90 % | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| Filesystem almost full | At this rate, the filesystem will be full within four days (predict_linear over 6 h; rising filesystems only). | ≥ 100 % | 30 min | Advisory (warning) | 6 h |
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| UPS on battery | The UPS is powering the load from battery (dumbmonit_ups_output_source == 5). | > 0 | 30 s | Warning (critical) | 15 min |
| UPS battery low | The UPS battery is low or depleted (dumbmonit_ups_battery_status, 3 = low, 4 = depleted). | ≥ 3 | 1 min | Warning (critical) | 30 min |
Network ports (SNMP)
Section titled “Network ports (SNMP)”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Port accumulating errors | A network port logged more than fifty errors in the last hour: usually a failing cable, a dirty fibre or a duplex mismatch (increase_prometheus(dumbmonit_if_errors_in[1h]) + increase_prometheus(dumbmonit_if_errors_out[1h]); nothing fires during the first hour of a new monitor, so errors counted before it was added are not reported as new). | > 50 | 15 min | Advisory (warning) | 24 h |
| Port flapping | A network port went down and up again more than twice in thirty minutes (changes_prometheus(dumbmonit_if_oper_status[30m])). A port disabled on purpose has no series and never fires. | > 4 | 5 min | Advisory (warning) | 1 h |
Server hardware (Redfish)
Section titled “Server hardware (Redfish)”Health series are 0 (OK), 1 (Warning) or 2 (Critical). An empty slot (Absent) or a disabled one has no series, so a server delivered with one power supply out of two never fires.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Server fan failed | The management controller reports a fan as failed (dumbmonit_redfish_fan_health). | ≥ 2 | 2 min | Warning (critical) | 6 h |
| Server temperature above critical | A temperature sensor is above the critical threshold the controller itself declares (dumbmonit_redfish_temperature_celsius >= dumbmonit_redfish_temperature_upper_critical_celsius). | > 0 | 5 min | Warning (critical) | 6 h |
| Power supply redundancy lost | The power supplies are no longer redundant: one more failure and the server goes down (dumbmonit_redfish_power_redundancy_health). | ≥ 1 | 5 min | Advisory (warning), escalates after 1 h | 6 h |
| Power supply failed | The controller reports a power supply as failed (dumbmonit_redfish_psu_health). | ≥ 2 | 2 min | Warning (critical) | 6 h |
| Drive failure predicted | A drive reports a predicted failure: replace it while it still works (dumbmonit_redfish_drive_failure_predicted). | > 0 | 10 min | Advisory (warning) | 24 h |
| Server health critical | The controller reports the system’s own health as Critical (dumbmonit_redfish_system_health). | ≥ 2 | 2 min | Warning (critical) | 6 h |
Uptime monitors and heartbeats
Section titled “Uptime monitors and heartbeats”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Service down | The service has not responded correctly for three minutes (dumbmonit_probe_success{probe!="push"} == bool 0). | > 0 | 3 min | Warning (critical) | 30 min |
| Heartbeat missed | The job has not called in within its expected interval plus grace period, or reported a failure itself (dumbmonit_probe_success{probe="push"} == bool 0; the delay is set per monitor, see Heartbeat). | > 0 | 2 min | Advisory (warning) | 6 h |
| Service flapping | The service changed state more than six times in thirty minutes (changes(dumbmonit_probe_success{probe!="push"}[30m])). | > 6 | 5 min | Advisory (warning) | 1 h |
| Slow service | The service takes more than three seconds to respond; failed checks are left out (dumbmonit_probe_duration_seconds and dumbmonit_probe_success == 1). | > 3 s | 10 min | Advisory (warning) | 6 h |
| Certificate expiring soon | The certificate expires in less than fourteen days. | < 14 d | 1 h | Advisory (warning) | 24 h |
| Certificate expired | The certificate has expired. | < 0 d | 5 min | Warning (critical) | 24 h |
Docker containers
Section titled “Docker containers”Reported by the agent; see Agent. The alert names the container.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Container stopped | The container has been stopped for two minutes (dumbmonit_container_up == bool 0). A container Docker would not restart is left out (dumbmonit_container_stop_expected = 1): a one-shot docker run that finished, a container never started, or one stopped by hand without a restart policy. A crash (any other exit code), an out-of-memory kill, or an always / unless-stopped container still counts. | > 0 | 2 min | Advisory (warning) | 1 h |
| Container unhealthy | The container’s health check has been failing for three minutes (dumbmonit_container_health == 2; 0 none, 1 healthy, 3 starting). A stopped container is left out: Docker keeps its last verdict, which no longer means anything. | > 0 | 3 min | Advisory (warning) | 1 h |
| Container restarting | The container restarted three times or more in fifteen minutes (increase(dumbmonit_container_restart_count[15m])): it is rarely stopped long enough for the previous rule. | ≥ 3 | 1 min | Advisory (warning) | 1 h |
| Container update available | A newer image is available in the registry for this container (dumbmonit_container_update_available == 1). | > 0 | 1 h | Info (info) | 24 h |
Plakar backups
Section titled “Plakar backups”Reported by the agent, per kloset and per source. None of these series exists while Plakar is not detected, so neither rule can fire on a machine without it.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Plakar backup too old | No new Plakar snapshot for this source for more than two days (dumbmonit_backup_last_success_seconds). | > 2 d | 1 h | Advisory (warning) | 24 h |
| Plakar backup failed | The latest snapshot has errors (dumbmonit_backup_last_status{source!="*"} < 1). | < 1 | 10 min | Warning (critical) | 24 h |
| Plakar kloset unreadable | The agent cannot open the kloset: storage unreachable, passphrase or plugin missing; the agent’s log says which, once (dumbmonit_backup_last_status{source="*"} < 1). | < 1 | 30 min | Advisory (warning) | 24 h |
restic and Borg backups
Section titled “restic and Borg backups”Reported by the agent for each repository declared under restic_repos or
borg_repos in agent.yaml; see Agent.
Without a declared repository there is no series, so neither rule can fire.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| restic/Borg backup too old | No new restic or Borg snapshot in this repository for more than 48 hours (dumbmonit_agent_backup_repo_last_snapshot_age_seconds). | > 48 h | 1 h | Advisory (warning) | 24 h |
| restic/Borg repository unreachable | The agent cannot read the repository: unreachable, password refused or binary missing; the agent’s log says which (dumbmonit_agent_backup_repo_reachable < 1). | < 1 | 30 min | Advisory (warning) | 24 h |
WireGuard tunnels
Section titled “WireGuard tunnels”Reported by the agent; see Agent.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| WireGuard peer silent | A peer with a persistent keepalive has not completed a handshake for more than 15 minutes: the tunnel is down (dumbmonit_agent_wireguard_peer_handshake_age_seconds and on (target, interface, peer) dumbmonit_agent_wireguard_peer_keepalive_seconds > 0). A peer that never connected counts its silence from the moment the agent first saw it. Peers without keepalive (a phone, a laptop) idle for days and are left out; change the threshold to change N, or drop the and on … part to watch every peer. | > 15 min | 5 min | Advisory (warning) | 6 h |
IPsec tunnels
Section titled “IPsec tunnels”Reported by the agent; see Agent. The series exists only for a CHILD_SA declared in the strongSwan configuration, so the rule fires on a tunnel that never came up, not only on one that dropped.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| IPsec tunnel down | A configured IPsec tunnel has no installed session (dumbmonit_agent_ipsec_tunnel_up < 1). | < 1 | 10 min | Advisory (warning) | 6 h |
Internet speed test
Section titled “Internet speed test”Reported by the agent; see Agent.
The test is off by default and each rule stays silent until the matching
threshold is set in agent.yaml: nobody can guess what “slow” means for your line.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Internet download rate low | The last test measured a download rate below speedtest_min_download_mbps (dumbmonit_agent_speedtest_download_min_bytes_per_second - dumbmonit_agent_speedtest_download_bytes_per_second > 0). | > 0 | 10 min | Advisory (warning) | 24 h |
| Internet upload rate low | Same, for speedtest_min_upload_mbps. | > 0 | 10 min | Advisory (warning) | 24 h |
| Internet latency high | The last test measured a latency above speedtest_max_latency_ms. | > 0 | 10 min | Advisory (warning) | 24 h |
Windows event log
Section titled “Windows event log”Raised by the event log rules of a Windows agent machine; see Windows event log alerts. No event log rule exists by default, so these rules stay silent until you add one. One alert per machine and severity, quoting the latest event; it speaks again at most every 10 minutes while events keep coming, and resolves 15 minutes after the last one.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Windows event (critical) | An event matched a rule of severity Warning (dumbmonit_agent_eventlog_matches_critical: events in the last 15 minutes). | > 0 | 30 s | Warning (critical) | none |
| Windows event (warning) | Same, for rules of severity Advisory (dumbmonit_agent_eventlog_matches_warning). | > 0 | 30 s | Advisory (warning) | none |
| Windows event (info) | Same, for rules of severity Info (dumbmonit_agent_eventlog_matches_info). | > 0 | 30 s | Info (info) | none |
Agent disks, ZFS and sensors
Section titled “Agent disks, ZFS and sensors”Reported by the agent; see Agent.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Disk SMART failing | The disk’s own SMART self-assessment reports it as failing (dumbmonit_agent_disk_smart_ok == bool 0). Replace it and check the backups. | > 0 | 10 min | Warning (critical) | 24 h |
| ZFS pool degraded | A ZFS pool is no longer healthy: a device is missing, faulted or removed (dumbmonit_agent_zfs_pool_health: 0 online, 1 degraded, 2 anything else). | > 0 | 5 min | Warning (critical) | 6 h |
| ZFS scrub found errors | The last ZFS scrub found errors on this pool, or files are permanently lost (dumbmonit_agent_zfs_pool_scrub_errors or dumbmonit_agent_zfs_pool_data_errors). The pool can stay ONLINE. | > 0 | 10 min | Warning (critical) | 24 h |
| Temperature above critical | A sensor is above the critical temperature its own hardware declares (dumbmonit_agent_sensor_temperature_celsius >= dumbmonit_agent_sensor_temperature_critical_celsius); no fixed value. | > 0 | 5 min | Warning (critical) | 6 h |
Proxmox VE
Section titled “Proxmox VE”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Backup too old | No successful Proxmox VE backup for more than seven days (dumbmonit_proxmox_backup_last_age_seconds). | > 7 d | 1 h | Advisory (warning) | 24 h |
| VM or container stopped | The guest was running within the last two hours and has been stopped for five minutes (1 - dumbmonit_proxmox_guest_running, and’d with max_over_time(…[2h]) == 1). | > 0 | 5 min | Advisory (warning) | 6 h |
| VM or container CPU high | A guest has used more than 90 % of its allocated cores for fifteen minutes (dumbmonit_proxmox_guest_cpu_percent). The notification names the guest, nextcloud (202). | > 90 % | 15 min | Advisory (warning) | 6 h |
| VM or container memory high | A guest has used more than 95 % of its memory for ten minutes (dumbmonit_proxmox_guest_memory_percent). | > 95 % | 10 min | Advisory (warning) | 6 h |
| VM or container disk almost full | The root disk of a guest is more than 90 % full (dumbmonit_proxmox_guest_disk_used_percent): containers always, VMs only when the QEMU guest agent reports usage. | > 90 % | 15 min | Advisory (warning) | 24 h |
| Guest locked | A VM or container has been locked for six hours (dumbmonit_proxmox_guest_locked, label lock): an interrupted backup or migration left the lock behind and no operation can run on the guest, not even the next backup. | > 0 | 6 h | Advisory (warning) | 24 h |
| Old snapshot | The oldest snapshot of a guest is more than thirty days old (dumbmonit_proxmox_guest_snapshot_oldest_age_seconds). | > 30 d | 1 h | Info (info) | 7 d |
| HA resource in error | A Proxmox HA resource is in error or fenced (dumbmonit_proxmox_ha_resource_error). | > 0 | 2 min | Warning (critical) | 1 h |
| HA manager not reporting | The HA local resource manager of a node has stopped writing its state (dumbmonit_proxmox_ha_lrm_stale): that node’s HA guests will be neither restarted nor relocated. | > 0 | 5 min | Warning (critical) | 1 h |
| Cluster lost quorum | The Proxmox cluster is not quorate (1 - dumbmonit_proxmox_cluster_quorate): guests can no longer be started. | > 0 | 1 min | Warning (critical) | 30 min |
| Proxmox node offline | A cluster node is reported offline (1 - dumbmonit_proxmox_node_up). | > 0 | 2 min | Warning (critical) | 30 min |
| Proxmox service down | A core Proxmox daemon is not running on a node (dumbmonit_proxmox_node_core_services_down: pve-cluster, pvedaemon, pveproxy, pvestatd, pve-firewall, corosync, watchdog-mux, each counted only where the node enables it). | > 0 | 5 min | Warning (critical) | 1 h |
| Node network interface down | An interface set to start at boot is not up on a node (dumbmonit_proxmox_node_interface_offline). An interface brought up on demand is never reported. | > 0 | 5 min | Advisory (warning) | 6 h |
| Proxmox storage almost full | A Proxmox storage is more than 85% full (dumbmonit_proxmox_storage_used_percent); “Disk almost full” takes over at 90%. | > 85 % | 15 min | Advisory (warning) | 24 h |
| Thin pool almost full | An LVM thin pool is more than 90 % full: every guest stored on it goes read-only when it fills (dumbmonit_proxmox_node_thinpool_used_percent); clears below 88 %. | > 90 % | 15 min | Warning (critical) | 6 h |
| Thin pool metadata almost full | The metadata volume of an LVM thin pool is more than 80 % full (dumbmonit_proxmox_node_thinpool_metadata_used_percent). It is far smaller than the data volume and often saturates first, with the same consequence. | > 80 % | 15 min | Warning (critical) | 6 h |
| Proxmox disk wearing out | An SSD or NVMe of a node has used more than 90 % of its rated life (dumbmonit_proxmox_node_disk_wearout_percent). The notification reads pve1 · /dev/nvme0n1. | > 90 % | 1 h | Advisory (warning) | 7 d |
| Proxmox disk SMART failure | A disk of a node reports SMART health FAILED (dumbmonit_proxmox_node_disk_smart_failed). | > 0 | 5 min | Warning (critical) | 24 h |
| Proxmox ZFS pool degraded | A ZFS pool of a node is not ONLINE (dumbmonit_proxmox_node_zfs_pool_degraded). The notification reads pve2 · tank. | > 0 | 5 min | Warning (critical) | 6 h |
| Backup job failed | The last vzdump job on a node, or a scheduled backup job, failed (1 - dumbmonit_proxmox_backup_job_last_ok). | > 0 | 10 min | Warning (critical) | 24 h |
| Backup job excludes a disk | A scheduled backup job skips a disk of one of the guests it covers (dumbmonit_proxmox_backup_job_guest_excluded_volumes): the job succeeds every night with that data missing. CD-ROM drives and cloudinit images are excluded by design and never counted. | > 0 | 1 h | Advisory (warning) | 7 d |
| Replication failed | A replication job reports an error or a non-zero fail count (dumbmonit_proxmox_replication_job_error): the standby copy is stale. | > 0 | 10 min | Warning (critical) | 6 h |
| Ceph health error | Ceph reports HEALTH_ERR (dumbmonit_proxmox_ceph_health, 0 OK, 1 WARN, 2 ERR). | ≥ 2 | 2 min | Warning (critical) | 30 min |
| Ceph health warning | Ceph reports HEALTH_WARN for more than fifteen minutes (dumbmonit_proxmox_ceph_health == 1). | > 0 | 15 min | Advisory (warning) | 6 h |
| Ceph OSD down | A Ceph OSD has been down for five minutes (1 - dumbmonit_proxmox_ceph_osd_up). The notification names the OSD and its host. | > 0 | 5 min | Warning (critical) | 1 h |
| Ceph OSD nearly full | A single Ceph OSD is more than 85 % full (dumbmonit_proxmox_ceph_osd_used_percent): writes stop when it reaches its limit, whatever the cluster average says; clears below 82 %. | > 85 % | 15 min | Advisory (warning) | 24 h |
| Ceph pool almost full | A Ceph pool is more than 85 % full (dumbmonit_proxmox_ceph_pool_used_percent); clears below 82 %. | > 85 % | 15 min | Advisory (warning) | 24 h |
| Ceph noout flag left on | The Ceph noout flag has been set for two hours (dumbmonit_proxmox_ceph_flag{flag="noout"}): rebalancing is disabled, and the cluster reads healthy while its redundancy shrinks. | > 0 | 2 h | Info (info) | 24 h |
| Proxmox updates pending | More than twenty package updates are pending on a node (dumbmonit_proxmox_node_updates_pending). | > 20 | 1 h | Info (info) | 7 d |
| Proxmox security updates pending | At least one pending update comes from a security archive (dumbmonit_proxmox_node_updates_security_pending). | > 0 | 1 h | Advisory (warning) | 7 d |
| Proxmox packages changed | The installed version of a Proxmox package changed between two probes — someone ran apt upgrade (dumbmonit_proxmox_node_packages_changed, published for one hour with the detail in its changes label: pve1 · pve-manager 8.2.4→8.2.7). Resolves by itself an hour later. | > 0 | 1 min | Info (info) | 24 h |
| Node certificate expiring | A Proxmox node certificate expires in less than fourteen days (dumbmonit_proxmox_node_certificate_expiry_days). | < 14 d | 1 h | Advisory (warning) | 24 h |
Proxmox Backup Server
Section titled “Proxmox Backup Server”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| PBS datastore almost full | PBS datastore more than 90% full. | > 90 % | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| PBS datastore filling up | At the current rate, PBS estimates the datastore full within seven days. | < 7 d | 1 h | Advisory (warning) | 24 h |
| PBS datastore not mounted | A removable datastore is not mounted (dumbmonit_pbs_datastore_removable_unmounted): nothing can be written to it. | > 0 | 6 h | Advisory (warning) | 24 h |
| PBS backup too old | No new snapshot for this backup group (one per machine) for more than two days (dumbmonit_pbs_backup_last_age_seconds). Edit the threshold for machines backed up weekly; the notification names the group and datastore. | > 2 d | 1 h | Advisory (warning) | 24 h |
| PBS backup verification failed | Verification of the latest snapshot for this machine failed (dumbmonit_pbs_backup_last_verified < 1). | < 1 | 30 min | Warning (critical) | 24 h |
| PBS task failed | At least one PBS task failed in the review window (dumbmonit_pbs_tasks_failed). | > 0 | 10 min | Advisory (warning) | 24 h |
| PBS garbage collection too old | No successful garbage collection on this datastore for more than eight days (dumbmonit_pbs_gc_last_success_age_seconds). | > 8 d | 1 h | Advisory (warning) | 24 h |
| PBS garbage collection failed | The last garbage collection on a datastore failed (dumbmonit_pbs_gc_last_run_ok < 1; from /gc on PBS 3.3+, from the latest GC task before). | < 1 | 10 min | Advisory (warning) | 24 h |
| PBS corrupt chunks found | The garbage collection found unreadable chunks on a datastore (dumbmonit_pbs_gc_bad_chunks): some backups can no longer be restored. | > 0 | 10 min | Warning (critical) | 24 h |
| PBS prune job failed | The last run of a PBS prune job failed: nothing is pruned and the datastore keeps filling up (dumbmonit_pbs_job_last_ok{kind="prune"} < 1). The notification names the job and datastore. | < 1 | 10 min | Advisory (warning) | 24 h |
| PBS verification job failed | The last run of a PBS verification job failed (dumbmonit_pbs_job_last_ok{kind="verify"} < 1). | < 1 | 10 min | Advisory (warning) | 24 h |
| PBS sync job failed | The last run of a PBS sync job failed (dumbmonit_pbs_sync_job_last_ok < 1). | < 1 | 10 min | Warning (critical) | 24 h |
| PBS tape backup failed | The last tape backup run failed (dumbmonit_pbs_tape_backup_job_last_ok < 1): the offline copy is not being made. | < 1 | 10 min | Warning (critical) | 24 h |
| PBS job never ran | A job is enabled and scheduled but has never run (dumbmonit_pbs_job_never_run). | > 0 | 2 d | Advisory (warning) | 7 d |
| PBS service down | A service Proxmox Backup Server needs is not running (dumbmonit_pbs_node_service_active{expected="1"} < 1); no backup can be taken in while it is down. | < 1 | 5 min | Warning (critical) | 6 h |
| PBS disk SMART failure | A disk of the backup server reports SMART health FAILED (dumbmonit_pbs_node_disk_smart_failed). The notification reads /dev/sdb. | > 0 | 5 min | Warning (critical) | 24 h |
| PBS SSD worn out | An SSD of the backup server has used more than 90 % of its rated endurance (dumbmonit_pbs_node_disk_wearout_percent). | > 90 % | 1 h | Advisory (warning) | 7 d |
| PBS ZFS pool degraded | A ZFS pool of the backup server is not ONLINE (dumbmonit_pbs_node_zfs_pool_degraded). | > 0 | 5 min | Warning (critical) | 6 h |
| PBS certificate expiring | The certificate served by the backup server expires in less than three weeks (dumbmonit_pbs_node_certificate_expires_seconds). Needs the certificates option. | < 21 d | 1 h | Advisory (warning) | 48 h |
| PBS restart pending after upgrade | The package was upgraded but the running daemon is still the old one (dumbmonit_pbs_node_running_version_stale). | > 0 | 6 h | Info (info) | 7 d |
| PBS updates pending | More than twenty package updates are pending on the backup server (dumbmonit_pbs_node_updates_pending). | > 20 | 1 h | Info (info) | 7 d |
Proxmox Datacenter Manager
Section titled “Proxmox Datacenter Manager”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Federated instance unreachable | The datacenter console cannot reach one of the Proxmox VE clusters or backup servers it federates (dumbmonit_pdm_remote_reachable < 1). The notification names the instance and its product. | < 1 | 10 min | Warning (critical) | 6 h |
| Federated instance behind | An instance runs an older version than another instance of the same product in the same estate (dumbmonit_pdm_remote_version_behind). Compared between instances, never against what Proxmox publishes. | > 0 | 1 h | Info (info) | 7 d |
| Task failed on a federated instance | At least one task failed on a federated instance in the review window (dumbmonit_pdm_remote_tasks_failed). | > 0 | 10 min | Advisory (warning) | 24 h |
| Datacenter console disk almost full | The root filesystem of the datacenter console is more than 90 % full (dumbmonit_pdm_node_rootfs_percent): full, the console stops recording what it collects. | > 90 % | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| Datacenter console certificate expiring | A certificate of the datacenter console expires in less than fourteen days (dumbmonit_pdm_node_certificate_expiry_days). | < 14 d | 1 h | Advisory (warning) | 24 h |
| Datacenter console updates pending | More than twenty package updates are pending on the datacenter console (dumbmonit_pdm_node_updates_pending). | > 20 | 1 h | Info (info) | 7 d |
Proxmox Mail Gateway
Section titled “Proxmox Mail Gateway”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Mail queue growing | The deferred mail queue has grown by more than a hundred messages in two hours (delta(dumbmonit_pmg_queue_messages{queue="deferred"}[2h])): the next hop is probably refusing mail. Growth rather than size, because a busy gateway always keeps a few dozen deferred. | > 100 | 15 min | Advisory (warning) | 6 h |
| Mail stuck in the queue | A message has been waiting in a queue for more than four hours (dumbmonit_pmg_queue_oldest_age_seconds). The age is a lower bound — qshape reports brackets — so the rule fires late rather than early. The notification names the queue. | > 4 h | 15 min | Warning (critical) | 6 h |
| Mail gateway service stopped | A service the gateway needs to accept and filter mail is stopped (dumbmonit_pmg_service_running for postfix, pmg-smtp-filter, pmgpolicy, pmgproxy, pmgdaemon). A unit that is not installed has no series and cannot fire. | < 1 | 5 min | Warning (critical) | 6 h |
| Virus signatures out of date | The ClamAV daily virus signatures are more than two days old (dumbmonit_pmg_signature_age_seconds{family="virus",database="daily"}): freshclam has stopped updating them and the gateway keeps filtering with last week’s list. | > 2 d | 1 h | Advisory (warning) | 24 h |
| Spam rules out of date | The SpamAssassin rules are more than eight days old (dumbmonit_pmg_signature_age_seconds{family="spam"}). A channel that has never been updated has no date and no series. | > 8 d | 1 h | Advisory (warning) | 7 d |
| Quarantine filling up | The spam quarantine has taken in more than a thousand messages in six hours (delta(dumbmonit_pmg_quarantine_messages{kind="spam"}[6h])): a campaign, or a rule that just put a legitimate domain in the bin. | > 1000 | 30 min | Advisory (warning) | 12 h |
| Mail gateway cluster degraded | A gateway of the cluster is no longer in sync with the others (dumbmonit_pmg_cluster_node_insync): it filters with an out-of-date rule database. A standalone gateway has no series. | < 1 | 15 min | Advisory (warning) | 6 h |
| Mail gateway certificate expiring | A certificate of the mail gateway expires in less than fourteen days (dumbmonit_pmg_certificate_expires_in_seconds). | < 14 d | 1 h | Advisory (warning) | 24 h |
| Mail gateway updates pending | More than twenty package updates are pending on the mail gateway (dumbmonit_pmg_node_updates_pending). | > 20 | 1 h | Info (info) | 7 d |
MDaemon and SecurityGateway
Section titled “MDaemon and SecurityGateway”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| MDaemon mail service down | A watched mail service of the MDaemon server does not answer, or greets to refuse service (dumbmonit_mdaemon_service_up). Only the services listed on the monitor have a series. The notification names the service and its port. | < 1 | 5 min | Warning (critical) | 6 h |
| MDaemon XML API not answering | The XML API has stopped answering while the mail services still do (dumbmonit_mdaemon_api_up): Remote Administration is probably stopped. Only a monitor with an account has the series. | < 1 | 15 min | Advisory (warning) | 12 h |
| MDaemon mail queue growing | The inbound, local or remote queue has grown by more than a hundred messages in two hours (delta(dumbmonit_mdaemon_queue_messages{queue=~"inbound|local|remote"}[2h])): MDaemon has stopped delivering from it. Read by the agent on the MDaemon server, so the alert is on the agent’s monitor. | > 100 | 15 min | Advisory (warning) | 6 h |
| MDaemon Bad queue not empty | Messages are waiting in the Bad queue (dumbmonit_mdaemon_queue_messages{queue="bad"}): MDaemon could neither deliver nor bounce them, and they stay there until an administrator releases or deletes them. | > 0 | 30 min | Advisory (warning) | 24 h |
| MDaemon Retry queue high | More than fifty messages wait in the Retry queue (dumbmonit_mdaemon_queue_messages{queue="retry"}): remote servers keep refusing mail temporarily. | > 50 | 1 h | Advisory (warning) | 6 h |
| Mail server Windows service stopped | A Windows service of MDaemon or SecurityGateway that starts automatically is not running (dumbmonit_agent_mail_service_up{start="auto"}). Read by the agent on the mail server, so the alert is on the agent’s monitor. Services set to manual start or disabled do not alert. | < 1 | 5 min | Warning (critical) | 6 h |
| Mail queue stuck | The oldest message of an inbound, local or remote queue has been waiting for more than two hours (dumbmonit_agent_mail_queue_oldest_seconds{queue=~"inbound|local|remote"}): the queue is frozen or delivery is blocked. Counted from the queue folders by the agent on the mail server. | > 7200 s | 15 min | Advisory (warning) | 6 h |
| MDaemon queue frozen | MDaemon reports one of its queues as frozen (dumbmonit_mdaemon_queue_frozen): it no longer processes it. Read from the performance counters by the agent on the mail server. | > 0 | 15 min | Advisory (warning) | 6 h |
| SecurityGateway service down | A watched service of the gateway does not answer, or greets to refuse service (dumbmonit_securitygateway_service_up). | < 1 | 5 min | Warning (critical) | 6 h |
| SecurityGateway API not answering | The REST API has stopped answering while the services still do (dumbmonit_securitygateway_api_up). Only a monitor with an API key has the series. | < 1 | 15 min | Advisory (warning) | 12 h |
| SecurityGateway delivery queue growing | The delivery queue has grown by more than a hundred messages in two hours (delta(dumbmonit_securitygateway_counter{counter=~".*(delivery_queue|queued_for_delivery).*"}[2h])): the mail server behind the gateway is probably refusing mail. The counter names come from the API as they are; if your version names the queue differently, adjust the matcher. | > 100 | 15 min | Advisory (warning) | 6 h |
OPNsense
Section titled “OPNsense”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Internet gateway down | dpinger no longer gets an answer from a gateway (dumbmonit_opnsense_gateway_up). On a multi-WAN firewall the traffic has already moved to another link, quietly. loss and delay are warnings, not failures, and do not fire this rule. | < 1 | 5 min | Warning (critical) | 6 h |
| Gateway losing packets | A gateway is losing more than a fifth of the packets sent to it (dumbmonit_opnsense_gateway_loss_percent): the link answers, badly. Clears below 10 %. | > 20 % | 10 min | Advisory (warning) | 6 h |
| Gateway slow | A gateway’s round-trip time has stayed above 300 ms (dumbmonit_opnsense_gateway_delay_seconds). Three hundred milliseconds leaves a mobile backup link alone. | > 300 ms | 15 min | Info (info) | 12 h |
| Firewall state table filling up | pf is tracking more than 80 % of the connections it is allowed (dumbmonit_opnsense_pf_states_used_percent). Past the limit the firewall drops new connections with no other symptom. | > 80 % | 10 min | Advisory (warning) | 6 h |
| Firewall network buffers exhausted | More than 90 % of the FreeBSD network buffers are in use (dumbmonit_opnsense_mbuf_used_percent). A firewall that runs out stops forwarding with the CPU still idle. | > 90 % | 10 min | Advisory (warning) | 6 h |
| VPN tunnel down | A configured VPN tunnel has had no session for ten minutes (dumbmonit_opnsense_vpn_tunnel_up). A plugin that is not installed, or a connection that is disabled, has no series and cannot fire. | < 1 | 10 min | Advisory (warning) | 6 h |
| Firewall resolver stopped | Unbound has stopped answering (dumbmonit_opnsense_unbound_running). The firewall still routes; the machines behind it no longer resolve. Turn this rule off if your firewall resolves with something else. | < 1 | 5 min | Advisory (warning) | 6 h |
| Firewall core service stopped | dpinger, which watches the gateways, or configd, which runs everything else, is stopped (dumbmonit_opnsense_service_running{service=~"dpinger|configd"}). Only those two, because a service switched off on purpose is not a failure. | < 1 | 5 min | Warning (critical) | 6 h |
| Firewall too hot | A sensor reads above 85 °C (dumbmonit_opnsense_temperature_celsius): the fanless box in the cupboard is about to start throttling. Clears at 78 °C. | > 85 °C | 10 min | Advisory (warning) | 6 h |
| Firewall left in CARP maintenance mode | CARP has been left in persistent maintenance mode (dumbmonit_opnsense_carp_maintenance_mode): this firewall handed its virtual addresses to its partner and will not take them back on its own. | > 0 | 1 h | Info (info) | 24 h |
| Firewall reboot pending | An update is installed but needs a reboot to take effect (dumbmonit_opnsense_firmware_reboot_required). | > 0 | 1 h | Info (info) | 7 d |
| Firewall updates pending | The firewall has packages waiting to be updated (dumbmonit_opnsense_firmware_updates_pending). It is the most exposed machine on the network. | > 0 | 1 h | Info (info) | 7 d |
TrueNAS
Section titled “TrueNAS”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| NAS pool degraded | ZFS no longer calls a pool healthy (dumbmonit_truenas_pool_healthy). A degraded pool still serves its data, which is exactly why nobody notices until the next disk fails. The monitor page names the disk. | < 1 | 5 min | Warning (critical) | 6 h |
| NAS disk errors | Disks of a pool have reported read, write or checksum errors since the last zpool clear (dumbmonit_truenas_pool_device_errors, one series per kind). | > 0 | 15 min | Advisory (warning) | 24 h |
| NAS pool almost full | A pool is more than 85 % full (dumbmonit_truenas_pool_used_percent). ZFS slows down well before it is full. Clears at 80 %. | > 85 % | 30 min | Advisory (warning), escalates after 7 d | 24 h |
| NAS scrub found errors | The last scrub of a pool found damaged data (dumbmonit_truenas_pool_last_scrub_errors). | > 0 | 5 min | Warning (critical) | 24 h |
| NAS scrub overdue | A pool has not completed a scrub in more than 45 days (dumbmonit_truenas_pool_last_scrub_age_seconds). Absent after a resilver, which erases the record of the last scrub. | > 45 d | 1 h | Info (info) | 7 d |
| NAS disk failed its SMART test | A SMART self-test in a disk’s log failed (dumbmonit_truenas_disk_smart_failed). A disk never tested has no series. | > 0 | 5 min | Warning (critical) | 24 h |
| NAS disk too hot | A disk is above 55 °C (dumbmonit_truenas_disk_temperature_celsius, read from TrueNAS’s cache). Clears at 50 °C. | > 55 °C | 15 min | Advisory (warning) | 6 h |
| NAS dataset near its quota | A dataset has used more than 90 % of its quota (dumbmonit_truenas_dataset_quota_used_percent): at 100 % its writes fail while the pool still has room. No quota, no series. | > 90 % | 30 min | Advisory (warning) | 24 h |
| NAS replication failed | An enabled replication task is in error (dumbmonit_truenas_replication_error): the copy on the other side is getting older. | > 0 | 10 min | Advisory (warning) | 12 h |
| NAS snapshot task failed | An enabled periodic snapshot task is in error (dumbmonit_truenas_snapshot_task_error). | > 0 | 10 min | Advisory (warning) | 12 h |
| NAS snapshots stale | An enabled periodic snapshot task has not run for more than eight days (dumbmonit_truenas_snapshot_task_last_run_age_seconds): a task that stops running does not fail, it goes quiet. | > 8 d | 1 h | Advisory (warning) | 24 h |
| TrueNAS alert raised | TrueNAS itself has raised an alert of level ERROR or above (dumbmonit_truenas_alerts). Its own sentence is on the monitor page. | > 0 | 10 min | Advisory (warning) | 12 h |
| NAS service stopped | A service set to start with the NAS is stopped (dumbmonit_truenas_service_running). A service switched off on purpose has no series. | < 1 | 10 min | Advisory (warning) | 6 h |
Synology DSM
Section titled “Synology DSM”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Synology disk SMART warning | DSM no longer reports this disk’s SMART status as normal (dumbmonit_synology_disk_smart_status: 1 attention, 2 critical, unknown words count as 1). | > 0 | 15 min | Advisory (warning) | 24 h |
| Synology disk failed | DSM reports the disk as crashed or failed (dumbmonit_synology_disk_status). | ≥ 2 | 5 min | Warning (critical) | 6 h |
| Synology disk bad sectors | The disk’s bad-sector count exceeds the threshold set in DSM (dumbmonit_synology_disk_bad_sector_exceeded). | > 0 | 15 min | Warning (critical) | 24 h |
| Synology disk bad sectors growing | New unreadable sectors appeared in the last 24 hours (delta(dumbmonit_synology_disk_unc_count[24h])). | > 0 | 15 min | Advisory (warning) | 24 h |
| Synology SSD wearing out | An SSD has less than 10 % of its rated life left (dumbmonit_synology_disk_remaining_life_percent); clears above 12 %. | < 10 % | 1 h | Advisory (warning) | 7 d |
| Synology volume degraded | A volume is degraded or crashed: its redundancy is gone (dumbmonit_synology_volume_status). | ≥ 2 | 5 min | Warning (critical) | 6 h |
| Synology storage pool degraded | A storage pool or SSD cache is degraded or crashed: a disk failed and the redundancy is gone or rebuilding (dumbmonit_synology_pool_status or dumbmonit_synology_ssd_cache_status). | ≥ 2 | 5 min | Warning (critical) | 6 h |
| Synology volume almost full | A volume is 90 % full or more, not counting the space reserved by thick-provisioned iSCSI LUNs (dumbmonit_synology_volume_effective_used_percent, equal to the raw usage without LUNs); clears below 88 %. A volume filled by a thick LUN is full by design and does not alert. | ≥ 90 % | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| Synology thin LUN over-allocation | The thin LUNs of a volume were promised more space than the volume still has free (dumbmonit_synology_volume_thin_overcommit_percent: promised and unwritten space as a percentage of the free space); clears below 95 %. | ≥ 100 % | 1 h | Advisory (warning) | 24 h |
| Synology thin LUN almost full | A thin LUN has 95 % or more of its size written (dumbmonit_synology_lun_used_percent). A thick LUN is opaque from the NAS: the rules of the monitor that uses it (datastore, filesystem) alert instead. | ≥ 95 % | 30 min | Advisory (warning) | 24 h |
| Synology temperature high | A disk is above 55 °C, or DSM raised its own temperature warning (dumbmonit_synology_disk_temperature_celsius, dumbmonit_synology_temperature_warning); clears below 52 °C. | > 55 °C | 15 min | Advisory (warning) | 6 h |
| Synology memory high | Memory usage of the NAS above 95 % (dumbmonit_synology_memory_usage_percent); clears below 90 %. | > 95 % | 15 min | Advisory (warning) | 6 h |
| Active Backup task failed | The last run of an Active Backup for Business task failed, or backed up only part of its devices (dumbmonit_abb_task_last_status == bool 0). The notification names the task. | > 0 | 10 min | Warning (critical) | 24 h |
| Active Backup too old | No successful Active Backup for Business run for this task for more than two days (dumbmonit_abb_task_last_success_seconds). The series does not exist for a task that has never succeeded; “Active Backup task failed” speaks then. | > 2 d | 1 h | Advisory (warning) | 24 h |
| Active Backup task disabled | The task has no schedule, or its continuous backup is paused (dumbmonit_abb_task_enabled == bool 0): it will not back up anything until someone runs it. | > 0 | 1 h | Info (info) | 7 d |
| Active Backup device overdue | A device has gone longer without a successful Active Backup for Business run than its own rhythm allows, off-days excluded (dumbmonit_abb_device_overdue, see Synology). | > 0 | 30 min | Advisory (warning) | 24 h |
| Active Backup device failing | The last two or more attempts of a device failed (dumbmonit_abb_device_consecutive_failures); a cancelled run counts as neither. | ≥ 2 | 10 min | Warning (critical) | 24 h |
Log and metrics servers
Section titled “Log and metrics servers”VictoriaMetrics, VictoriaLogs, Loki and Graylog (monitors). Counters go through increase_prometheus, so a server added with a history of refused data does not fire on its first measurement.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Metrics or log storage read-only | Free disk space fell below -storage.minFreeDiskSpaceBytes and the storage refuses every write (dumbmonit_victoriametrics_read_only or dumbmonit_victorialogs_read_only). | ≥ 1 | 2 min | Warning (critical) | 1 h |
| Metrics or log storage almost read-only | Less than 10% of the disk left before read-only (dumbmonit_victoriametrics_disk_headroom_percent or dumbmonit_victorialogs_disk_headroom_percent); clears above 12 %. | < 10 % | 30 min | Advisory (warning) | 24 h |
| Metrics or log server unhealthy | /health does not answer OK (dumbmonit_victoriametrics_healthy or dumbmonit_victorialogs_healthy). | < 1 | 3 min | Warning (critical) | 6 h |
| Metrics or log data refused | Samples or log lines refused in the last hour: timestamps outside retention, labels too long, series limit (increase_prometheus(dumbmonit_victoriametrics_rows_rejected_total[1h]) or increase_prometheus(dumbmonit_victorialogs_rows_rejected_total[1h])). | > 0 | 15 min | Advisory (warning) | 24 h |
| VictoriaMetrics slow inserts | More than 5% of new samples took the slow path over 15 minutes: not enough memory for the active series ((100 * increase_prometheus(dumbmonit_victoriametrics_slow_inserts_total[15m]) / increase_prometheus(dumbmonit_victoriametrics_rows_added_total[15m])) and on (target) dumbmonit_victoriametrics_uptime_seconds > 900); clears below 3 %. Only judged once the server has run for 15 minutes, so a restart (cold cache, counters back to zero inside the window) does not leave it pending. | > 5 % | 30 min | Advisory (warning) | 24 h |
| Loki not ready | /ready does not answer ready (dumbmonit_loki_ready). | < 1 | 5 min | Warning (critical) | 6 h |
| Loki refusing log lines | Lines refused in the last hour: too old, rate or stream limit, line too long (increase_prometheus(dumbmonit_loki_discarded_lines_total[1h])). | > 0 | 15 min | Advisory (warning) | 24 h |
| Loki cannot flush to storage | Chunks failed to reach storage in the last 30 minutes; they are lost if Loki restarts (increase_prometheus(dumbmonit_loki_flush_failures_total[30m])). | > 0 | 10 min | Warning (critical) | 1 h |
| Loki write-ahead log disk full | WAL writes lost to a full disk (increase_prometheus(dumbmonit_loki_wal_disk_full_failures_total[30m])). | > 0 | 5 min | Warning (critical) | 1 h |
| Loki request errors | More than five requests answered with a server error in 15 minutes (increase_prometheus(dumbmonit_loki_request_errors_total[15m])). | > 5 | 15 min | Advisory (warning) | 6 h |
| Graylog not processing | Message processing paused on the node (dumbmonit_graylog_processing). | < 1 | 5 min | Warning (critical) | 6 h |
| Graylog search cluster down | OpenSearch or Elasticsearch red, or unreachable from Graylog (dumbmonit_graylog_indexer_status: 2 red, 3 unreachable). | ≥ 2 | 5 min | Warning (critical) | 1 h |
| Graylog search cluster yellow | Replica shards unassigned for half an hour (dumbmonit_graylog_indexer_status == 1). | > 0 | 30 min | Advisory (warning) | 24 h |
| Graylog journal filling | The journal is more than half full: the output does not keep up (dumbmonit_graylog_journal_used_percent); clears below 40 %. | > 50 % | 10 min | Advisory (warning), escalates after 1 h | 6 h |
| Graylog input failed | An input failed to start (dumbmonit_graylog_inputs_failed). | > 0 | 5 min | Advisory (warning) | 24 h |
| Graylog indexing failures | Writes to the search cluster or message processing failed in the last hour (increase_prometheus(dumbmonit_graylog_output_failures_total[1h]) + increase_prometheus(dumbmonit_graylog_processing_failures_total[1h])). | > 0 | 15 min | Advisory (warning) | 24 h |
Self-hosted applications
Section titled “Self-hosted applications”Nextcloud, Immich, Paperless-ngx, Jellyfin and Plex (monitors). They rarely fall over outright: the page opens while syncing, thumbnails, imports or library scans have stopped. These rules watch those states; an application that does not answer at all is Monitor unreachable.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Nextcloud stuck in maintenance | Maintenance mode is on (dumbmonit_nextcloud_maintenance): nobody can log in or sync. | > 0 | 30 min | Advisory (warning), escalates after 3 h | 6 h |
| Nextcloud database upgrade pending | The code was updated and Nextcloud waits for occ upgrade (dumbmonit_nextcloud_needs_db_upgrade). | > 0 | 10 min | Warning (critical) | 1 h |
| Nextcloud update available | Nextcloud’s update check found a newer release (dumbmonit_nextcloud_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Nextcloud OPcache full | PHP’s OPcache is out of room and pages get slow (dumbmonit_nextcloud_opcache_full). | > 0 | 30 min | Advisory (warning) | 24 h |
| Nextcloud free space low | Less than 5 GiB free for the data directory (dumbmonit_nextcloud_free_space_bytes / 1073741824); clears above 6 GiB. | < 5 GiB | 30 min | Advisory (warning) | 24 h |
| Immich jobs stalled | Jobs wait in a queue that is not paused and runs nothing (dumbmonit_immich_queue_waiting > 0 and on (target, queue) dumbmonit_immich_queue_active == 0 unless on (target, queue) dumbmonit_immich_queue_paused == 1). The notification names the queue. | > 0 | 1 h | Advisory (warning) | 6 h |
| Immich queue paused | A job queue has been paused for a day (dumbmonit_immich_queue_paused). | > 0 | 24 h | Info (info) | 7 d |
| Immich storage almost full | The disk holding the library is more than 90 % full (dumbmonit_immich_storage_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Immich update available | Immich’s version check knows a newer release (dumbmonit_immich_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Immich jobs failing | A queue other than machine learning holds 25 or more failed jobs (dumbmonit_immich_queue_failed). The notification names the queue. | ≥ 25 | 30 min | Advisory (warning) | 24 h |
| Immich machine learning failing | The smart search, faces, duplicates and text recognition queues hold 10 or more failed jobs: the machine learning container is probably down (dumbmonit_immich_queue_failed). | ≥ 10 | 30 min | Advisory (warning) | 24 h |
| Immich in maintenance mode | Immich has stayed in maintenance mode (dumbmonit_immich_maintenance_mode). | > 0 | 30 min | Advisory (warning) | 6 h |
| Immich user quota almost full | An account with a quota used more than 90 % of it (dumbmonit_immich_user_quota_used_percent); clears below 85 %. | > 90 % | 1 h | Advisory (warning) | 24 h |
| Immich database backup stale | The newest database backup is more than three days old (dumbmonit_immich_db_backup_last_age_seconds); needs databaseBackup.list. | > 3 d | 1 h | Advisory (warning) | 24 h |
| Paperless task queue down | Redis or Celery in error (max by (target) (dumbmonit_paperless_component_status{component=~"redis|celery"}): 2 is an error): nothing new is imported. | ≥ 2 | 5 min | Warning (critical) | 1 h |
| Paperless index or sanity check error | The search index, the classifier or the sanity check (missing or altered files) reports an error (dumbmonit_paperless_component_status{component=~"index|classifier|sanity_check"}). | ≥ 2 | 30 min | Advisory (warning) | 24 h |
| Paperless migrations pending | Database migrations left to apply (dumbmonit_paperless_unapplied_migrations). | > 0 | 30 min | Advisory (warning) | 24 h |
| Paperless task failed | A task this user can see failed within the failed task window and was not dismissed (dumbmonit_paperless_tasks_failed_recent). | > 0 | 5 min | Advisory (warning) | 24 h |
| Paperless storage almost full | The disk holding the documents is more than 90 % full (dumbmonit_paperless_storage_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Paperless update available | Paperless-ngx’s version check knows a newer release (dumbmonit_paperless_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Jellyfin scheduled task failed | A scheduled task (a library scan, a cleanup) failed or was aborted on its last run (dumbmonit_jellyfin_scheduled_tasks_failed). | > 0 | 10 min | Advisory (warning) | 24 h |
| Jellyfin plugin broken | A plugin failed to load or does not support this version (dumbmonit_jellyfin_plugins_broken). | > 0 | 15 min | Advisory (warning) | 24 h |
| Jellyfin restart pending | Jellyfin has waited a day for a restart after a plugin install or update (dumbmonit_jellyfin_pending_restart). | > 0 | 24 h | Info (info) | 7 d |
| Jellyfin library scan stale | The scheduled library scan has not finished for three days (dumbmonit_jellyfin_library_scan_age_seconds, only while the scan has a periodic trigger). | > 3 d | 30 min | Advisory (warning) | 24 h |
| Jellyfin storage almost full | A disk used by Jellyfin (library, data, cache, transcodes) is more than 90 % full (dumbmonit_jellyfin_storage_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Jellyfin heavy CPU transcoding | Three or more streams transcoded without hardware acceleration (dumbmonit_jellyfin_transcodes_software). | ≥ 3 | 15 min | Info (info) | 12 h |
| Jellyfin failed sign-ins | Ten or more failed sign-ins in the last hour (dumbmonit_jellyfin_failed_logins_recent). | ≥ 10 | 5 min | Advisory (warning) | 6 h |
| Plex update available | Plex’s update check found a newer release (dumbmonit_plex_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Plex transcoder too slow | The slowest video transcode being played runs below real time (dumbmonit_plex_transcode_speed_min): playback will buffer. | < 0.95 | 10 min | Advisory (warning) | 6 h |
| Plex transcode error | A transcode session reports an error (dumbmonit_plex_transcode_errors). | > 0 | 5 min | Advisory (warning) | 12 h |
| Plex remote access failed | Plex reports its remote access (port mapping) failed (dumbmonit_plex_remote_access_failed). | > 0 | 30 min | Advisory (warning) | 24 h |
| Plex signed out of plex.tv | A claimed server lost its plex.tv sign-in (dumbmonit_plex_signin_ok). | < 1 | 1 h | Advisory (warning) | 24 h |
| Plex database backup off | The “Backup database” maintenance task is turned off (dumbmonit_plex_database_backup_enabled); ignore it if another backup covers the database. | < 1 | 24 h | Info (info) | 30 d |
Self-hosted forges
Section titled “Self-hosted forges”GitLab (self-managed) and Forgejo/Gitea (monitors). An instance that answers nothing at all is Monitor unreachable; these rules watch the parts that keep serving pages while quietly falling behind — Sidekiq, CI/CD runners, a scheduled task that stopped running.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| GitLab runners all offline | Every CI/CD runner GitLab knows is offline, and at least one is registered (dumbmonit_gitlab_runners_online and on (target) dumbmonit_gitlab_runners_total > 0). | < 1 | 10 min | Advisory (warning) | 6 h |
| GitLab Sidekiq backlog high | A Sidekiq queue has more than 500 jobs waiting (max by (target) (dumbmonit_gitlab_sidekiq_queue_backlog)): notifications, webhooks and imports fall behind. | > 500 | 15 min | Advisory (warning) | 6 h |
| GitLab migrations pending | Database migrations from an upgrade are left to apply (dumbmonit_gitlab_migrations_pending). | > 0 | 1 h | Info (info) | 24 h |
| GitLab license expiring | The Enterprise Edition licence expires in less than 30 days (dumbmonit_gitlab_license_expiry_seconds). | < 30 d | 1 h | Advisory (warning) | 24 h |
| Forgejo/Gitea runners all offline | Every Actions runner is offline, and at least one is registered (dumbmonit_forgejo_runners_online and on (target) dumbmonit_forgejo_runners_total > 0). | < 1 | 10 min | Advisory (warning) | 6 h |
| Forgejo/Gitea scheduled task overdue | A scheduled task (repository check, cleanup, update checker) did not run at its own interval (dumbmonit_forgejo_cron_task_overdue). | > 0 | 30 min | Advisory (warning) | 24 h |
Media, identity and mail applications
Section titled “Media, identity and mail applications”Sonarr, Radarr, Lidarr, Prowlarr, qBittorrent, UniFi Protect, Frigate, Keycloak, authentik, Vaultwarden and mailcow (monitors). These keep answering while the service they provide is broken: a download client down, a camera dropped, a database gone behind a login page. An application that does not answer at all is Monitor unreachable.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Sonarr health error | Sonarr reports an error on its System → Status page: a failing download client or indexer, a missing root folder (dumbmonit_sonarr_health_errors). | > 0 | 15 min | Advisory (warning) | 6 h |
| Sonarr queue has errors | An item of the download queue has an error (failed download, import blocked) for half an hour (dumbmonit_sonarr_queue_has_errors). | > 0 | 30 min | Advisory (warning) | 6 h |
| Sonarr disk almost full | A disk Sonarr reports is more than 90 % full (dumbmonit_sonarr_disk_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Sonarr update available | A newer Sonarr release exists (dumbmonit_sonarr_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Radarr health error | Radarr reports an error on its System → Status page: a failing download client or indexer, a missing root folder (dumbmonit_radarr_health_errors). | > 0 | 15 min | Advisory (warning) | 6 h |
| Radarr queue has errors | An item of the download queue has an error (failed download, import blocked) for half an hour (dumbmonit_radarr_queue_has_errors). | > 0 | 30 min | Advisory (warning) | 6 h |
| Radarr disk almost full | A disk Radarr reports is more than 90 % full (dumbmonit_radarr_disk_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Radarr update available | A newer Radarr release exists (dumbmonit_radarr_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Lidarr health error | Lidarr reports an error on its System → Status page: a failing download client or indexer, a missing root folder (dumbmonit_lidarr_health_errors). | > 0 | 15 min | Advisory (warning) | 6 h |
| Lidarr queue has errors | An item of the download queue has an error (failed download, import blocked) for half an hour (dumbmonit_lidarr_queue_has_errors). | > 0 | 30 min | Advisory (warning) | 6 h |
| Lidarr disk almost full | A disk Lidarr reports is more than 90 % full (dumbmonit_lidarr_disk_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Lidarr update available | A newer Lidarr release exists (dumbmonit_lidarr_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Prowlarr health error | Prowlarr reports an error on its System → Status page: a failing download client or indexer, a missing root folder (dumbmonit_prowlarr_health_errors). | > 0 | 15 min | Advisory (warning) | 6 h |
| Prowlarr disk almost full | A disk Prowlarr reports is more than 90 % full (dumbmonit_prowlarr_disk_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Prowlarr update available | A newer Prowlarr release exists (dumbmonit_prowlarr_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Prowlarr indexers disabled | Prowlarr disabled indexers after repeated failures (dumbmonit_prowlarr_indexers_disabled): Sonarr, Radarr and Lidarr stop finding releases through them. | > 0 | 1 h | Advisory (warning) | 24 h |
| qBittorrent torrents in error | Torrents are in error or have lost their files (dumbmonit_qbittorrent_torrents_errored). | > 0 | 15 min | Advisory (warning) | 24 h |
| qBittorrent disconnected | qBittorrent is firewalled or disconnected (dumbmonit_qbittorrent_connected == bool 0): nothing is downloaded or seeded. | > 0 | 15 min | Advisory (warning) | 6 h |
| qBittorrent free space low | Less than 5 GiB free in the download folder (dumbmonit_qbittorrent_free_space_bytes / 1073741824); clears above 6 GiB. | < 5 GiB | 30 min | Advisory (warning) | 24 h |
| Protect camera offline | A UniFi Protect camera is disconnected from its NVR (dumbmonit_unifiprotect_camera_online == bool 0). The notification names the camera. | > 0 | 5 min | Advisory (warning) | 6 h |
| Protect camera not recording | A camera is connected and set to record but does not (dumbmonit_unifiprotect_cameras_not_recording). | > 0 | 15 min | Advisory (warning) | 6 h |
| Protect NVR disk unhealthy | The NVR reports a failing disk (dumbmonit_unifiprotect_disks_unhealthy). | > 0 | 5 min | Warning (critical) | 1 h |
| Protect recording disabled | Recording is switched off on the NVR (dumbmonit_unifiprotect_recording_disabled). | > 0 | 10 min | Advisory (warning) | 6 h |
| Frigate camera offline | A camera delivers no frames to Frigate (dumbmonit_frigate_camera_online == bool 0). The notification names the camera. | > 0 | 5 min | Advisory (warning) | 6 h |
| Frigate detector stuck | A detector has been on the same inference for more than 30 seconds (dumbmonit_frigate_detector_busy_seconds). | > 30 s | 2 min | Advisory (warning) | 1 h |
| Frigate storage almost full | The disk holding the recordings is more than 90 % full (dumbmonit_frigate_storage_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| Frigate update available | A newer Frigate release exists (dumbmonit_frigate_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Keycloak not ready | Keycloak answers but its readiness check is DOWN, usually the database (dumbmonit_keycloak_ready == bool 0): nobody can log in. | > 0 | 5 min | Warning (critical) | 1 h |
| Keycloak realm disabled | A realm is disabled (dumbmonit_keycloak_realms_disabled): its users cannot log in. | > 0 | 1 h | Info (info) | 7 d |
| authentik not ready | authentik answers but its readiness check fails (dumbmonit_authentik_ready == bool 0): database or cache down. | > 0 | 5 min | Warning (critical) | 1 h |
| authentik has no worker | No worker reports (dumbmonit_authentik_workers == bool 0): synchronisation, e-mails and clean-up stop. | > 0 | 10 min | Warning (critical) | 1 h |
| authentik task failed | A system task failed on its last run (dumbmonit_authentik_tasks_failed). The notification names the task. | > 0 | 1 h | Advisory (warning) | 24 h |
| authentik outpost disconnected | An outpost that serves providers has not reported for five minutes (dumbmonit_authentik_outposts_disconnected). | > 0 | 10 min | Advisory (warning) | 6 h |
| authentik update available | A newer authentik release exists (dumbmonit_authentik_update_available). | > 0 | 1 h | Info (info) | 7 d |
| Vaultwarden sign-ups open | Anyone who can reach the vault can create an account (dumbmonit_vaultwarden_signups_open). | > 0 | 1 h | Info (info) | 7 d |
| mailcow container down | A mailcow container is not running (dumbmonit_mailcow_container_up == bool 0). The notification names the container. | > 0 | 5 min | Warning (critical) | 1 h |
| mailcow mail queue stuck | The oldest message in Postfix’s queue has waited more than four hours (dumbmonit_mailcow_queue_oldest_seconds / 3600). | > 4 h | 15 min | Advisory (warning) | 6 h |
| mailcow mail volume almost full | The volume holding the mailboxes is more than 90 % full (dumbmonit_mailcow_vmail_used_percent); clears below 88 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| mailcow mailbox near quota | A mailbox is at 90 % of its quota or more (dumbmonit_mailcow_mailboxes_near_quota). | > 0 | 1 h | Info (info) | 24 h |
Client devices
Section titled “Client devices”One rule, shared by every integration that tracks client devices connecting and backing up — Immich’s phones, tablets and desktops (monitors), Proxmox Backup Server’s backed-up machines, one per backup group (monitors), and Veeam’s protected computers and VMs, one per object with a restore point (monitors), today; more to follow. Each integration computes the gap to its own target’s staleness window (device_stale_days, 3 days by default) and publishes it as dumbmonit_client_device_stale_seconds, labelled device, type, os, user, kind and signal (connection or backup); this one rule then fires for any of them, naming the monitor in the notification. Synology Active Backup for Business tracks its own monitors separately, through a rhythm-aware mechanism (synology_abb_device_*) rather than this generic family — see Synology.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Client device stale | A device has not connected or not backed up within the staleness window configured for this target (dumbmonit_client_device_stale_seconds). | > 0 | 1 h | Advisory (warning) | 24 h |
Pi-hole
Section titled “Pi-hole”Pi-hole v6, read through its REST API.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Pi-hole blocking disabled | Blocking is off, paused or failed (dumbmonit_pihole_blocking_enabled). The hold lets a five-minute pause from the web interface go by. | < 1 | 15 min | Advisory (warning) | 6 h |
| Pi-hole blocklists stale | The blocklists (gravity) have not been rebuilt for more than eight days (dumbmonit_pihole_gravity_age_seconds): the weekly update fails. | > 8 d | 1 h | Advisory (warning) | 24 h |
| Pi-hole update available | A newer version is available (dumbmonit_pihole_updates_available): the image in Docker, core, web interface or FTL otherwise. Pi-hole checks once a day. | > 0 | 1 h | Info (info) | 7 d |
| Pi-hole diagnosis messages | Pi-hole lists messages on its diagnosis page (dumbmonit_pihole_messages): a blocklist that failed to download, a rate-limited client, an overloaded host. | > 0 | 15 min | Info (info) | 24 h |
| Pi-hole SERVFAIL replies | More than 5% of the DNS replies of the last 24 hours are SERVFAIL (100 * dumbmonit_pihole_replies_servfail_24h / dumbmonit_pihole_queries_24h), once at least 200 queries were answered: an upstream resolver is failing. | > 5 % | 30 min | Advisory (warning) | 6 h |
| Pi-hole upstream resolver slow | An upstream resolver answers in more than a second on average (dumbmonit_pihole_upstream_response_seconds, one series per resolver that was asked). | > 1 s | 30 min | Advisory (warning) | 6 h |
| Pi-hole blocklist download failing | At least one enabled blocklist failed to download on the last gravity update (dumbmonit_pihole_lists_download_failed): it is stale or empty. | > 0 | 1 h | Advisory (warning) | 24 h |
| Pi-hole receives no queries | No DNS query in the last 24 hours (dumbmonit_pihole_queries_24h): no client uses it any more, or FTL is stuck. | < 1 | 1 h | Info (info) | 24 h |
AdGuard Home
Section titled “AdGuard Home”The DNS filter (monitor). Query and block figures are sums over AdGuard Home’s own statistics window, not counters: no rule is built on them.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| AdGuard Home protection off | Protection disabled, or paused and not back yet (dumbmonit_adguard_protection_enabled). | < 1 | 15 min | Advisory (warning) | 6 h |
| AdGuard Home DNS not running | The web interface answers but the DNS server is stopped (dumbmonit_adguard_running). | < 1 | 5 min | Warning (critical) | 1 h |
| AdGuard Home upstream failing | An upstream DNS server fails AdGuard Home’s own test, or stays silent 4 seconds (dumbmonit_adguard_upstream_up, one series per upstream). | < 1 | 10 min | Advisory (warning) | 6 h |
| AdGuard Home filter lists stale | The oldest update of an enabled filter list is more than three days old (dumbmonit_adguard_filter_oldest_update_age_seconds). | > 3 days | 1 h | Advisory (warning) | 24 h |
| AdGuard Home update available | A newer version is announced by AdGuard Home’s own version check (dumbmonit_adguard_update_available); never fires when that check is disabled, as in the official Docker image. | > 0 | 1 h | Info (info) | 7 days |
| AdGuard Home slow to answer | The average time to process a DNS query is above half a second (dumbmonit_adguard_avg_processing_seconds): the host is overloaded or an upstream is slow. | > 0.5 s | 30 min | Advisory (warning) | 6 h |
| AdGuard Home upstream slow | An upstream answers in more than a second on average over the statistics window (dumbmonit_adguard_upstream_avg_seconds, one series per upstream). | > 1 s | 30 min | Advisory (warning) | 6 h |
| AdGuard Home certificate invalid | The certificate or private key of the encrypted DNS does not load or does not match (dumbmonit_adguard_tls_valid); only when encryption is on. | < 1 | 10 min | Advisory (warning) | 12 h |
| AdGuard Home certificate expiring | The certificate of the encrypted DNS expires in less than 14 days (dumbmonit_adguard_tls_expiry_seconds). | < 14 d | 1 h | Advisory (warning) | 24 h |
| AdGuard Home DHCP range almost full | More than 90% of the DHCP address range is leased (dumbmonit_adguard_dhcp_pool_used_percent); only when the DHCP server is on. | > 90 % | 30 min | Advisory (warning) | 12 h |
| AdGuard Home receives no queries | No query over the whole statistics window, when statistics are on and cover at least a day (dumbmonit_adguard_dns_queries). | < 1 | 1 h | Info (info) | 24 h |
UPS behind NUT
Section titled “UPS behind NUT”UPS read from a NUT server. Every series carries the ups
label, so an alert names the UPS when one server publishes several. The status
flags are 0 or 1 at every read, so a rule clears by itself once mains power is
back. UPS read over SNMP are covered by the UPS rules above.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| UPS on battery (NUT) | The UPS is powering the load from battery (dumbmonit_nut_ups_on_battery, the OB flag of ups.status). | > 0 | 30 s | Warning (critical) | 15 min |
| UPS battery low (NUT) | The UPS reports a low battery, the LB flag (dumbmonit_nut_ups_low_battery): the machines it powers are about to shut down. | > 0 | 1 min | Warning (critical) | 30 min |
| UPS battery needs replacing (NUT) | The UPS asks for its battery to be replaced, the RB flag (dumbmonit_nut_ups_replace_battery). | > 0 | 1 h | Advisory (warning) | 7 d |
| UPS load high (NUT) | The load has stayed above 80 % (dumbmonit_nut_ups_load_percent); clears below 75 %. | > 80 % | 10 min | Advisory (warning) | 24 h |
| UPS runtime short (NUT) | The UPS estimates less than five minutes of runtime on battery (dumbmonit_nut_battery_runtime_seconds); clears above six. On mains power, a worn battery or a UPS too small for its load. | < 300 s | 5 min | Advisory (warning) | 24 h |
| UPS data stale (NUT) | The NUT server answers but has no fresh data from the UPS: cable, driver or UPS silent (dumbmonit_nut_ups_data_stale). An outage would go unseen meanwhile. | > 0 | 5 min | Advisory (warning), escalates after 1 h | 6 h |
MikroTik RouterOS
Section titled “MikroTik RouterOS”Series carry the interface or sensor label, so a notification names the port or the sensor (monitor). Counters go through increase_prometheus: a router added with years of errors behind it does not fire on its first measurement.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| MikroTik CPU high | The router’s CPU has stayed above 90 % (dumbmonit_mikrotik_cpu_load_percent): routing, the firewall or a script is saturating it. Clears below 80 %. | > 90 % | 15 min | Advisory (warning) | 6 h |
| MikroTik memory high | More than 90 % of the memory is in use (dumbmonit_mikrotik_memory_used_percent). Clears below 85 %. | > 90 % | 15 min | Advisory (warning) | 6 h |
| MikroTik storage almost full | The flash storage is more than 90 % full (dumbmonit_mikrotik_storage_used_percent): configuration changes and upgrades may fail. Clears below 85 %. | > 90 % | 30 min | Advisory (warning) | 24 h |
| MikroTik temperature high | A temperature sensor reads above 80 °C (dumbmonit_mikrotik_temperature_celsius). Clears at 75 °C. | > 80 °C | 10 min | Advisory (warning) | 6 h |
| MikroTik power supply or fan failed | A psu*-state or fan*-state sensor is not ok (dumbmonit_mikrotik_health_ok). A redundant power supply that died cuts nothing, until the second one does. | < 1 | 5 min | Warning (critical) | 1 h |
| MikroTik interface accumulating errors | More than fifty receive and transmit errors on an interface in the last hour (increase_prometheus(dumbmonit_mikrotik_interface_rx_errors_total[1h]) + increase_prometheus(dumbmonit_mikrotik_interface_tx_errors_total[1h])). | > 50 | 15 min | Advisory (warning) | 24 h |
| MikroTik link flapping | An interface lost its link more than twice in thirty minutes (increase_prometheus(dumbmonit_mikrotik_interface_link_downs_total[30m])). A port unplugged for good counts once and does not fire. | > 2 | 5 min | Advisory (warning) | 1 h |
| MikroTik RouterOS update available | A newer RouterOS is available on the router’s update channel (dumbmonit_mikrotik_update_available). Only known when the router checks for updates itself: see the optional step of the setup. | > 0 | 1 h | Info (info) | 7 d |
| MikroTik firmware upgrade pending | RouterBOOT is older than the firmware bundled with the installed RouterOS (dumbmonit_mikrotik_firmware_upgrade_pending); it is upgraded only on request, then at the next reboot. | > 0 | 1 h | Info (info) | 7 d |
UniFi Network
Section titled “UniFi Network”Monitor page. Device series carry the device’s name, model and type, so the notification says “Living room AP”. A device waiting for adoption has no device_up series and never counts as offline.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| UniFi device offline | A UniFi device adopted by the controller is offline, isolated or failed to adopt (dumbmonit_unifi_device_up). | < 1 | 5 min | Advisory (warning), escalates after 1 h | 6 h |
| UniFi Internet down | The UniFi gateway reports no Internet connection (dumbmonit_unifi_internet_up). Without a UniFi gateway the series does not exist and the rule stays silent. | < 1 | 3 min | Warning (critical) | 1 h |
| UniFi WAN link down | A WAN link of the UniFi gateway is down (dumbmonit_unifi_wan_link_up): with two links, redundancy is lost while nothing seems broken. A disabled port has no series. | < 1 | 5 min | Advisory (warning) | 6 h |
| UniFi gateway CPU high | The UniFi gateway’s processor has been above 90% for fifteen minutes (dumbmonit_unifi_device_cpu_percent{type="gateway"}); clears below 80 %. | > 90 % | 15 min | Advisory (warning) | 6 h |
| UniFi device waiting for adoption | A device appeared on the network and waits to be adopted in UniFi (dumbmonit_unifi_devices{state="pending"}). | > 0 | 30 min | Info (info) | 24 h |
| UniFi firmware update available | A firmware update is available for UniFi devices (dumbmonit_unifi_devices_upgradable). | > 0 | 1 h | Info (info) | 7 d |
| UniFi alarms | UniFi logged critical events or raised alarms in the last 24 hours (dumbmonit_unifi_alarms). Only read with a View Only account. | > 0 | 5 min | Advisory (warning) | 24 h |
Home Assistant
Section titled “Home Assistant”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Home Assistant in recovery mode | Home Assistant started in recovery mode: its configuration could not be loaded and automations do not run (dumbmonit_homeassistant_recovery_mode). | > 0 | 2 min | Warning (critical) | 1 h |
| Home Assistant not running | Home Assistant answers but has not finished starting for ten minutes (dumbmonit_homeassistant_running). | < 1 | 10 min | Advisory (warning) | 6 h |
| Home Assistant entities went unavailable | At least five more entities are unavailable than an hour ago: an integration, a hub or a radio stopped (sum by (target) (dumbmonit_homeassistant_entities_unavailable) - sum by (target) (dumbmonit_homeassistant_entities_unavailable offset 1h)). The rise counts, not the total, which a real installation never has at zero. | ≥ 5 | 10 min | Advisory (warning) | 6 h |
| Home Assistant battery low | A battery reported by Home Assistant is low (dumbmonit_homeassistant_battery_low, one series per battery below the device’s threshold, 20% by default; its value is the level). | ≥ 0 % | 30 min | Info (info) | 3 d |
| Home Assistant update available | Home Assistant has updates waiting: core, operating system, add-ons or device firmware (dumbmonit_homeassistant_updates_available). | > 0 | 1 h | Info (info) | 7 d |
| Home Assistant repair to address | Home Assistant has an open repair of error or critical severity (dumbmonit_homeassistant_repair{severity=~"error|critical"}). | > 0 | 15 min | Advisory (warning) | 24 h |
| Home Assistant safety sensor triggered | A smoke, gas, carbon monoxide, water leak or safety sensor is on (dumbmonit_homeassistant_safety_alert, one series per sensor). | > 0 | 1 min | Warning (critical) | 30 min |
| Home Assistant alarm triggered | An alarm panel is in the triggered state (dumbmonit_homeassistant_alarm_triggered). | > 0 | 30 s | Warning (critical) | 30 min |
| Home Assistant problem sensor on | A binary sensor of class problem or tamper has been on for thirty minutes (dumbmonit_homeassistant_problem). | > 0 | 30 min | Advisory (warning) | 12 h |
| Home Assistant automation unavailable | An automation is unavailable: its configuration no longer loads, so it never runs (dumbmonit_homeassistant_entities_unavailable{domain="automation"}). | > 0 | 30 min | Advisory (warning) | 24 h |
VMware vSphere
Section titled “VMware vSphere”Monitor page. Host series carry esxi, VM series vm, datastore series datastore, so the notification names the object.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| vSphere host not responding | A vSphere host is not responding to vCenter (dumbmonit_vsphere_host_connection_state: 2 not responding). A host disconnected on purpose (1) does not fire. | ≥ 2 | 3 min | Warning (critical) | 1 h |
| vSphere host health red | vSphere reports a host’s overall health as red (dumbmonit_vsphere_host_status == 2). Gray means unknown and does not fire. | > 0 | 5 min | Warning (critical) | 1 h |
| vSphere host left in maintenance | A vSphere host has been in maintenance mode for a day: its VMs were moved away and it may have been forgotten there (dumbmonit_vsphere_host_maintenance). | > 0 | 24 h | Info (info) | 24 h |
| vSphere datastore almost full | A vSphere datastore is 90% full or more (dumbmonit_vsphere_datastore_used_percent); clears below 85 %. | ≥ 90 % | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| vSphere datastore inaccessible | A vSphere datastore is inaccessible: the VMs stored on it cannot run (dumbmonit_vsphere_datastore_accessible). | < 1 | 5 min | Warning (critical) | 1 h |
| vSphere VM tools not running | VMware Tools are installed but not running in a powered-on VM: the guest may be hung (dumbmonit_vsphere_vm_tools_status == 2). VMs without Tools do not fire. | > 0 | 30 min | Info (info) | 24 h |
| vSphere red alarm | vSphere raised a red alarm that nobody acknowledged (dumbmonit_vsphere_alarm{status="red"}, one series per alarm, named after the alarm and the object). | > 0 | 5 min | Advisory (warning) | 6 h |
Databases, message broker and security engine
Section titled “Databases, message broker and security engine”Redis / Valkey, MongoDB, RabbitMQ and CrowdSec (Redis, MongoDB, RabbitMQ, CrowdSec). Counters go through increase_prometheus or changes_prometheus, so nothing fires on a monitor’s first measurement.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Redis memory near maxmemory | Memory used above 90 % of maxmemory (dumbmonit_redis_memory_used_percent); clears below 85 %. No series without maxmemory. | > 90 % | 10 min | Advisory (warning) | 6 h |
| Redis refusing connections | Client connections refused in the last 15 minutes: maxclients reached (increase_prometheus(dumbmonit_redis_rejected_connections_total[15m])). | > 0 | 5 min | Warning (critical) | 1 h |
| Redis replication link down | A replica lost its link to the primary (dumbmonit_redis_master_link_up). | < 1 | 2 min | Warning (critical) | 1 h |
| Redis replica lagging | Seen from the primary, a replica has not acknowledged for more than 30 seconds (dumbmonit_redis_replica_lag_seconds). | > 30 s | 5 min | Advisory (warning) | 6 h |
| Redis snapshot failed | The last BGSAVE failed; with stop-writes-on-bgsave-error, writes are refused (dumbmonit_redis_rdb_last_save_ok). | < 1 | 5 min | Warning (critical) | 1 h |
| Redis append-only file write failed | The last AOF write failed (dumbmonit_redis_aof_last_write_ok). | < 1 | 5 min | Warning (critical) | 1 h |
| MongoDB replica set without primary | No member is primary: no write is accepted (dumbmonit_mongodb_replset_primary_present). | < 1 | 2 min | Warning (critical) | 1 h |
| MongoDB member unreachable | A replica set member is unreachable from the others (dumbmonit_mongodb_replset_member_health). | < 1 | 5 min | Warning (critical) | 1 h |
| MongoDB replication lag | A secondary is more than a minute behind the primary (dumbmonit_mongodb_replset_member_lag_seconds); clears below 30 s. | > 60 s | 10 min | Advisory (warning) | 6 h |
| MongoDB connections near the limit | More than 80 % of the available connections in use (dumbmonit_mongodb_connections_used_percent); clears below 75 %. | > 80 % | 10 min | Advisory (warning) | 6 h |
| MongoDB cache under pressure | More than 20 % of the WiredTiger cache is dirty (dumbmonit_mongodb_wiredtiger_cache_dirty_percent); clears below 15 %. | > 20 % | 10 min | Advisory (warning) | 6 h |
| MongoDB internal errors | Regular assertions in the last hour (increase_prometheus(dumbmonit_mongodb_asserts_total{type="regular"}[1h])). | > 0 | 15 min | Advisory (warning) | 24 h |
| RabbitMQ memory or disk alarm | A node raised a resource alarm: every publisher is blocked (dumbmonit_rabbitmq_alarm, one series per node and resource). | > 0 | 1 min | Warning (critical) | 1 h |
| RabbitMQ node down | A cluster node is not running (dumbmonit_rabbitmq_node_running). | < 1 | 2 min | Warning (critical) | 1 h |
| RabbitMQ network partition | A node sees a network partition (dumbmonit_rabbitmq_node_partitions). | > 0 | 2 min | Warning (critical) | 1 h |
| RabbitMQ queue without consumer | A queue holds messages and has no consumer (dumbmonit_rabbitmq_queue_messages_ready and dumbmonit_rabbitmq_queue_consumers == 0); the value is the number of messages waiting. | > 0 | 15 min | Advisory (warning) | 6 h |
| RabbitMQ queue unavailable | A queue is not running: quorum queue without majority, or its node is down (dumbmonit_rabbitmq_queue_running). | < 1 | 5 min | Advisory (warning) | 6 h |
| CrowdSec Local API down | The Local API does not answer /health (dumbmonit_crowdsec_lapi_up). | < 1 | 5 min | Warning (critical) | 1 h |
| CrowdSec bouncer stopped pulling | A bouncer’s requests did not change in 30 minutes (changes_prometheus(dumbmonit_crowdsec_bouncer_requests_total[30m])). | < 1 | 5 min | Advisory (warning) | 6 h |
| CrowdSec reads no logs | Not a single log line read in six hours (increase_prometheus(dumbmonit_crowdsec_lines_read_total[6h])). | < 1 | 30 min | Advisory (warning) | 24 h |
More databases and Kafka
Section titled “More databases and Kafka”SQL Server, Oracle, Elasticsearch / OpenSearch, InfluxDB and Kafka (SQL Server, Oracle, Elasticsearch, InfluxDB, Kafka). Counters go through increase_prometheus, so nothing fires on a monitor’s first measurement.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| SQL Server database unavailable | A database is SUSPECT, RECOVERY_PENDING, OFFLINE or EMERGENCY (dumbmonit_mssql_database_online). | < 1 | 5 min | Warning (critical) | 1 h |
| SQL Server blocked requests | A request has been waiting on a lock for more than 5 minutes: a transaction was probably left open (dumbmonit_mssql_longest_block_seconds). | > 300 s | 5 min | Advisory (warning) | 6 h |
| SQL Server volume almost full | A volume holding database files is more than 90 % full; clears below 85 % (dumbmonit_mssql_volume_used_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| SQL Server Agent job failed | The last run of an enabled SQL Server Agent job failed (backups, maintenance) (dumbmonit_mssql_agent_job_last_ok). | < 1 | 5 min | Advisory (warning) | 24 h |
| SQL Server AlwaysOn replica unhealthy | A database replica of an availability group is not synchronizing in good health (dumbmonit_mssql_alwayson_sync_health). | < 2 | 10 min | Warning (critical) | 1 h |
| SQL Server connections near the limit | User sessions use more than 90 % of the configured connection limit. No series without a limit (dumbmonit_mssql_sessions_used_percent). | > 90 % | 10 min | Advisory (warning) | 6 h |
| Oracle instance not open | The instance status is not OPEN (MOUNTED, STARTED…): the database serves nothing (dumbmonit_oracle_instance_open). | < 1 | 2 min | Warning (critical) | 1 h |
| Oracle tablespace almost full | A tablespace is more than 90 % full of its maximum size; clears below 85 % (dumbmonit_oracle_tablespace_used_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Oracle sessions near the limit | Sessions use more than 90 % of the sessions parameter (ORA-00018 when it is reached) (dumbmonit_oracle_sessions_used_percent). | > 90 % | 10 min | Advisory (warning) | 6 h |
| Oracle Data Guard apply lag | A standby applies redo more than 10 minutes behind the primary (dumbmonit_oracle_dataguard_apply_lag_seconds). | > 600 s | 10 min | Advisory (warning) | 6 h |
| Oracle recovery area almost full | The fast recovery area is more than 90 % full: archiving can stall and freeze the database (dumbmonit_oracle_recovery_area_used_percent). | > 90 % | 10 min | Warning (critical) | 1 h |
| Oracle scheduler job failed | At least one scheduler job run failed in the last 24 hours (dumbmonit_oracle_scheduler_jobs_failed_24h). | > 0 | 15 min | Advisory (warning) | 24 h |
| Elasticsearch cluster red | The cluster status is red: at least one primary shard is not allocated and part of the data is unavailable (dumbmonit_elasticsearch_cluster_status). | > 1 | 2 min | Warning (critical) | 1 h |
| Elasticsearch cluster yellow | The cluster has been yellow for 30 minutes: replicas are missing and the next node failure will lose data (dumbmonit_elasticsearch_cluster_status == 1). | > 0 | 30 min | Advisory (warning) | 6 h |
| Elasticsearch unassigned shards | Shards have stayed unassigned for 15 minutes (dumbmonit_elasticsearch_unassigned_shards). | > 0 | 15 min | Advisory (warning) | 6 h |
| Elasticsearch node disk almost full | A node’s data disk is more than 85 % full (the high watermark is 90 %, the read-only flood stage 95 %); clears below 80 % (dumbmonit_elasticsearch_node_disk_used_percent). | > 85 % | 10 min | Advisory (warning) | 6 h |
| Elasticsearch JVM heap pressure | A node’s JVM heap stays above 90 % for 15 minutes: garbage collection will slow the node (dumbmonit_elasticsearch_node_heap_used_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| InfluxDB unhealthy | InfluxDB answers its health check with a failure (dumbmonit_influxdb_healthy). | < 1 | 2 min | Warning (critical) | 1 h |
| InfluxDB not ready | InfluxDB is up but does not report itself ready for queries and writes (dumbmonit_influxdb_ready). | < 1 | 5 min | Advisory (warning) | 6 h |
| InfluxDB server errors | More than 10 API requests answered with a 5xx status in the last 15 minutes (increase_prometheus(dumbmonit_influxdb_http_requests_5xx_total[15m])). | > 10 | 5 min | Advisory (warning) | 6 h |
| Kafka partitions offline | A partition has no leader: reads and writes to it fail (dumbmonit_kafka_offline_partitions). | > 0 | 2 min | Warning (critical) | 1 h |
| Kafka under-replicated partitions | Partitions have fewer in-sync replicas than replicas for 10 minutes: a broker is down or lagging (dumbmonit_kafka_under_replicated_partitions). | > 0 | 10 min | Advisory (warning) | 6 h |
| Kafka without controller | The cluster reports no controller: no leader election is possible (dumbmonit_kafka_controller_present). | < 1 | 2 min | Warning (critical) | 1 h |
| Kafka consumer group lag | A consumer group is more than 10 000 messages behind for 15 minutes. Adjust the threshold to the traffic of your topics (dumbmonit_kafka_consumer_group_lag_total). | > 10000 | 15 min | Advisory (warning) | 6 h |
Reverse proxies and domains
Section titled “Reverse proxies and domains”Traefik, Caddy, Nginx Proxy Manager and domain registrations (Traefik, Caddy, Nginx Proxy Manager, Domain expiry), and the DNS check’s Alert when the answer changes option (Services). Series carry router, service, server, upstream, proxy_host or cert, so the notification names what failed.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Traefik router disabled | A router is disabled by a configuration error (missing service or middleware, invalid rule): its sites are not served (dumbmonit_traefik_router_status: 0 enabled, 1 warning, 2 disabled). | ≥ 2 | 5 min | Advisory (warning) | 6 h |
| Traefik backend server down | A server of a load-balanced service fails its health check (dumbmonit_traefik_server_up). | < 1 | 5 min | Advisory (warning) | 6 h |
| Traefik service without a server | No server of a service answers its health check: its sites get 503 (dumbmonit_traefik_service_servers_up). | < 1 | 2 min | Warning (critical) | 1 h |
| Traefik certificate not obtained | Routers ask a certificate resolver for a certificate that no certificate held by Traefik covers yet: Traefik serves its default self-signed certificate (dumbmonit_traefik_resolver_routers_uncovered, needs Traefik’s /api/certificates). | > 0 | 1 h | Advisory (warning) | 24 h |
| Caddy upstream unhealthy | A backend fails Caddy’s active or passive health check (dumbmonit_caddy_upstream_healthy). | < 1 | 5 min | Advisory (warning) | 6 h |
| Caddy configuration reload failed | Caddy refused its last configuration reload and still runs the previous one (dumbmonit_caddy_config_last_reload_ok). | < 1 | 5 min | Advisory (warning) | 24 h |
| Nginx Proxy Manager host offline | nginx refused an enabled host’s configuration: the host is not served (dumbmonit_npm_host_online). | < 1 | 10 min | Advisory (warning) | 6 h |
| Proxy certificate not renewed | A certificate Traefik holds or a Nginx Proxy Manager host presents expires in less than 14 days: automatic renewal starts at 30, so it is failing (dumbmonit_traefik_cert_expiry_days or dumbmonit_npm_cert_expiry_days). | < 14 d | 1 h | Advisory (warning) | 24 h |
| Reverse proxy answering 5xx | More than 5 % of the answers over ten minutes were 5xx, with at least fifty requests (increase_prometheus of dumbmonit_{traefik,caddy}_requests_5xx_total over …_requests_total). Needs the proxy’s metrics. | > 5 % | 10 min | Advisory (warning) | 6 h |
| Domain expiring | The domain expires in less than 30 days (dumbmonit_domain_expiry_days >= 7). | < 30 d | 1 h | Advisory (warning) | 24 h |
| Domain about to expire | The domain expires in less than 7 days, or has expired (dumbmonit_domain_expiry_days). | < 7 d | 1 h | Warning (critical) | 12 h |
| Domain on hold | The registry put the domain in clientHold or serverHold: it is out of the DNS (dumbmonit_domain_on_hold). | > 0 | 30 min | Warning (critical) | 6 h |
| Domain in redemption | The domain is in redemptionPeriod, pendingDelete or pendingRestore: it has expired (dumbmonit_domain_redemption). | > 0 | 30 min | Warning (critical) | 12 h |
| Domain name servers changed | The name servers of the delegation differ from the previous reading (changes_prometheus(dumbmonit_domain_nameservers_fingerprint[1d])). Resolves by itself a day later. | > 0 | 1 min | Advisory (warning) | 24 h |
| Domain registrar changed | The registrar differs from the previous reading: a transfer happened (changes_prometheus(dumbmonit_domain_registrar_fingerprint[1d])). | > 0 | 1 min | Advisory (warning) | 24 h |
| Domain DNSSEC lost | DNSSEC was signed in the last 7 days and is not any more (dumbmonit_domain_dnssec). | < 1 | 1 h | Advisory (warning) | 24 h |
| DNS answer changed | The answer of a DNS check with Alert when the answer changes differs from the one before (changes_prometheus(dumbmonit_probe_dns_answer_fingerprint[1h])). Resolves by itself an hour later. | > 0 | 1 min | Advisory (warning) | 24 h |
DNS blocklists
Section titled “DNS blocklists”Mail server reputation and IP range blocklist. The lists are only asked every 6 hours; the last result is written again at every probe, hence the short hold. The monitor page links to the removal page of every listing.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Listed on a major blocklist | An address or the domain is on Spamhaus (ZEN or DBL), Barracuda or SpamCop: a large share of its mail is refused (dumbmonit_blacklist_listed_major_lists). | > 0 | 5 min | Warning (critical) | 24 h |
| Listed on a blocklist | An address or the domain is on a smaller list: Mailspike, UCEPROTECT, PSBL, SpamRATS, SURBL or URIBL (dumbmonit_blacklist_listed_lists - dumbmonit_blacklist_listed_major_lists). | > 0 | 5 min | Advisory (warning) | 24 h |
| Mail server reverse DNS mismatch | An address of the mail server has no reverse DNS, or its name does not resolve back to it: Gmail, Outlook.com and Yahoo refuse or junk such mail (dumbmonit_blacklist_ptr_ok == bool 0). | > 0 | 5 min | Advisory (warning) | 24 h |
| SPF or DMARC missing | The domain has no SPF record (or several), or no DMARC record: Gmail and Yahoo require both from bulk senders (dumbmonit_blacklist_mail_auth_missing). | > 0 | 5 min | Advisory (warning) | 24 h |
Website changes
Section titled “Website changes”See Website changes. The monitor page shows each change before and after.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Website changed | A check of a watched page or site found pages that changed, appeared or disappeared (dumbmonit_webchange_last_check_changes). Each such check is notified and names the pages, under the same alert while it fires; it resolves at the first check that finds nothing new. | > 0 | 30 s | Info (info) | none |
Network configuration backups
Section titled “Network configuration backups”See Network configuration backup. The device page shows each version and the line diff, secrets masked.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Network configuration changed | The last backup found a configuration different from the previous version, volatile lines (timestamps, counters) aside (dumbmonit_netconfig_config_changed). It resolves at the next backup that finds no change. | > 0 | 30 s | Info (info) | none |
| SSH host key changed | The device presented an SSH host key other than the remembered one: the connection was cut before any credential was sent, and backups stop until an administrator accepts the new key (dumbmonit_netconfig_host_key_changed). | > 0 | none | Warning (critical) | 6 h |
Kubernetes
Section titled “Kubernetes”See Kubernetes / k3s. Every series is labelled with the node, or the namespace and the pod, workload or claim concerned.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Kubernetes node not ready | A node is not Ready: its kubelet no longer reports, and its pods will be evicted (dumbmonit_k8s_node_ready == bool 0). | > 0 | 5 min | Warning (critical) | 6 h |
| Kubernetes node under pressure | A node reports memory, disk or PID pressure (dumbmonit_k8s_node_pressure, label condition). | > 0 | 5 min | Advisory (warning) | 6 h |
| Kubernetes pod crash looping | A container is in CrashLoopBackOff and still restarting: at least two restarts in 15 minutes (max_over_time(dumbmonit_k8s_pod_crashlooping[10m]) and on (target, namespace, pod) increase_prometheus(dumbmonit_k8s_pod_restarts[15m]) >= 2; the state disappears for a moment at each restart attempt). Containers killed by a node or runtime restart (exit 137, 143 or 255, not OOMKilled) do not count, so a reboot does not look like a crash loop. | > 0 | 10 min | Warning (critical) | 6 h |
| Kubernetes pod stuck pending | A pod has been Pending for fifteen minutes: no node can take it, or a volume or an image it needs does not come (dumbmonit_k8s_pod_pending). | > 0 | 15 min | Advisory (warning) | 6 h |
| Kubernetes pod restarting | A pod restarted more than five times in the last hour without being in CrashLoopBackOff (increase_prometheus(dumbmonit_k8s_pod_restarts[1h]) unless on (target, namespace, pod) max_over_time(dumbmonit_k8s_pod_crashlooping[10m]) > 0). | > 5 | 5 min | Advisory (warning) | 6 h |
| Kubernetes workload missing replicas | A Deployment, StatefulSet or DaemonSet has fewer available replicas than it asks for (dumbmonit_k8s_workload_unavailable). | > 0 | 10 min | Advisory (warning), escalates after 1 h | 6 h |
| Kubernetes volume claim pending | A PersistentVolumeClaim that a pod needs, or that a provisioning error concerns, is still not bound (dumbmonit_k8s_pvc_pending). | > 0 | 15 min | Advisory (warning) | 6 h |
Active Directory
Section titled “Active Directory”See Active Directory. The domain controller
DumbMonit queries is covered by Monitor unreachable; the other domain
controllers of the domain by Domain controller unreachable (label dc).
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Domain controller unreachable | A domain controller of the domain no longer answers on its LDAP port (dumbmonit_ad_dc_reachable == bool 0). A name that does not resolve from DumbMonit produces no series and never fires. | > 0 | 5 min | Warning (critical) | 6 h |
| Active Directory bind failed | The domain controller answers but refuses the service account: expired password, disabled or locked-out account (dumbmonit_ad_bind_ok == bool 0). | > 0 | 5 min | Advisory (warning) | 24 h |
| Privileged group membership changed | A member joined or left Domain Admins, Enterprise Admins, Schema Admins, Administrators or an operators group, nesting included (changes_prometheus(dumbmonit_ad_privileged_group_fingerprint[1h]), label group). Resolves by itself an hour later. | > 0 | 1 min | Advisory (warning) | 24 h |
| Active Directory replication failing | The queried domain controller cannot replicate from a partner (dumbmonit_ad_replication_consecutive_failures). Only when the account may read replication status. | > 0 | 30 min | Advisory (warning) | 6 h |
| krbtgt password older than 180 days | The krbtgt password was last changed more than 180 days ago (dumbmonit_ad_krbtgt_password_age_seconds). | > 180 days | 10 min | Info (info) | none |
pfSense
Section titled “pfSense”See pfSense. Gateways are labelled gateway, interfaces interface and descr, services service.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| pfSense gateway down | A pfSense gateway is down: its monitor address no longer answers, and the traffic it carries stops or fails over (dumbmonit_pfsense_gateway_up). | < 1 | 3 min | Warning (critical) | 6 h |
| pfSense gateway losing packets | A pfSense gateway loses more than 10 % of the probes to its monitor address: the line is degraded (dumbmonit_pfsense_gateway_loss_percent unless on (target, gateway) dumbmonit_pfsense_gateway_up == 0). | > 10 % | 10 min | Advisory (warning) | 6 h |
| pfSense gateway slow | The round trip to a pfSense gateway’s monitor address has stayed above 500 ms for fifteen minutes (dumbmonit_pfsense_gateway_delay_milliseconds). | > 500 ms | 15 min | Advisory (warning) | 6 h |
| pfSense interface without link | An enabled pfSense interface has no link: cable, switch port or network card (dumbmonit_pfsense_interface_down). | > 0 | 5 min | Advisory (warning) | 6 h |
| pfSense service stopped | A pfSense service that is enabled is not running: DNS resolver, VPN, NTP… (dumbmonit_pfsense_service_stopped). | > 0 | 5 min | Advisory (warning) | 6 h |
| pfSense disk nearly full | The pfSense disk is more than 90 % full: logs, reports and package updates stop being written (dumbmonit_pfsense_disk_used_percent). | > 90 % | 30 min | Advisory (warning) | 1 d |
Unraid
Section titled “Unraid”See Unraid. Disks are labelled disk and role (parity, data, cache), containers container.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Unraid array stopped | The Unraid array is not started: shares, containers and VMs that live on it are unavailable (dumbmonit_unraid_array_started). | < 1 | 10 min | Warning (critical) | 6 h |
| Unraid disk disabled or missing | An Unraid disk is disabled, missing or invalid: the array runs degraded, and one more failure loses data (dumbmonit_unraid_disk_ok). | < 1 | 5 min | Warning (critical) | 6 h |
| Unraid disk read errors | An Unraid disk has reported read errors since the array started: it may be disabled at the next one (dumbmonit_unraid_disk_errors). | > 0 | 5 min | Advisory (warning) | 1 d |
| Unraid disk too hot | An Unraid array or parity disk has stayed above 55 °C for fifteen minutes (dumbmonit_unraid_disk_temperature_celsius{role=~"data|parity"}). | > 55 °C | 15 min | Advisory (warning) | 6 h |
| Unraid parity check found errors | The last Unraid parity check failed or found sync errors: parity no longer matches the data (dumbmonit_unraid_parity_check_ok). | < 1 | 5 min | Advisory (warning) | 1 d |
| Unraid parity check overdue | The last Unraid parity check is more than 40 days old: a monthly check was missed (dumbmonit_unraid_parity_check_age_seconds). | > 40 d | 1 h | Advisory (warning) | 1 d |
| Unraid cache nearly full | An Unraid cache pool is more than 90 % full: writes fall back to the slow array, or fail (dumbmonit_unraid_disk_used_percent{role="cache"}). | > 90 % | 30 min | Advisory (warning) | 1 d |
| Unraid container stopped | A Docker container set to start with the array is not running (dumbmonit_unraid_container_autostart_stopped). | > 0 | 10 min | Advisory (warning) | 6 h |
Veeam Backup & Replication
Section titled “Veeam Backup & Replication”See Veeam Backup & Replication. Jobs are labelled job and type, repositories repository.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Veeam job failed | The last run of an enabled Veeam job failed: what it protects has no new restore point (dumbmonit_veeam_job_failed). | > 0 | 5 min | Warning (critical) | 1 d |
| Veeam job ended with a warning | The last run of an enabled Veeam job ended with a warning: some objects may not be protected (dumbmonit_veeam_job_warning). | > 0 | 5 min | Advisory (warning) | 1 d |
| Veeam backup too old | No successful run of a job scheduled at least daily for more than two days (dumbmonit_veeam_job_last_success_age_seconds). | > 2 d | 1 h | Advisory (warning) | 1 d |
| Veeam repository nearly full | A Veeam backup repository is more than 90 % full: the next jobs will fail for lack of space (dumbmonit_veeam_repository_used_percent). | > 90 % | 30 min | Advisory (warning) | 1 d |
| Veeam license expiring | The Veeam license expires in less than 30 days: jobs stop at the end of the grace period (dumbmonit_veeam_license_expiry_seconds). | < 30 d | 1 h | Advisory (warning) | 1 d |
Windows without an agent
Section titled “Windows without an agent”See Windows without an agent (WinRM).
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Windows automatic services stopped | One or more Windows services set to start automatically are stopped (dumbmonit_windows_services_auto_stopped). | > 0 | 10 min | Advisory (warning) | 6 h |
See Kopia. Sources are labelled source (user@host:/path). Beta: not yet tested on a real server.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Kopia snapshot with errors | The last snapshot of a Kopia source finished with errors: some files were not saved (dumbmonit_kopia_source_last_snapshot_errors). | > 0 | 5 min | Advisory (warning) | 1 d |
| Kopia snapshot too old | A scheduled Kopia source has no snapshot for more than two days (dumbmonit_kopia_source_last_snapshot_age_seconds, scheduled sources only). | > 2 d | 30 min | Advisory (warning) | 1 d |
| Kopia repository disconnected | The Kopia server is not connected to its repository: no snapshot can run (dumbmonit_kopia_repo_connected). | < 1 | 10 min | Warning (critical) | 6 h |
Duplicati
Section titled “Duplicati”See Duplicati. Backups are labelled backup and backup_id. Beta: not yet tested on a real server.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Duplicati backup failed | The last run of a Duplicati backup failed: what it protects has no new version (dumbmonit_duplicati_backup_failed). | > 0 | 5 min | Warning (critical) | 1 d |
| Duplicati backup too old | A scheduled Duplicati backup has not succeeded for more than two days (dumbmonit_duplicati_backup_last_success_age_seconds, scheduled backups only). | > 2 d | 30 min | Advisory (warning) | 1 d |
Tailscale
Section titled “Tailscale”See Tailscale. Devices are labelled device, the first label of their MagicDNS name.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Tailscale device offline | A Tailscale device that must stay online (a tagged device by default) has been disconnected for more than 15 minutes (dumbmonit_tailscale_device_offline_seconds and on (target, device) dumbmonit_tailscale_device_watched > 0). | > 15 min | 5 min | Advisory (warning) | 6 h |
| Tailscale node key expiring | A Tailscale device’s node key expires in less than 14 days: it will leave the tailnet until someone signs it in again (dumbmonit_tailscale_device_key_expiry_seconds). | < 14 d | 1 h | Advisory (warning) | 1 d |
| Tailscale device awaiting approval | A device joined the tailnet and awaits approval: approve it if it is expected, remove it otherwise (dumbmonit_tailscale_device_authorized). | < 1 | 5 min | Advisory (warning) | 1 d |
| Tailscale client update | A Tailscale device that must stay online runs a client with an update available (dumbmonit_tailscale_device_update_available and on (target, device) dumbmonit_tailscale_device_watched > 0). | > 0 | 6 h | Info (info) | 7 d |
Microsoft 365
Section titled “Microsoft 365”See Microsoft 365. Rules for Microsoft 365 tenants read through Microsoft Graph: service health, licenses and risky users.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Microsoft 365 service degraded | A Microsoft 365 service is degraded, interrupted or under investigation according to the service health dashboard. (dumbmonit_m365_services_degraded) | > 0 | 10 min | Advisory (warning) | 6 h |
| Microsoft 365 licenses exhausted | Every unit of a paid license is assigned: the next user cannot be given it. (dumbmonit_m365_license_used_percent) | ≥ 100 % | 1 h | Advisory (warning) | 1 d |
| Microsoft 365 licenses nearly full | More than 95 % of a paid license is assigned. (dumbmonit_m365_license_used_percent < 99.99) | > 95 % | 1 h | Info (info) | 7 d |
| Microsoft 365 users at risk | Entra ID Protection flags at least one user as at risk. (dumbmonit_m365_risky_users) | > 0 | 15 min | Advisory (warning) | 1 d |
Microsoft Entra ID
Section titled “Microsoft Entra ID”See Microsoft Entra ID. Rules for Microsoft Entra ID tenants read through Microsoft Graph: application secrets, directory sync and administrator MFA.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Entra application secret expiring | A secret or certificate of an Entra application expires in less than 30 days: the application stops signing in on that day. (dumbmonit_entra_credential_expiry_seconds > 0) | < 30 d | 1 h | Advisory (warning) | 1 d |
| Entra application secret expired | A secret or certificate of an Entra application expired in the last 30 days: remove it, or renew it if the application still needs it. (dumbmonit_entra_credential_expiry_seconds) | expired | 1 h | Advisory (warning) | 1 d |
| Entra directory sync late | The last synchronization with the on-premises Active Directory is more than 3 hours old: check Entra Connect. (dumbmonit_entra_ad_sync_age_seconds) | > 3 h | 10 min | Advisory (warning) | 6 h |
| Entra administrator without MFA | At least one administrator has no multifactor authentication method registered. (dumbmonit_entra_admin_without_mfa) | > 0 | 1 h | Advisory (warning) | 1 d |
Microsoft Azure
Section titled “Microsoft Azure”See Microsoft Azure. Rules for an Azure subscription read through Resource Health and Azure Monitor alerts.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Azure resource unavailable | Resource Health reports a resource of the subscription as unavailable for 10 minutes (not counting resources stopped by their owner). (dumbmonit_azure_resources_unavailable) | > 0 | 10 min | Warning (critical) | 6 h |
| Azure resource degraded | Resource Health reports a resource of the subscription as degraded for 15 minutes. (dumbmonit_azure_resources_degraded) | > 0 | 15 min | Advisory (warning) | 6 h |
| Azure Monitor alert Sev0 or Sev1 | An Azure Monitor alert of severity Sev0 (critical) or Sev1 (error) is fired and not closed. (dumbmonit_azure_alerts_high_active) | > 0 | 5 min | Advisory (warning) | 6 h |
AWS CloudWatch
Section titled “AWS CloudWatch”See AWS (CloudWatch). AWS alarms, EC2 instances and RDS databases read through CloudWatch.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| AWS CloudWatch alarm in ALARM | A CloudWatch alarm has been in the ALARM state for 5 minutes. (dumbmonit_aws_alarm_in_alarm) | > 0 | 5 min | Advisory (warning) | 6 h |
| AWS EC2 status check failed | An EC2 instance has failed an instance or system status check for 10 minutes. (dumbmonit_aws_ec2_status_check_failed) | > 0 | 10 min | Advisory (warning) | 6 h |
| AWS EC2 CPU high | An EC2 instance has had a CPU peak above 90 % for 30 minutes. (dumbmonit_aws_ec2_cpu_percent) | > 90 % | 30 min | Advisory (warning) | 6 h |
| AWS RDS CPU high | An RDS database has had a CPU peak above 90 % for 30 minutes. (dumbmonit_aws_rds_cpu_percent) | > 90 % | 30 min | Advisory (warning) | 6 h |
| AWS RDS free storage low | An RDS database has less than 5 GiB of free storage for 15 minutes. (dumbmonit_aws_rds_free_storage_bytes) | < 5 GiB | 15 min | Advisory (warning) | 6 h |
Cloudflare
Section titled “Cloudflare”See Cloudflare. Rules for the zones, tunnels, certificates and load balancer pools of a Cloudflare account.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Cloudflare tunnel down | A Cloudflare tunnel has no connection left and cannot carry traffic. (dumbmonit_cloudflare_tunnel_down) | > 0 | 5 min | Warning (critical) | 6 h |
| Cloudflare tunnel degraded | A Cloudflare tunnel has lost some of its connections to the edge. (dumbmonit_cloudflare_tunnel_degraded) | > 0 | 15 min | Advisory (warning) | 6 h |
| Cloudflare certificate expiring | An active certificate of a zone expires in less than 14 days; Cloudflare should have renewed it by now. (dumbmonit_cloudflare_certificate_expiry_seconds) | < 14 d | 1 h | Advisory (warning) | 1 d |
| Cloudflare zone not active | A zone is not active (pending name server change, moved or deactivated). (dumbmonit_cloudflare_zone_active) | < 1 | 1 h | Advisory (warning) | 1 d |
| Cloudflare 5xx rate high | More than 5% of the responses of a zone are 5xx over the last one to two hours (zones with at least 100 requests). (dumbmonit_cloudflare_zone_http_5xx_percent) | > 5 % | 15 min | Advisory (warning) | 6 h |
| Cloudflare load balancer pool unhealthy | An enabled load balancer pool is unhealthy. (dumbmonit_cloudflare_lb_pool_healthy) | < 1 | 5 min | Advisory (warning) | 6 h |
Hetzner Cloud
Section titled “Hetzner Cloud”See Hetzner Cloud. Rules for the servers, volumes and backups of a Hetzner Cloud project.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Hetzner server not running | A Hetzner Cloud server is not in the running state (off, migrating, stuck starting). (dumbmonit_hetzner_server_running) | < 1 | 10 min | Advisory (warning) | 6 h |
| Hetzner server CPU sustained high | A Hetzner server averages more than 90% CPU over ten minutes, for 30 minutes. (dumbmonit_hetzner_server_cpu_percent) | > 90 % | 30 min | Advisory (warning) | 6 h |
| Hetzner backup too old | The newest backup of a server with backups enabled is more than 3 days old. (dumbmonit_hetzner_server_backup_age_seconds) | > 3 d | 1 h | Advisory (warning) | 1 d |
| Hetzner volume not attached | A volume is attached to no server, and is billed anyway. (dumbmonit_hetzner_volume_attached) | < 1 | 1 h | Info (info) | 7 d |
OVHcloud
Section titled “OVHcloud”See OVHcloud. Rules for the VPS and dedicated servers of an OVHcloud account.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| OVHcloud VPS not running | An OVHcloud VPS is not in the running state for 10 minutes. (dumbmonit_ovh_vps_up) | < 1 | 10 min | Advisory (warning) | 6 h |
| OVHcloud dedicated server in error | OVHcloud monitoring reports a dedicated server in error, or powered off, for 5 minutes. (dumbmonit_ovh_dedicated_up) | < 1 | 5 min | Warning (critical) | 6 h |
| OVHcloud task in progress | One or more tasks (reboot, reinstall, maintenance) have been pending or running on the servers for an hour. (dumbmonit_ovh_tasks_active) | > 0 | 1 h | Info (info) | 1 d |
Infomaniak
Section titled “Infomaniak”See Infomaniak. Rules for the products of an Infomaniak account.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Infomaniak product expiring | A product (domain, hosting, mail) expires in less than 14 days. (dumbmonit_infomaniak_product_expiry_seconds) | < 14 d | 1 h | Advisory (warning) | 1 d |
| Infomaniak product expired | The expiry date of a product has passed. (dumbmonit_infomaniak_product_expiry_seconds) | < 0 | 1 h | Warning (critical) | 1 d |
| Infomaniak product locked | Infomaniak flags a product as locked for an hour. (dumbmonit_infomaniak_product_locked) | > 0 | 1 h | Advisory (warning) | 1 d |
| Infomaniak product in maintenance | Infomaniak flags a product as under maintenance for an hour. (dumbmonit_infomaniak_product_maintenance) | > 0 | 1 h | Info (info) | 1 d |
FortiGate
Section titled “FortiGate”See FortiGate. Tunnels are labelled tunnel, interfaces interface and alias, licences license.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| FortiGate IPsec tunnel down | No phase 2 of a FortiGate site-to-site IPsec tunnel is up: the site behind it is cut off (dumbmonit_fortigate_ipsec_tunnel_up). | < 1 | 5 min | Warning (critical) | 6 h |
| FortiGate interface without link | An enabled FortiGate interface with an address has no link: cable, transceiver or the device in front (dumbmonit_fortigate_interface_down). | > 0 | 5 min | Advisory (warning) | 6 h |
| FortiGate HA out of sync | The members of a FortiGate HA cluster no longer have the same configuration checksum: a failover would run another configuration (dumbmonit_fortigate_ha_in_sync). | < 1 | 15 min | Advisory (warning) | 6 h |
| FortiGate HA member lost | A FortiGate HA cluster has fewer members than in the last 24 hours: there is no standby any more (max_over_time(dumbmonit_fortigate_ha_members[1d]) - dumbmonit_fortigate_ha_members). | > 0 | 5 min | Warning (critical) | 6 h |
| FortiGate memory near conserve mode | FortiGate memory has stayed above 85 % for ten minutes: at 88 % (by default) it enters conserve mode and stops inspecting new sessions (dumbmonit_fortigate_memory_used_percent). | > 85 % | 10 min | Advisory (warning) | 6 h |
| FortiGate CPU high | FortiGate CPU has stayed above 90 % for fifteen minutes (dumbmonit_fortigate_cpu_usage_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| FortiGate licence expiring | A FortiGuard or FortiCare licence of the FortiGate expires in less than 30 days, or has expired (dumbmonit_fortigate_license_expiry_seconds). | < 30 d | 1 h | Advisory (warning) | 1 d |
| FortiGate firmware update | FortiGuard offers a newer patch release of the FortiOS branch the FortiGate runs (dumbmonit_fortigate_firmware_update_available). | > 0 | 1 h | Info (info) | 7 d |
Sophos Firewall
Section titled “Sophos Firewall”See Sophos Firewall.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Sophos interface without link | A Sophos Firewall interface that is switched on and bound to a zone has no link (dumbmonit_sophos_interface_down). | > 0 | 5 min | Advisory (warning) | 6 h |
Cisco Meraki
Section titled “Cisco Meraki”See Cisco Meraki (beta). Devices are labelled device, serial, model and network, uplinks interface.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Meraki device offline | A Cisco Meraki device has been offline (out of reach of the cloud dashboard) for ten minutes. Dormant devices, unseen for a long time, are left out (dumbmonit_meraki_device_offline). | > 0 | 10 min | Advisory (warning) | 6 h |
| Meraki uplink failed | A Meraki router lost the Internet access of a configured uplink (the wired and cellular sockets without a cable are not counted) (dumbmonit_meraki_uplink_failed). | > 0 | 5 min | Advisory (warning) | 6 h |
| Meraki license problem | The Meraki license is not in compliance (expired, missing or out of compliance): the dashboard may stop managing the devices (dumbmonit_meraki_license_ok). | < 1 | 1 h | Advisory (warning) | 1 d |
| Meraki license expiring | A Meraki license expires in less than 30 days (dumbmonit_meraki_license_expiry_seconds). | < 30 d | 1 h | Advisory (warning) | 1 d |
TP-Link Omada
Section titled “TP-Link Omada”See TP-Link Omada (beta). Devices are labelled device, mac, model and site.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Omada device offline | An Omada access point, switch or gateway has been disconnected from the controller for ten minutes (dumbmonit_omada_device_offline). | > 0 | 10 min | Advisory (warning) | 6 h |
| Omada device CPU high | An Omada device has had its CPU above 90 % for fifteen minutes (dumbmonit_omada_device_cpu_usage_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Omada device memory high | An Omada device has had its memory above 90 % for fifteen minutes (dumbmonit_omada_device_memory_used_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Omada firmware update | The Omada controller offers a firmware update for a device (dumbmonit_omada_device_upgrade_available). | > 0 | 1 h | Info (info) | 7 d |
Aruba Central
Section titled “Aruba Central”See Aruba Central. Devices are labelled device, type, serial and site.
| Aruba Central device offline | Aruba Central reports an access point, switch or gateway offline (dumbmonit_arubacentral_device_online). | < 1 | 10 min | Advisory (warning) | 6 h |
| Aruba Central critical alert | Aruba Central has at least one active critical alert (dumbmonit_arubacentral_alerts_active_by_severity{severity="critical"}). | > 0 | 10 min | Advisory (warning) | 6 h |
| Aruba Central devices in poor health | Aruba Central rates at least one device in poor health for an hour (dumbmonit_arubacentral_devices_poor_health). | > 0 | 1 h | Info (info) | 7 d |
Stormshield SNS
Section titled “Stormshield SNS”Read over SNMP by the Stormshield profile; see Stormshield SNS.
| Stormshield HA node dead | A node of a Stormshield HA cluster is declared dead: there is no standby any more (dumbmonit_stormshield_ha_dead_nodes). | > 0 | 5 min | Advisory (warning) | 6 h |
| Stormshield HA link faulty | A link between the nodes of a Stormshield HA cluster is faulty (dumbmonit_stormshield_ha_faulty_links). | > 0 | 5 min | Advisory (warning) | 6 h |
Palo Alto PAN-OS
Section titled “Palo Alto PAN-OS”See Palo Alto PAN-OS (beta). Interfaces are labelled interface and zone, licences license.
| Palo Alto interface without link | A Palo Alto firewall interface bound to a zone has no link: cable, transceiver or the device in front (dumbmonit_paloalto_interface_down). | > 0 | 5 min | Advisory (warning) | 6 h |
| Palo Alto HA peer lost | The peer of a Palo Alto HA pair is no longer active or passive, or the HA link to it is down: there is no standby any more (dumbmonit_paloalto_ha_peer_up). | < 1 | 2 min | Warning (critical) | 6 h |
| Palo Alto HA out of sync | The running configuration of a Palo Alto HA pair is not synchronized: a failover would run another configuration (dumbmonit_paloalto_ha_in_sync). | < 1 | 15 min | Advisory (warning) | 6 h |
| Palo Alto HA member not functional | This Palo Alto firewall reports its HA state as suspended or non-functional (dumbmonit_paloalto_ha_local_functional). | < 1 | 5 min | Warning (critical) | 6 h |
| Palo Alto session table filling up | The session table of a Palo Alto firewall has stayed above 90 % of its capacity for ten minutes: new sessions will be refused when it is full (dumbmonit_paloalto_session_usage_percent). | > 90 % | 10 min | Advisory (warning) | 6 h |
| Palo Alto management CPU high | The management plane CPU of a Palo Alto firewall has stayed above 90 % for fifteen minutes (dumbmonit_paloalto_cpu_usage_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Palo Alto management memory high | The management plane memory of a Palo Alto firewall has stayed above 90 % for fifteen minutes (dumbmonit_paloalto_memory_used_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Palo Alto licence expiring | A licence or subscription of a Palo Alto firewall expires in less than 30 days, or has expired (dumbmonit_paloalto_license_expiry_seconds). | < 30 d | 1 h | Advisory (warning) | 1 d |
| Palo Alto commit failed | The last configuration commit of a Palo Alto firewall failed: the running configuration is not the one that was saved (dumbmonit_paloalto_last_commit_ok). | < 1 | 5 min | Info (info) | 1 d |
SonicWall SonicOS
Section titled “SonicWall SonicOS”See SonicWall SonicOS (beta).
| SonicWall interface without link | A SonicWall interface bound to a zone has no link (dumbmonit_sonicwall_interface_down). | > 0 | 5 min | Advisory (warning) | 6 h |
QNAP QTS
Section titled “QNAP QTS”See QNAP QTS / QuTS hero. Volumes are labelled volume, disks disk and model.
| QNAP volume nearly full | A QNAP volume is more than 90 % full: shares, snapshots and apps stop being able to write (dumbmonit_qnap_volume_used_percent). | > 90 % | 30 min | Advisory (warning) | 1 d |
| QNAP disk health not good | The SMART health of a QNAP disk is no longer good (warning or abnormal): replace it before the RAID loses redundancy (dumbmonit_qnap_disk_ok). | < 1 | 15 min | Advisory (warning) | 6 h |
| QNAP disk too hot | A QNAP disk has stayed above 55 °C for fifteen minutes (dumbmonit_qnap_disk_temperature_celsius). | > 55 °C | 15 min | Advisory (warning) | 6 h |
| QNAP processor too hot | The QNAP processor has stayed above 90 °C for fifteen minutes (dumbmonit_qnap_cpu_temperature_celsius). | > 90 °C | 15 min | Advisory (warning) | 6 h |
| QNAP firmware update | QNAP offers a newer QTS or QuTS hero release (dumbmonit_qnap_firmware_update_available). | > 0 | 1 h | Info (info) | 7 d |
See Ceph. OSDs are labelled osd and host, pools pool.
| Ceph health error | The Ceph cluster reports HEALTH_ERR: data may be unavailable or at risk (dumbmonit_ceph_health_status). | > 1 | 5 min | Warning (critical) | 1 h |
| Ceph health warning | The Ceph cluster has reported HEALTH_WARN for 30 minutes: see its health checks (dumbmonit_ceph_health_status). | > 0 | 30 min | Advisory (warning) | 6 h |
| Ceph OSD down | One or more Ceph OSDs are down: the cluster rebuilds their data elsewhere and loses redundancy meanwhile (dumbmonit_ceph_osds_down). | > 0 | 10 min | Advisory (warning) | 6 h |
| Ceph placement groups not clean | Some Ceph placement groups have not been active+clean for 30 minutes: data is degraded, misplaced or inactive (dumbmonit_ceph_pgs_unclean). | > 0 | 30 min | Advisory (warning) | 6 h |
| Ceph objects unfound | Ceph reports unfound objects: no copy of some data can be found (dumbmonit_ceph_objects_unfound). | > 0 | 5 min | Warning (critical) | 1 h |
| Ceph capacity high | The Ceph cluster is more than 80 % full: it turns read-only at 95 %, and rebuilding after a failure needs free space (dumbmonit_ceph_capacity_used_percent). | > 80 % | 30 min | Advisory (warning) | 1 d |
| Ceph OSD nearly full | A Ceph OSD is more than 85 % full (the default nearfull ratio): writes to it stop at the full ratio (dumbmonit_ceph_osd_used_percent). | > 85 % | 15 min | Advisory (warning) | 12 h |
| Ceph pool nearly full | A Ceph pool is more than 90 % full (dumbmonit_ceph_pool_used_percent). | > 90 % | 15 min | Advisory (warning) | 12 h |
| Ceph monitor out of quorum | A Ceph monitor is out of the quorum: one more loss and the cluster stops (dumbmonit_ceph_mons_out_of_quorum). | > 0 | 5 min | Warning (critical) | 1 h |
See MinIO.
| MinIO node offline | A MinIO node is offline: the cluster runs with less parity (dumbmonit_minio_nodes_offline). | > 0 | 5 min | Warning (critical) | 1 h |
| MinIO drive offline | A MinIO drive is offline: objects are rebuilt from parity until it is back (dumbmonit_minio_drives_offline). | > 0 | 5 min | Advisory (warning) | 6 h |
| MinIO capacity high | The usable capacity of the MinIO cluster is more than 85 % used (dumbmonit_minio_capacity_usable_used_percent). | > 85 % | 30 min | Advisory (warning) | 1 d |
| MinIO heal failures | MinIO could not heal some objects in its current self-healing run (dumbmonit_minio_heal_objects_failed). | > 0 | 30 min | Advisory (warning) | 6 h |
| MinIO replication failures | Objects of a MinIO bucket failed to replicate to the remote site (dumbmonit_minio_replication_failed_objects). | > 0 | 30 min | Advisory (warning) | 6 h |
| MinIO S3 server errors | More than 5 % of the S3 requests MinIO served in the last 10 minutes failed with a server error (5xx) (100 * rate(dumbmonit_minio_s3_requests_5xx_total[10m]) / rate(dumbmonit_minio_s3_requests_total[10m])). | > 5 % | 10 min | Advisory (warning) | 6 h |
KVM (libvirt)
Section titled “KVM (libvirt)”Reported by the Linux agent on a KVM host; see Agent.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| KVM domain crashed | libvirt reports a KVM domain as crashed: the hypervisor had to stop it (dumbmonit_agent_libvirt_domain_crashed). | > 0 | 1 min | Warning (critical) | 1 h |
| libvirt pool almost full | A libvirt storage pool’s allocation reaches 90% of its capacity (dumbmonit_agent_libvirt_pool_used_percent). | ≥ 90 % (clears below 88 %) | 15 min | Advisory (warning), escalates after 24 h | 6 h |
Synology NAS (agent package)
Section titled “Synology NAS (agent package)”Reported by the agent of the Synology package. A UPS
plugged into the NAS is reported under the NUT names, so the
UPS behind NUT rules apply to it; the SMART verdict, when
smartctl is readable, goes through “Disk SMART failing”.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| NAS RAID degraded | A RAID array of the NAS lost a disk or is rebuilding (dumbmonit_agent_dsm_raid_status: 0 normal, 1 resyncing, 2 degraded, 3 inactive). | ≥ 2 | 5 min | Warning (critical) | 6 h |
| NAS volume read-only | A /volumeN is mounted read-only, which DSM does to a crashed volume (dumbmonit_agent_dsm_volume_read_only). | > 0 | 5 min | Warning (critical) | 6 h |
| NAS disk temperature high | A disk is above 55 °C (dumbmonit_agent_dsm_disk_temperature_celsius); clears below 52 °C. | > 55 °C | 15 min | Advisory (warning) | 6 h |
| NAS disk bad sectors growing | New unreadable sectors appeared in the last 24 hours (delta(dumbmonit_agent_dsm_disk_unc_count[24h])). | > 0 | 15 min | Advisory (warning) | 24 h |
| NAS SSD wearing out | An SSD has less than 10 % of its rated life left (dumbmonit_agent_dsm_disk_remaining_life_percent); clears above 12 %. | < 10 % | 1 h | Advisory (warning) | 7 d |
| NAS package stopped | A package that should run (enabled) is stopped (dumbmonit_agent_dsm_package_down). Needs synopkg to answer the agent; a package stopped by hand does not alert. | > 0 | 15 min | Advisory (warning) | 24 h |
XCP-ng / XenServer
Section titled “XCP-ng / XenServer”See XCP-ng / XenServer.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| XCP-ng host offline | A host of the XCP-ng pool no longer answers the coordinator (dumbmonit_xcpng_host_live). | < 1 | 5 min | Warning (critical) | 1 h |
| XCP-ng SR almost full | An XCP-ng storage repository is more than 90% full (dumbmonit_xcpng_sr_used_percent). | ≥ 90 % (clears below 88 %) | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| XCP-ng HA-protected VM down | A VM protected by the pool’s high availability is halted (dumbmonit_xcpng_vm_ha_protected_down). | > 0 | 2 min | Warning (critical) | 1 h |
| XCP-ng pool HA overcommitted | The pool no longer has a plan to survive the host failures it is set to tolerate (dumbmonit_xcpng_pool_ha_overcommitted). | > 0 | 5 min | Advisory (warning), escalates after 1 h | 6 h |
Nutanix
Section titled “Nutanix”See Nutanix AHV.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Nutanix host degraded | Prism reports a Nutanix host as degraded (dumbmonit_nutanix_host_degraded). | > 0 | 5 min | Warning (critical) | 1 h |
| Nutanix storage container almost full | A Nutanix storage container is more than 90% full (dumbmonit_nutanix_storage_container_used_percent). | ≥ 90 % (clears below 88 %) | 15 min | Advisory (warning), escalates after 24 h | 6 h |
| Nutanix critical alerts | Prism Central reports one or more unresolved critical alerts (dumbmonit_nutanix_alerts_critical). | > 0 | 5 min | Warning (critical) | 1 h |
Hyper-V
Section titled “Hyper-V”Reported by the Windows agent on a Hyper-V host; see Agent.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Hyper-V VM health critical | Hyper-V reports a virtual machine in critical health: a resource it needs, most often disk space, is exhausted (dumbmonit_hyperv_vms_health_critical). | > 0 | 5 min | Warning (critical) | 6 h |
| Hyper-V host CPU high | The Hyper-V host’s logical processors have been more than 90 % busy for fifteen minutes, guests included (dumbmonit_hyperv_host_cpu_percent). | > 90 % | 15 min | Advisory (warning) | 6 h |
| Hyper-V virtual disk errors | A Hyper-V virtual disk reported I/O errors in the last hour (increase_prometheus(dumbmonit_hyperv_vhd_errors_total[1h])). | > 0 | 5 min | Advisory (warning) | 6 h |
Security score
Section titled “Security score”Computed by the server every 15 minutes for every monitor whose kind has security checks, and stored as dumbmonit_security_score (0–100), only while the security score is turned on in Settings (off by default). See Security score.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| Security score dropped | The score fell by 10 points or more against its best value of the last 24 hours (max_over_time(dumbmonit_security_score[1d]) - dumbmonit_security_score): a check started failing. | ≥ 10 pts | 30 min | Advisory (warning) | 24 h |
| Security score low | The score is below 40, grade F (dumbmonit_security_score). Raise or lower the threshold to match your own bar. | < 40 | 1 h | Info (info) | 7 d |
Network inventory
Section titled “Network inventory”Raised from the scheduled network scans and their inventory (see Scheduled scans and inventory). All three rules are disabled by default; the first two have severity info, the MAC change one warning: turn them on in the rules list when you want them. There is one series, so one alert, per new device or port; it stays open until the item is marked as known (the metric stops, the alert resolves within a few minutes) and has no reminder. The first scan of a network never raises anything.
| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| New device on the network | A scheduled scan found a device that is not in the inventory (dumbmonit_netscan_unknown_device). | > 0 | none | Info (info), disabled | none |
| New open port on the network | A scheduled scan found a new open port on a known device (dumbmonit_netscan_unknown_port). | > 0 | none | Info (info), disabled | none |
| MAC address changed for an IP | A new device took the address of a known device under another MAC address: spoofing or replaced hardware (dumbmonit_netscan_mac_change). Open until marked as known or ignored. | > 0 | none | Advisory (warning), disabled | none |
DumbMonit itself
Section titled “DumbMonit itself”| Rule | What | Default threshold | Hold | Severity | Reminder |
|---|---|---|---|---|---|
| DumbMonit backup did not run | The scheduled local backup of the DumbMonit database has not run for more than two days (time() - dumbmonit_instance_backup_last_success_seconds). | > 2 d | 1 h | Advisory (warning) | 24 h |
| DumbMonit data volume almost full | The volume that holds /data has less than 10% free space (100 * dumbmonit_instance_data_free_bytes / dumbmonit_instance_data_size_bytes). | < 10 % | 15 min | Advisory (warning) | 24 h |
| DumbMonit metrics database restarting in a loop | The embedded VictoriaMetrics was restarted three times or more in 30 minutes (increase_prometheus(dumbmonit_instance_vm_restarts_total[30m])). | ≥ 3 | 1 min | Warning (critical) | 6 h |
| DumbMonit scheduler stalled | No scheduling round for more than two minutes: monitors are no longer polled (dumbmonit_instance_scheduler_lag_seconds). | > 120 s | 5 min | Warning (critical) | 6 h |
| DumbMonit polls running late | Monitors stay overdue after every scheduling round (min_over_time(dumbmonit_instance_scheduler_backlog[15m])). | > 0 | 15 min | Advisory (warning) | 24 h |
| DumbMonit metric writes failing | Batches refused by VictoriaMetrics or samples dropped in the last 15 minutes (dumbmonit_instance_sample_writes_failed_total, dumbmonit_instance_samples_dropped_total). | > 0 | 5 min | Advisory (warning) | 6 h |
| DumbMonit database growing fast | dumbmonit.db more than doubled in a day and is over 100 MB (dumbmonit_instance_database_bytes). | > 2× | 1 h | Advisory (warning) | 24 h |
These rules and the outgoing heartbeat are explained in Watching DumbMonit itself.
Every built-in rule applies to all monitors and to every enabled channel.
Editing a rule in the UI
Section titled “Editing a rule in the UI”On the Alerts page, the Rules section lists every rule with its severity, a Built-in mark and, when relevant, how many alerts it is firing now. Each rule has:
- a toggle to enable or disable it;
- Edit, which unfolds an inline editor for name, description, operator,
threshold, hold (
for), severity, reminder (repeat every) and escalation (escalate after). The query of a built-in rule is shown but not editable; the query of your own rules is. Baseline rules show their detection parameters read-only; - Delete, for your own rules only.
Per-monitor overrides
Section titled “Per-monitor overrides”A rule can be tuned for one monitor without touching the rule itself: a different threshold, a different clear threshold, or turned off for that monitor alone. Each rule’s editor has a Per-monitor overrides panel to add or remove one directly.
The one-click Ignore action on an alert (in the Now tab or the
alert’s monitor page) is the same mechanism, written with enabled: false:
the monitor stops carrying that rule, immediately, reversibly. The monitor page
lists what it ignores, with Stop ignoring; the rule itself shows Ignored
on N monitors in Alerts → Rules, where the same per-monitor overrides panel
removes it. See Per-monitor overrides
for the API.
A monitor’s Edit page also lists, under Monitored alerts, every rule
that reads a metric the monitor has written in the last 24 hours: for an agent
with Docker, the container rules; for a website monitor, the probe rules. Each
has a switch, on by default. Turning one off writes the same enabled: false
override when you save the form; the data is still collected and graphed. A
monitor added a moment ago shows the list after its first checks.
Creating a threshold rule
Section titled “Creating a threshold rule”Click New rule. The form needs:
| Field | Meaning |
|---|---|
| Name | Shown in alerts and notifications. |
| Query | A MetricsQL expression, evaluated per monitor. Example: dumbmonit_cpu_usage_percent. |
| Operator | >, >=, < or <=. |
| Threshold | The number the query result is compared with. |
| Severity | Info, Advisory or Warning. |
| Hold for | Seconds the condition must last before firing. A non-zero hold absorbs an isolated bad sample. |
Every series returned by the query becomes one potential alert, attached to the
monitor named by its target label. Aggregate with avg by (target, host) (…)
when you want one alert per monitor rather than one per core or per interface.
The alert’s identity ignores the host and tag_* labels whenever target is
present, so renaming a monitor keeps its alerts; a series whose target does
not match an existing, enabled monitor is ignored.
The query language
Section titled “The query language”Queries are MetricsQL, the
PromQL superset of VictoriaMetrics. The metrics are the dumbmonit_* series
described in the metrics reference, and every series
carries a target label (the monitor id) and a host label (its name).
Some patterns used by the built-in rules:
# Age of the last sample, freshest series per device: what "unreachable" meansbottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
# One value per device, whatever the sourceavg by (target, host) (dumbmonit_cpu_load_percent or dumbmonit_proxmox_node_cpu_percent or dumbmonit_cpu_usage_percent)
# A percentage computed from two gauges100 * dumbmonit_storage_bytes_used / dumbmonit_storage_bytes_total
# A boolean condition: the series is returned only when truedumbmonit_ups_output_source == 5
# Extrapolation done by VictoriaMetricspredict_linear(dumbmonit_pbs_datastore_used_percent[6h], 345600) and deriv(dumbmonit_pbs_datastore_used_percent[6h]) > 0
# Availability of a service over thirty daysavg_over_time(dumbmonit_probe_success[30d])Rule kinds in the API
Section titled “Rule kinds in the API”kind is threshold (compare with a threshold), predict (same comparison,
on a predict_linear query; shown as a forecast) or anomaly (seasonal
baseline; params holds k, alpha, mad_floor_abs, mad_floor_rel,
min_samples). The UI creates threshold rules; the other two kinds can be
created through the API.