Prometheus Monitoring System Setup: From Metric Collection to Alert Notifications

Prometheus Monitoring System Setup: From Metric Collection to Alert Notifications

Prometheus is a graduated project of the Cloud Native Computing Foundation (CNCF) and has become the standard tool in the monitoring space.

Architecture Overview

Exporter → Prometheus Server → AlertManager → Notification Channels
                                  ↓
                              Grafana (Visualization)

Installation and Configuration

# docker-compose.yml
version: '3'
services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin

  node-exporter:
    image: prom/node-exporter:latest
    ports:
      - "9100:9100"

Common PromQL Queries

# CPU usage
100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# Memory usage
100 * (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)

# Disk usage
100 * (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"})

Alert Rules

groups:
  - name: server
    rules:
      - alert: HighCPUUsage
        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "CPU usage above 80%"