Monitoring with Grafana and Victoria Metrics
Deployment of an intelligent monitoring stack, including Grafana for creating dynamic dashboards, Victoria Metrics for storing and processing metrics, Node Exporter for monitoring servers, cAdvisor for analyzing containers, Xray Checker for controlling network connections, and Netbird for building a secure network architecture. This solution provides maximum transparency, performance, and security for your infrastructure.
The guide and the dashboards come from two authors:
-
Grafana - an instrument that transforms various forms of metrics received from Victoria Metrics into visual forms for analytical and system monitoring.
-
Victoria Metrics (VMetrics) - is directly a system for collecting and storing metrics from all machines received from the agent, vmagent - a carrier of these metrics to vmetrics.
-
Node Exporter - an instrument for collecting metrics. It collects information about various resources of the node, such as processor load, memory usage, disk input-output, network statistics, and others.
-
cAdvisor - also an instrument for collecting metrics. It collects information about the performance of Docker containers.
-
Xray-Checker - checks the state of your hosts and sends metrics to Victoria Metrics.
Scheme of work
Section titled “Scheme of work”Configuration of your server with Grafana and Victoria Metrics
Section titled “Configuration of your server with Grafana and Victoria Metrics”-
Docker installation:
root@server: ~ curl -fsSL https://get.docker.com | sh -
Create directories and
docker-compose.yml:root@server: ~ mkdir -p /opt/monitoring/{cadvisor,nodeexporter,nginx,vmagent/conf.d} && cd /opt/monitoring && nano docker-compose.yml
The web server configuration is your choice; paste your variant into docker-compose.yml:
services: vmsingle: image: victoriametrics/victoria-metrics:v1.153.0 container_name: vmsingle hostname: vmsingle restart: unless-stopped volumes: - ./victoria-metrics-data:/victoria-metrics-data command: - '--storageDataPath=/victoria-metrics-data' networks: - grafana-network ports: - IP:8428:8428 # Here we specify the NetBird address (in this case the Grafana server)
grafana: image: grafana/grafana:13.2.3 container_name: grafana hostname: grafana restart: unless-stopped volumes: - grafana-data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning environment: - GF_SERVER_DOMAIN=yourdomain.com # The main domain before Grafana, don't forget to add a DNS record for it (if you mounted a subdomain, you need to write the target domain here) - GF_SERVER_ROOT_URL=https://grafana.yourdomain.com/grafana # Full path to Grafana - GF_SERVER_SERVE_FROM_SUB_PATH=true - GF_SERVER_HTTP_PORT=3000 - GF_SERVER_PROTOCOL=http - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=admin - GF_ANALYTICS_REPORTING_ENABLED=false networks: - grafana-network
xray-checker: image: kutovoys/xray-checker:v1.3.1 container_name: xray-checker hostname: xraychecker restart: unless-stopped environment: - "SUBSCRIPTION_URL=https://subscription_link" # Here we specify the subscription link for XrayChecker - "PROXY_STATUS_CHECK_URL=http://google.com/generate_204" - "PROXY_CHECK_INTERVAL=60" ports: - 127.0.0.1:2112:2112 networks: - grafana-network
grafana-nginx: image: nginx:1.30.5 container_name: grafana-nginx volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro - /etc/letsencrypt/live/yourdomain.com/fullchain.pem:/etc/nginx/ssl/yourdomain.com/fullchain.pem:ro - /etc/letsencrypt/live/yourdomain.com/privkey.pem:/etc/nginx/ssl/yourdomain.com/privkey.pem:ro restart: unless-stopped ports: - 443:443 networks: - grafana-network
volumes: grafana-data:
networks: grafana-network: name: grafana-network driver: bridge external: falseCreate nginx.conf:
nano /opt/monitoring/nginx/nginx.confInsert the following content:
ssl_protocols TLSv1.2 TLSv1.3; ssl_ecdh_curve X25519:prime256v1:secp384r1; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305; ssl_prefer_server_ciphers on; ssl_session_timeout 1d; ssl_session_cache shared:MozSSL:10m; ssl_session_tickets off;
server { listen 443 ssl; http2 on; server_name grafana.yourdomain.com; # Write the domain, or subdomain for external access to grafana
ssl_certificate "/etc/nginx/ssl/yourdomain.com/fullchain.pem"; # Path to the certificate ssl_certificate_key "/etc/nginx/ssl/yourdomain.com/privkey.pem"; # Path to the private key ssl_trusted_certificate "/etc/nginx/ssl/yourdomain.com/fullchain.pem"; # Path to the certificate
location /grafana { proxy_pass http://grafana:3000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }Instead of grafana-nginx we use a Caddy container: it obtains the domain certificate on its own, and port 80 must be free.
services: vmsingle: image: victoriametrics/victoria-metrics:v1.153.0 container_name: vmsingle hostname: vmsingle restart: unless-stopped volumes: - ./victoria-metrics-data:/victoria-metrics-data command: - '--storageDataPath=/victoria-metrics-data' networks: - grafana-network ports: - IP:8428:8428 # Here we specify the NetBird address (in this case the Grafana server)
grafana: image: grafana/grafana:13.2.3 container_name: grafana hostname: grafana restart: unless-stopped volumes: - grafana-data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning environment: - GF_SERVER_DOMAIN=yourdomain.com - GF_SERVER_ROOT_URL=https://grafana.yourdomain.com/grafana - GF_SERVER_SERVE_FROM_SUB_PATH=true - GF_SERVER_HTTP_PORT=3000 - GF_SERVER_PROTOCOL=http - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=admin - GF_ANALYTICS_REPORTING_ENABLED=false networks: - grafana-network
xray-checker: image: kutovoys/xray-checker:v1.3.1 container_name: xray-checker hostname: xraychecker restart: unless-stopped environment: - "SUBSCRIPTION_URL=https://subscription_link" - "PROXY_STATUS_CHECK_URL=http://google.com/generate_204" - "PROXY_CHECK_INTERVAL=60" ports: - 127.0.0.1:2112:2112 networks: - grafana-network
grafana-caddy: image: caddy:2.11.6 container_name: grafana-caddy restart: unless-stopped ports: - 80:80 - 443:443 volumes: - ./Caddyfile:/etc/caddy/Caddyfile:ro - caddy_data:/data networks: - grafana-network
volumes: grafana-data: caddy_data:
networks: grafana-network: name: grafana-network driver: bridge external: falseCreate the Caddyfile next to the compose:
nano /opt/monitoring/Caddyfilegrafana.yourdomain.com { reverse_proxy /grafana* grafana:3000}Run docker-compose:
docker compose up -d && docker compose logs -f -tInstallation and configuration of the Grafana server
Section titled “Installation and configuration of the Grafana server”-
Installation of cadvisor, node_exporter, vmagent:
root@server: ~ cd /opt/monitoring/cadvisor && wget https://github.com/google/cadvisor/releases/download/v0.60.6/cadvisor-v0.60.6-linux-amd64 && mv cadvisor-v0.60.6-linux-amd64 cadvisor && chmod +x cadvisor && cd /opt/monitoring/nodeexporter && wget https://github.com/prometheus/node_exporter/releases/download/v1.12.1/node_exporter-1.12.1.linux-amd64.tar.gz && tar -xvf node_exporter-1.12.1.linux-amd64.tar.gz && cd node_exporter-1.12.1.linux-amd64 && mv node_exporter /opt/monitoring/nodeexporter && cd .. && chmod +x node_exporter && rm -f node_exporter-1.12.1.linux-amd64.tar.gz && rm -r node_exporter-1.12.1.linux-amd64 && cd /opt/monitoring/vmagent && wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.153.0/vmutils-linux-amd64-v1.153.0.tar.gz && tar -xvf vmutils-linux-amd64-v1.153.0.tar.gz && mv vmagent-prod vmagent && find . ! -name 'vmagent' -type f -delete && chmod +x vmagent && cd -
Create vmagent configuration file:
root@server: ~ nano /opt/monitoring/vmagent/scrape.ymlInsert the following content:
scrape.yml scrape_config_files:- "/opt/monitoring/vmagent/conf.d/*.yml"global:scrape_interval: 15s -
Create cadvisor configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/cadvisor.ymlInsert the following content:
cadvisor.yml - job_name: integrations/cAdvisorscrape_interval: 15sstatic_configs:- targets: ['localhost:9101']labels:instance: "your_instance_name(server Grafana)" -
Create node_exporter configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/nodeexporter.ymlInsert the following content:
nodeexporter.yml - job_name: integrations/node_exporterscrape_interval: 15sstatic_configs:- targets: ['localhost:9100']labels:instance: "your_instance_name(server Grafana)" -
Create xray checker configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/xraychecker.ymlInsert the following content:
xraychecker.yml - job_name: xray_checkerscrape_interval: 15sstatic_configs:- targets: ['127.0.0.1:2112']labels:instance: "your_instance_name(server Grafana)" -
Create cadvisor service file:
root@server: ~ nano /etc/systemd/system/cadvisor.serviceInsert the following content:
cadvisor.service [Unit]Description=cAdvisorWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/cadvisor/cadvisor \-listen_ip=127.0.0.1 \-logtostderr \-port=9101 \-docker_only=trueRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create node_exporter service file:
root@server: ~ nano /etc/systemd/system/nodeexporter.serviceInsert the following content:
nodeexporter.service [Unit]Description=Node ExporterWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/nodeexporter/node_exporter --web.listen-address=127.0.0.1:9100Restart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create vmagent service file:
root@server: ~ nano /etc/systemd/system/vmagent.serviceInsert the following content:
vmagent.service [Unit]Description=VictoriaMetrics AgentWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/vmagent/vmagent \-httpListenAddr=127.0.0.1:8429 \-promscrape.config=/opt/monitoring/vmagent/scrape.yml \-promscrape.configCheckInterval=60s \-remoteWrite.url=http://IP:8428/api/v1/writeRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Start services and add them to autostart:
root@server: ~ systemctl daemon-reload && systemctl enable cadvisor nodeexporter vmagent && systemctl start cadvisor nodeexporter vmagent -
Check service status:
root@server: ~ systemctl status cadvisorsystemctl status nodeexportersystemctl status vmagent
Installation and configuration of the server with the Remnawave panel
Section titled “Installation and configuration of the server with the Remnawave panel”-
Create directories and install cadvisor, node_exporter, vmagent:
root@server: ~ mkdir -p /opt/monitoring/{cadvisor,nodeexporter,vmagent/conf.d} && cd /opt/monitoring/cadvisor && wget https://github.com/google/cadvisor/releases/download/v0.60.6/cadvisor-v0.60.6-linux-amd64 && mv cadvisor-v0.60.6-linux-amd64 cadvisor && chmod +x cadvisor && cd /opt/monitoring/nodeexporter && wget https://github.com/prometheus/node_exporter/releases/download/v1.12.1/node_exporter-1.12.1.linux-amd64.tar.gz && tar -xvf node_exporter-1.12.1.linux-amd64.tar.gz && cd node_exporter-1.12.1.linux-amd64 && mv node_exporter /opt/monitoring/nodeexporter && cd .. && chmod +x node_exporter && rm -f node_exporter-1.12.1.linux-amd64.tar.gz && rm -r node_exporter-1.12.1.linux-amd64 && cd /opt/monitoring/vmagent && wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.153.0/vmutils-linux-amd64-v1.153.0.tar.gz && tar -xvf vmutils-linux-amd64-v1.153.0.tar.gz && mv vmagent-prod vmagent && find . ! -name 'vmagent' -type f -delete && chmod +x vmagent && cd -
Create vmagent configuration file:
root@server: ~ nano /opt/monitoring/vmagent/scrape.ymlInsert the following content:
scrape.yml scrape_config_files:- "/opt/monitoring/vmagent/conf.d/*.yml"global:scrape_interval: 15s -
Create cadvisor configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/cadvisor.ymlInsert the following content:
cadvisor.yml - job_name: integrations/cAdvisorscrape_interval: 15sstatic_configs:- targets: ['localhost:9101']labels:instance: "your_instance_name(server Remnawave panel)" -
Create node_exporter configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/nodeexporter.ymlInsert the following content:
nodeexporter.yml - job_name: integrations/node_exporterscrape_interval: 15sstatic_configs:- targets: ['localhost:9100']labels:instance: "your_instance_name(server Remnawave panel)" -
Create remnawave panel configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/remnawave.ymlInsert the following content:
Example
.env ### PROMETHEUS ###### Metrics are available at /api/metricsMETRICS_USER=gjlToiPHMETRICS_PASS=VsCNutHFremnawave.yml - job_name: remnawavescrape_interval: 15sstatic_configs:- targets: ['127.0.0.1:3001']labels:instance: "your_instance_name(server Remnawave panel)"basic_auth:username: "xxxxxxx"password: "xxxxxxx" -
Forward the metrics port in
docker-compose.yml:root@server: ~ nano /opt/remnawave/docker-compose.ymlFind the following content and add:
docker-compose.yml remnawave:image: remnawave/backend:latestcontainer_name: remnawavehostname: remnawaverestart: alwaysenv_file:- .envports:- '127.0.0.1:3000:3000'- '127.0.0.1:3001:3001' # Metrics portnetworks:- remnawave-networkdepends_on:remnawave-db:condition: service_healthyremnawave-redis:condition: service_healthylogging:driver: 'json-file'options:max-size: '30m'max-file: '5' -
Restart docker-compose:
root@server: ~ cd /opt/remnawave && docker compose down && docker compose up -d && docker compose logs -f -t -
Create cadvisor service file:
root@server: ~ nano /etc/systemd/system/cadvisor.serviceInsert the following content:
cadvisor.service [Unit]Description=cAdvisorWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/cadvisor/cadvisor \-listen_ip=127.0.0.1 \-logtostderr \-port=9101 \-docker_only=trueRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create node_exporter service file:
root@server: ~ nano /etc/systemd/system/nodeexporter.serviceInsert the following content:
nodeexporter.service [Unit]Description=Node ExporterWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/nodeexporter/node_exporter --web.listen-address=127.0.0.1:9100Restart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create vmagent service file:
root@server: ~ nano /etc/systemd/system/vmagent.serviceInsert the following content:
vmagent.service [Unit]Description=VictoriaMetrics AgentWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/vmagent/vmagent \-httpListenAddr=127.0.0.1:8429 \-promscrape.config=/opt/monitoring/vmagent/scrape.yml \-promscrape.configCheckInterval=60s \-remoteWrite.url=http://IP:8428/api/v1/writeRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Start services and add them to autostart:
root@server: ~ systemctl daemon-reload && systemctl enable cadvisor nodeexporter vmagent && systemctl start cadvisor nodeexporter vmagent -
Check service status:
root@server: ~ systemctl status cadvisorsystemctl status nodeexportersystemctl status vmagent
Installation and configuration of the server with the Remnawave node
Section titled “Installation and configuration of the server with the Remnawave node”-
Create directories and install cadvisor, node_exporter, vmagent:
root@server: ~ mkdir -p /opt/monitoring/{cadvisor,nodeexporter,vmagent/conf.d} && cd /opt/monitoring/cadvisor && wget https://github.com/google/cadvisor/releases/download/v0.60.6/cadvisor-v0.60.6-linux-amd64 && mv cadvisor-v0.60.6-linux-amd64 cadvisor && chmod +x cadvisor && cd /opt/monitoring/nodeexporter && wget https://github.com/prometheus/node_exporter/releases/download/v1.12.1/node_exporter-1.12.1.linux-amd64.tar.gz && tar -xvf node_exporter-1.12.1.linux-amd64.tar.gz && cd node_exporter-1.12.1.linux-amd64 && mv node_exporter /opt/monitoring/nodeexporter && cd .. && chmod +x node_exporter && rm -f node_exporter-1.12.1.linux-amd64.tar.gz && rm -r node_exporter-1.12.1.linux-amd64 && cd /opt/monitoring/vmagent && wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.153.0/vmutils-linux-amd64-v1.153.0.tar.gz && tar -xvf vmutils-linux-amd64-v1.153.0.tar.gz && mv vmagent-prod vmagent && find . ! -name 'vmagent' -type f -delete && chmod +x vmagent && cd -
Create vmagent configuration file:
root@server: ~ nano /opt/monitoring/vmagent/scrape.ymlInsert the following content:
scrape.yml scrape_config_files:- "/opt/monitoring/vmagent/conf.d/*.yml"global:scrape_interval: 15s -
Create cadvisor configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/cadvisor.ymlInsert the following content:
cadvisor.yml - job_name: integrations/cAdvisorscrape_interval: 15sstatic_configs:- targets: ['localhost:9101']labels:instance: "your_instance_name(server Remnawave node)" -
Create node_exporter configuration file:
root@server: ~ nano /opt/monitoring/vmagent/conf.d/nodeexporter.ymlInsert the following content:
nodeexporter.yml - job_name: integrations/node_exporterscrape_interval: 15sstatic_configs:- targets: ['localhost:9100']labels:instance: "your_instance_name(server Remnawave node)" -
Create cadvisor service file:
root@server: ~ nano /etc/systemd/system/cadvisor.serviceInsert the following content:
cadvisor.service [Unit]Description=cAdvisorWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/cadvisor/cadvisor \-listen_ip=127.0.0.1 \-logtostderr \-port=9101 \-docker_only=trueRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create node_exporter service file:
root@server: ~ nano /etc/systemd/system/nodeexporter.serviceInsert the following content:
nodeexporter.service [Unit]Description=Node ExporterWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/nodeexporter/node_exporter --web.listen-address=127.0.0.1:9100Restart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Create vmagent service file:
root@server: ~ nano /etc/systemd/system/vmagent.serviceInsert the following content:
vmagent.service [Unit]Description=VictoriaMetrics AgentWants=network-online.targetAfter=network-online.target[Service]User=rootGroup=rootType=simpleExecStart=/opt/monitoring/vmagent/vmagent \-httpListenAddr=127.0.0.1:8429 \-promscrape.config=/opt/monitoring/vmagent/scrape.yml \-promscrape.configCheckInterval=60s \-remoteWrite.url=http://IP:8428/api/v1/writeRestart=alwaysRestartSec=5[Install]WantedBy=multi-user.target -
Start services and add them to autostart:
root@server: ~ systemctl daemon-reload && systemctl enable cadvisor nodeexporter vmagent && systemctl start cadvisor nodeexporter vmagent -
Check service status:
root@server: ~ systemctl status cadvisorsystemctl status nodeexportersystemctl status vmagent
Configure Grafana Dashboards
Section titled “Configure Grafana Dashboards”-
Go to Grafana by the address you specified in
nginx.confGrafana URL Example https://grafana.yourdomain.com/grafana -
We meet the authorization page, enter
adminandadmin, then change the password -
We get to the main menu of Grafana

-
Go to the Connections –> Data Sources section
-
Click the Add data source button and select prometheus
-
In the Connection section, in the
URLfield, enterhttp://vmsingle:8428
-
Click the Save & Test button at the bottom of the page and check that the connection is established
Adding new Dashboard
Section titled “Adding new Dashboard”List of Dashboard:
- Remnawave Dashboard by kastov
- cAdvisor Dashboard by grafana official
- Node Overview by Kutovoys
- Network System by Kutovoys
- Memory System by Kutovoys
- Filesystem Disks by Kutovoys
- CPU System by Kutovoys
-
Go to the Dashboards section
-
Click the New button on the right and select Import
-
Load our Dashboard
-
Where victoriametrics select our prometheus

-
Click the Import button, this template will be loaded
