Pushgateway

https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-pushgateway

https://www.metricfire.com/blog/prometheus-pushgateways-everything-you-need-to-know/

https://github.com/prometheus/pushgateway

echo "some_metric 3.14" | curl --data-binary @- https://pushgateway.example/metrics/job/some_job

As node collector

https://github.com/prometheus/node_exporter/issues/279

curl -s http://localhost:9256/metrics | grep -v "\(\(^\| \)go_\|http_request\|http_requests\|http_response\|process_\)" | curl --data-binary @- http://pushgateway.example.com:9091/metrics/job/node/instance/<INSTANCE_NAME>

Using Cron job to push every 30 seconds

sudo nano /usr/local/bin/prompush

#!/bin/bash
set -eu
curl -s http://localhost:9100/metrics | grep -v "\(\(^\| \)go_\|http_request\|http_requests\|http_response\|process_\)" | curl --data-binary @- https://pushgateway.example.com/metrics/job/c1e52efa-af1c-11ee-be01-e78fa8f1ca96-linuxnode/instance/lxd0

sudo chmod +x /usr/local/bin/prompush

/etc/cron.d/pushgateway

* * * * * nobody /usr/local/bin/prompush
* * * * * nobody sleep 30; /usr/local/bin/prompush

Ingress NGINX Annotations

htpasswd -c auth default
kubectl create secret generic pushgateway-ingress-basic-auth --from-file=auth

  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8,192.168.0.0/16"
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: pushgateway-ingress-basic-auth
    # nginx.ingress.kubernetes.io/server-snippet: |
    #   satisfy  any;
    #   allow 66.220.144.0/20;
    #   deny   all;

And update script to use auth

#!/bin/bash
set -eu
basic="default:mysecret"
curl -s http://localhost:9100/metrics | grep -v "\(\(^\| \)go_\|http_request\|http_requests\|http_response\|process_\)" | curl --data-binary @- https://pushgateway.example.com/metrics/job/c1e52efa-af1c-11ee-be01-e78fa8f1ca96-linuxnode/instance/lxd0 -u ${basic}

Windows

Not working

$cred = Get-Credential
$r=(invoke-webrequest http://localhost:9182/metrics).content; $r.tostring() -split "[`r`n]" | select-string -pattern "go_|http_request|http_requests|http_response|process" | ConvertTo-Json | Invoke-WebRequest -Method POST -Uri https://pushgateway.example/metrics/job/windowsnode/instance/foo -cred $cred -ContentType 'application/json'