Busk at 01:23, 23 June 2024

2024-06-23T01:23:06Z

Busk: Created page with "``` Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage,..."

2024-06-23T00:58:55Z

Created page with "``` Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage,..."

New page

```
Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage, and the necessary Thanos components.

Here's an example of a docker-compose.yml configuration:

yaml
Copy code
version: '3.8'

services:
minio:
image: minio/minio
container_name: minio
environment:
MINIO_ROOT_USER: minio
MINIO_ROOT_PASSWORD: minio123
command: server /data
ports:
- "9000:9000"
- "9001:9001"
volumes:
- minio-data:/data

prometheus1:
image: prom/prometheus
container_name: prometheus1
volumes:
- ./prometheus1.yml:/etc/prometheus/prometheus.yml
- prometheus1-data:/prometheus
ports:
- "9090:9090"

prometheus2:
image: prom/prometheus
container_name: prometheus2
volumes:
- ./prometheus2.yml:/etc/prometheus/prometheus.yml
- prometheus2-data:/prometheus
ports:
- "9091:9090"

thanos-store:
image: thanosio/thanos:v0.24.0
container_name: thanos-store
command:
- store
- --objstore.config-file=/etc/thanos/minio.yml
- --http-address=0.0.0.0:10902
ports:
- "10902:10902"
volumes:
- ./minio.yml:/etc/thanos/minio.yml

thanos-sidecar-prometheus1:
image: thanosio/thanos:v0.24.0
container_name: thanos-sidecar-prometheus1
command:
- sidecar
- --prometheus.url=http://prometheus1:9090
- --tsdb.path=/prometheus
- --objstore.config-file=/etc/thanos/minio.yml
volumes:
- prometheus1-data:/prometheus
- ./minio.yml:/etc/thanos/minio.yml

thanos-sidecar-prometheus2:
image: thanosio/thanos:v0.24.0
container_name: thanos-sidecar-prometheus2
command:
- sidecar
- --prometheus.url=http://prometheus2:9090
- --tsdb.path=/prometheus
- --objstore.config-file=/etc/thanos/minio.yml
volumes:
- prometheus2-data:/prometheus
- ./minio.yml:/etc/thanos/minio.yml

thanos-querier:
image: thanosio/thanos:v0.24.0
container_name: thanos-querier
command:
- query
- --http-address=0.0.0.0:10903
- --store=thanos-store:10902
- --store=thanos-sidecar-prometheus1:10901
- --store=thanos-sidecar-prometheus2:10901
ports:
- "10903:10903"

volumes:
minio-data:
prometheus1-data:
prometheus2-data:
You will also need a prometheus1.yml and prometheus2.yml configuration file for your Prometheus instances, as well as a minio.yml for the Thanos object storage configuration.

prometheus1.yml / prometheus2.yml
yaml
Copy code
global:
scrape_interval: 15s

scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
minio.yml
yaml
Copy code
type: S3
config:
bucket: "thanos"
endpoint: "minio:9000"
access_key: "minio"
secret_key: "minio123"
insecure: true
To get started, place the docker-compose.yml, prometheus1.yml, prometheus2.yml, and minio.yml files in the same directory, and then run:

bash
Copy code
docker-compose up -d
This will set up the environment with two Prometheus servers, MinIO for storage, Thanos Store, Thanos Sidecar for each Prometheus, and Thanos Querier for querying the data.

```

← Older revision		Revision as of 01:23, 23 June 2024
Line 1:		Line 1:
		+	# With Pushgateway
		+
		+	```
		+	Ensure your docker-compose.yml is configured correctly, and your Prometheus configuration files (prometheus1.yml and prometheus2.yml) are set to scrape the Pushgateway.
		+
		+	docker-compose.yml (with corrected ports to avoid conflicts)
		+	yaml
		+	Copy code
		+	version: '3.8'
		+
		+	services:
		+	minio:
		+	image: minio/minio
		+	container_name: minio
		+	environment:
		+	MINIO_ROOT_USER: minio
		+	MINIO_ROOT_PASSWORD: minio123
		+	command: server /data
		+	ports:
		+	- "9000:9000"
		+	- "9001:9001"
		+	volumes:
		+	- minio-data:/data
		+
		+	prometheus1:
		+	image: prom/prometheus
		+	container_name: prometheus1
		+	volumes:
		+	- ./prometheus1.yml:/etc/prometheus/prometheus.yml
		+	- prometheus1-data:/prometheus
		+	ports:
		+	- "9090:9090"
		+
		+	prometheus2:
		+	image: prom/prometheus
		+	container_name: prometheus2
		+	volumes:
		+	- ./prometheus2.yml:/etc/prometheus/prometheus.yml
		+	- prometheus2-data:/prometheus
		+	ports:
		+	- "9092:9090" # Changed the external port for prometheus2 to 9092
		+
		+	pushgateway:
		+	image: prom/pushgateway
		+	container_name: pushgateway
		+	ports:
		+	- "9093:9091" # Changed the external port for Pushgateway to 9093
		+
		+	thanos-store:
		+	image: thanosio/thanos:v0.24.0
		+	container_name: thanos-store
		+	command:
		+	- store
		+	- --objstore.config-file=/etc/thanos/minio.yml
		+	- --http-address=0.0.0.0:10902
		+	ports:
		+	- "10902:10902"
		+	volumes:
		+	- ./minio.yml:/etc/thanos/minio.yml
		+
		+	thanos-sidecar-prometheus1:
		+	image: thanosio/thanos:v0.24.0
		+	container_name: thanos-sidecar-prometheus1
		+	command:
		+	- sidecar
		+	- --prometheus.url=http://prometheus1:9090
		+	- --tsdb.path=/prometheus
		+	- --objstore.config-file=/etc/thanos/minio.yml
		+	volumes:
		+	- prometheus1-data:/prometheus
		+	- ./minio.yml:/etc/thanos/minio.yml
		+
		+	thanos-sidecar-prometheus2:
		+	image: thanosio/thanos:v0.24.0
		+	container_name: thanos-sidecar-prometheus2
		+	command:
		+	- sidecar
		+	- --prometheus.url=http://prometheus2:9090
		+	- --tsdb.path=/prometheus
		+	- --objstore.config-file=/etc/thanos/minio.yml
		+	volumes:
		+	- prometheus2-data:/prometheus
		+	- ./minio.yml:/etc/thanos/minio.yml
		+
		+	thanos-querier:
		+	image: thanosio/thanos:v0.24.0
		+	container_name: thanos-querier
		+	command:
		+	- query
		+	- --http-address=0.0.0.0:10903
		+	- --store=thanos-store:10902
		+	- --store=thanos-sidecar-prometheus1:10901
		+	- --store=thanos-sidecar-prometheus2:10901
		+	ports:
		+	- "10903:10903"
		+
		+	volumes:
		+	minio-data:
		+	prometheus1-data:
		+	prometheus2-data:
		+	prometheus1.yml / prometheus2.yml
		+	Update your Prometheus configuration files to include the Pushgateway as a scrape target:
		+
		+	yaml
		+	Copy code
		+	global:
		+	scrape_interval: 15s
		+
		+	scrape_configs:
		+	- job_name: 'prometheus'
		+	static_configs:
		+	- targets: ['localhost:9090']
		+
		+	- job_name: 'pushgateway'
		+	static_configs:
		+	- targets: ['pushgateway:9091']
		+	Pushing Metrics to Pushgateway
		+	Use the following curl command to push metrics to the Pushgateway in the correct format:
		+
		+	sh
		+	Copy code
		+	curl -X POST http://localhost:9093/metrics/job/myjob/instance/myinstance --data-binary @- <<EOF
		+	sample_metric 123.45
		+	EOF
		+	Querying Data from Prometheus
		+	Query the data from Prometheus as follows:
		+
		+	Query the current value of sample_metric:
		+	sh
		+	Copy code
		+	# Query the current value of sample_metric from prometheus1
		+	curl 'http://localhost:9090/api/v1/query?query=sample_metric'
		+
		+	# Query the current value of sample_metric from prometheus2
		+	curl 'http://localhost:9092/api/v1/query?query=sample_metric'
		+	Query data over a time range:
		+	sh
		+	Copy code
		+	# Define end and start times
		+	end_time=$(date +%s)
		+	start_time=$(($end_time - 3600))
		+
		+	# Query the sample_metric over the last hour from prometheus1
		+	curl 'http://localhost:9090/api/v1/query_range' --data-urlencode "query=sample_metric" --data-urlencode "start=$start_time" --data-urlencode "end=$end_time" --data-urlencode "step=15s"
		+
		+	# Query the sample_metric over the last hour from prometheus2
		+	curl 'http://localhost:9092/api/v1/query_range' --data-urlencode "query=sample_metric" --data-urlencode "start=$start_time" --data-urlencode "end=$end_time" --data-urlencode "step=15s"
		+	This setup ensures that there are no port conflicts, metrics are pushed to the Pushgateway in the correct format, and you can query these metrics from Prometheus.
		+	```
		+
		+
		+	# Old no pushgateway
	```		```
	Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage, and the necessary Thanos components.		Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage, and the necessary Thanos components.

Thanos docker-compose - Revision history

Busk at 01:23, 23 June 2024

Busk: Created page with "``` Running multiple Prometheus servers with Thanos and MinIO for object storage. This setup assumes you want to run two Prometheus instances, one MinIO instance for storage,..."