Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Prometheus and Grafana

•Download as PPTX, PDF•
•
Lhouceine OUHAMZA
Lhouceine OUHAMZA

Prometheus is an open-source monitoring system that collects metrics from configured targets, stores time-series data, and allows users to query and visualize the data. It works by scraping metrics over HTTP from applications and servers, storing the data in its time-series database, and providing a UI and query language to analyze the data. Prometheus is useful for monitoring system metrics like CPU usage and memory as well as application metrics like HTTP requests and errors.

1 of 26
1 By OUHAMZA Lhouceine 2020 inge.lhouceine.ouhamza@gmail.com
2 Monitoring is one of the fundamental tasks in any system. It can help us to detect problems and take action, or simply to know the current state of our systems. ➔ helps you reduce (or even eliminate) downtime. ➔ You can detect problems, and solve them before they impact users Why monitoring?
3 We can: ➔ Expose Important metrics (data) over time ➔ Provide a way to configure rules, e.g (Cpu usage /80%) ➔ Send alerts (email) when rules are broken ➔ Provide visualization/Graphing Monitoring Monitoring tools: Sensu, Prometheus, Nagios
4 Prometheus is an in-memory dimensional time series database with a simple built-in UI, a custom query language. Prometheus is designed to scrape metrics from application instances periodically based on service discovery. ➔ A data scraper that pulls metrics data over HTTP periodically at a configured interval. ➔ A time-series database to store all the metrics data. ➔ A simple user interface where you can visualize, query, and monitor all the metrics. ➔ Written in Go, fully published in 2015. ➔ Second project hosted by SNCF after kubernetes Prometheus
5 How prometheus works ?
6 What does prometheus monitor? ➔ Linux / windows servers ( CPU status, memory, Disk space …) ➔ Apache server (On/Off status…) ➔ Single application ( Exceptions, Requests count, client latency…) ➔ Service like database (status, Query Performance…)
7 Prometheus
8 Metrics Metrics: all data as time series that prometheus can scrape it and stores it in his database. Metrics has two metadata lines: Help and Type Help: description of what the metrics is. Type: we have 3 metrics types: ➔ Count: how many times x happened? ➔ Gauge: what is the current value of x now? ➔ Histogram; How long or how big? OpenMetrics format
9 Some examples of metrics Current memory usage gauge HTTP request latency gauge CPU usage gauge Number of threads in use count Number of HTTP requests count
10 PromQL Prometheus provides a functional query language called PromQL (Prometheus Query Language) that lets the user select and aggregate time series data in real time. The result of an expression can either be shown as a graph, viewed as tabular data in Prometheus's expression browser, or consumed by external systems via the HTTP API. Example: Metric name Metric label
11 PromQL Example Queries: ➔ Http_requests_total{status!~”4..”} : Query all HTTP status codes except 4xx ones. ➔ rate(Http_requests_total[5m])[30m:] : Returns the 5-minutes rate of the http_requests_total metric for the past 30 min.
12 Target Endpoints and Exporters In order to monitor anything with Prometheus, you need an exporter – a process that exposes metrics for Prometheus to harvest. The available exporters can be find from here: https://prometheus.io/docs/instrumenting/exporters/
13 Monitoring your own application ➔ How many requests? ➔ How many exceptions? ➔ How many server resources are used? ➔ Example: micrometer pour spring boot Also you can use client libraries to expose metrics endpoints
14 Configuring prometheus ➔ How often Prometheus will scrape its targets ➔ Rules for aggregating metric values or creating alerts when condition met ➔ What resources Prometheus monitors prometheus.yml
15 AlertManager is a single binary which handles alerts sent by Prometheus server and notifies end user through E-mail, Slack or other tools. Example (custom rules): ➔ Is my server not running ➔ Average response time greater than 500Ms ➔ Is my server’s memory usage greater than 95%. ➔ Number of 404 errors greater than 10% of all requests. AlertManager
16 ➔ Is the open source analytics & monitoring solution for every database. ➔ Grafana allows you to bring data from various data sources like Elasticsearch, Prometheus, Graphite, InfluxDB etc, and visualize them with beautiful graphs. Grafana
17 ➔ Create graphs easily ➔ Visualize data in differents ways ➔ Show graphs for different time periods ➔ Create dashboard to be viewed by others ➔ Allows us to easily make changes Grafana features
18 Demo
19 We will monitor all this targets using prometheus and grafana
20 ➔ Have you ever wondered how you can monitor your entire Linux system performance easily? ➔ If you are looking for all those features, you are in the right place. ➔ Today we are taking a very special look at the Node Exporter, a Prometheus exporter specialized in exposing Linux metrics. Node Exporter
21 ➔ Spring Boot uses Micrometer, an application metrics facade to integrate actuator metrics with external monitoring systems. ➔ Micrometer supports several monitoring systems like Netflix Atlas, AWS Cloudwatch, Datadog, InfluxData, SignalFx, Graphite, Wavefront, Prometheus etc. ➔ You should use actuator ➔ To integrate actuator with Prometheus, you need to add the micrometer- registry-prometheus dependency. Spring boot Application
22 Postgres / Sql Server Databases ➔ We want to alert the average response time ➔ Viewing active sessions ➔ Shared buffers ➔ Transactions ➔ Number of active connection ➔ Percentage of memory utilization
23 Nginx ➔ NGINX Prometheus exporter fetches the metrics from a single NGINX, converts the metrics into appropriate Prometheus metrics types and finally exposes them via an HTTP server to be collected by Prometheus. ➔ For NGINX, the following metrics are exported: ● Connections. ● HTTP.
24 Kubernetes cluster Helm chart is the best way to deploy prometheus in k8s cluster. It create all configuration yml files in one command. helm install --name prometheus stable/prometheus-operator kubectl get all
25 Conclusion Prometheus is: ➔ Full Java Client ➔ Simple handling ➔ Supported by grafana ( Graphs ) ➔ Full monitoring thanks to numerous exporters and libraries ➔ Service discovery e.g (Consul) ➔ Alerting with AlertManager
26

More Related Content

Prometheus and Grafana

  • 2. 2 Monitoring is one of the fundamental tasks in any system. It can help us to detect problems and take action, or simply to know the current state of our systems. ➔ helps you reduce (or even eliminate) downtime. ➔ You can detect problems, and solve them before they impact users Why monitoring?
  • 3. 3 We can: ➔ Expose Important metrics (data) over time ➔ Provide a way to configure rules, e.g (Cpu usage /80%) ➔ Send alerts (email) when rules are broken ➔ Provide visualization/Graphing Monitoring Monitoring tools: Sensu, Prometheus, Nagios
  • 4. 4 Prometheus is an in-memory dimensional time series database with a simple built-in UI, a custom query language. Prometheus is designed to scrape metrics from application instances periodically based on service discovery. ➔ A data scraper that pulls metrics data over HTTP periodically at a configured interval. ➔ A time-series database to store all the metrics data. ➔ A simple user interface where you can visualize, query, and monitor all the metrics. ➔ Written in Go, fully published in 2015. ➔ Second project hosted by SNCF after kubernetes Prometheus
  • 6. 6 What does prometheus monitor? ➔ Linux / windows servers ( CPU status, memory, Disk space …) ➔ Apache server (On/Off status…) ➔ Single application ( Exceptions, Requests count, client latency…) ➔ Service like database (status, Query Performance…)
  • 8. 8 Metrics Metrics: all data as time series that prometheus can scrape it and stores it in his database. Metrics has two metadata lines: Help and Type Help: description of what the metrics is. Type: we have 3 metrics types: ➔ Count: how many times x happened? ➔ Gauge: what is the current value of x now? ➔ Histogram; How long or how big? OpenMetrics format
  • 9. 9 Some examples of metrics Current memory usage gauge HTTP request latency gauge CPU usage gauge Number of threads in use count Number of HTTP requests count
  • 10. 10 PromQL Prometheus provides a functional query language called PromQL (Prometheus Query Language) that lets the user select and aggregate time series data in real time. The result of an expression can either be shown as a graph, viewed as tabular data in Prometheus's expression browser, or consumed by external systems via the HTTP API. Example: Metric name Metric label
  • 11. 11 PromQL Example Queries: ➔ Http_requests_total{status!~”4..”} : Query all HTTP status codes except 4xx ones. ➔ rate(Http_requests_total[5m])[30m:] : Returns the 5-minutes rate of the http_requests_total metric for the past 30 min.
  • 12. 12 Target Endpoints and Exporters In order to monitor anything with Prometheus, you need an exporter – a process that exposes metrics for Prometheus to harvest. The available exporters can be find from here: https://prometheus.io/docs/instrumenting/exporters/
  • 13. 13 Monitoring your own application ➔ How many requests? ➔ How many exceptions? ➔ How many server resources are used? ➔ Example: micrometer pour spring boot Also you can use client libraries to expose metrics endpoints
  • 14. 14 Configuring prometheus ➔ How often Prometheus will scrape its targets ➔ Rules for aggregating metric values or creating alerts when condition met ➔ What resources Prometheus monitors prometheus.yml
  • 15. 15 AlertManager is a single binary which handles alerts sent by Prometheus server and notifies end user through E-mail, Slack or other tools. Example (custom rules): ➔ Is my server not running ➔ Average response time greater than 500Ms ➔ Is my server’s memory usage greater than 95%. ➔ Number of 404 errors greater than 10% of all requests. AlertManager
  • 16. 16 ➔ Is the open source analytics & monitoring solution for every database. ➔ Grafana allows you to bring data from various data sources like Elasticsearch, Prometheus, Graphite, InfluxDB etc, and visualize them with beautiful graphs. Grafana
  • 17. 17 ➔ Create graphs easily ➔ Visualize data in differents ways ➔ Show graphs for different time periods ➔ Create dashboard to be viewed by others ➔ Allows us to easily make changes Grafana features
  • 19. 19 We will monitor all this targets using prometheus and grafana
  • 20. 20 ➔ Have you ever wondered how you can monitor your entire Linux system performance easily? ➔ If you are looking for all those features, you are in the right place. ➔ Today we are taking a very special look at the Node Exporter, a Prometheus exporter specialized in exposing Linux metrics. Node Exporter
  • 21. 21 ➔ Spring Boot uses Micrometer, an application metrics facade to integrate actuator metrics with external monitoring systems. ➔ Micrometer supports several monitoring systems like Netflix Atlas, AWS Cloudwatch, Datadog, InfluxData, SignalFx, Graphite, Wavefront, Prometheus etc. ➔ You should use actuator ➔ To integrate actuator with Prometheus, you need to add the micrometer- registry-prometheus dependency. Spring boot Application
  • 22. 22 Postgres / Sql Server Databases ➔ We want to alert the average response time ➔ Viewing active sessions ➔ Shared buffers ➔ Transactions ➔ Number of active connection ➔ Percentage of memory utilization
  • 23. 23 Nginx ➔ NGINX Prometheus exporter fetches the metrics from a single NGINX, converts the metrics into appropriate Prometheus metrics types and finally exposes them via an HTTP server to be collected by Prometheus. ➔ For NGINX, the following metrics are exported: ● Connections. ● HTTP.
  • 24. 24 Kubernetes cluster Helm chart is the best way to deploy prometheus in k8s cluster. It create all configuration yml files in one command. helm install --name prometheus stable/prometheus-operator kubectl get all
  • 25. 25 Conclusion Prometheus is: ➔ Full Java Client ➔ Simple handling ➔ Supported by grafana ( Graphs ) ➔ Full monitoring thanks to numerous exporters and libraries ➔ Service discovery e.g (Consul) ➔ Alerting with AlertManager
  • 26. 26

Editor's Notes

  1. Sensu - A flexible and scalable telemetry and service health checking solution for monitoring servers, containers, services, applications, functions, and connected devices. Prometheus - Relies on the pull method to collect information, with a built-in database. Nagios - The legacy monitoring tool that introduced monitoring practices to a generation of operators.
  2. Metrics: Keys values pairs