Prometheus for logging, metering, profiling and tracing

As part of the training, we will learn how to work with monitoring tools based on the properties of the Prometheus tool, from the configuration of data collection on measured systems, through the transport layer to use and storage in a data repository. Prometheus for metric data processing, Loki for log data processing, Traps for profiling data processing, Tempo for trace data processing, Alermanager for alert processing and Grafana for data visualization.

Furthermore, we will focus on the possibilities of data evaluation and, if the input conditions are met, their subsequent processing using a communication interface, automatic processing by an orchestra tool, or analysis in expert systems. We will learn how to format queries into metric and log databases, their use in defining visualizations and notifications.

During the training, we will show sample examples of data collection settings on a real infrastructure and show how to access stored data, which will provide you with a good basis for further use of modern monitoring systems based on the Prometheus family of tools.

Audience

• DevOps engineers
• T system administrators
• IT systems architects
  
  

Goals

The training is divided into 2 days as follows:

1. Metering and profiling
2. Logging and tracing

Outline

1st day

Introduction to monitoring

• Distribution of monitoring by type
• Data types and working with data
• SRE monitoring

Installation and configuration of systems

• Prometheus for metric data collection
• Trap for collecting profile data

Autodiscovery of data collection services

• Cloud services
• Container services
• Network devices
• Observability and applications

Examples of collection configuration

• Collection of metric data using exporters
• Collection of profiling data using the Trap agent

Transformations and data queries

• Metric query formats
• Advanced PromQL language features

Monitoring-as-a-code

• Mixing monitoring
• Definition of alerts and rules
• Definition of dashboards

2nd day

Installation and configuration of systems

• Loki for log data collection
• Pace for trace data collection

Examples of collection configuration

• Log data collection using OpenSource tools
• Collection of trace data using OpenSource tools

Transformations and log queries

• Log query formats
• Practicing log queries
• Advanced LogQL language features

Definition of alerts and data processing

• Thresholds, histeresis
• Root causes and dependencies
• Error budgets according to SRE
• Statistical analysis and expert systems

Prerequisites

• Basics of Linux / Windows platform administration
• Experience with public clouds and services

Technical requirements (BYOD)

• Computer with any OS (Linux, Windows, OS X)
• SSH client
• Web browser