Monitoring system Prometheus 2.19 gives storage layer a polish

Monitoring system Prometheus 2.19 gives storage layer a polish

Prometheus 2.19 is good to go. The new version picks up where it left off, further improving the monitoring system’s storage layer TSDB amongst other things.

Once updated, the project’s time series database will memory map full chunks of head, which is the part of the layer that handles reads and writes of time series data within a time window, after flushing it to the disk. This will help to free up memory and speed up restarts. Should a head chunk be corrupted, the approach makes sure this doesn’t result in memory loss. It does that by first deleting both the corrupted and the following chunks and recovering the data from the write-ahead log afterwards.

TSDB also received some modifications to the isolation struct that allowed lowWatermark store appenders in doubly linked lists. The change is meant to keep goroutines from getting stuck in cases where many of them try to append data at the same time. An additional fix helps queries to return correct results when remote reads are configured in Prometheus, something which sometimes became an issue in earlier versions.

Other than that, the release is more about smaller enhancements. The Prometheus team for example added a counter for the total number of remote read queries, and time range parameters for label names and label values API. The latter fixes the issue of the head index reader ignoring timestamps passed from Querier mint(start)/maxt(end).

In v2.19, Prometheus also includes an optional configuration item server_type, which can be set to discover global zones. Meanwhile the timespan between the system resending an alert has been increased to prevent issues with AlertManager, since users reported the tool closing and re-opening alerts which were still open in Prometheus for earlier versions.
The release also comes with a number of corrections to the platform’s querying language PromQL, which “eliminate collision while checking for duplicate labels”, “correctly track number samples for a query” and “return NaN when histogram buckets have 0 observations. More details can be found in the project’s repository.