Call my Agent: New mode lets users turn Prometheus 2.32 into write-only scraper

Prometheus 2.32

The team behind monitoring system Prometheus has pushed version 2.32 out the door, using its last release of the year to fix TSDB bugs, add arm64 support for Windows, and introduce users to a new mode of operations.

As shared in a blog post in November, Prometheus 2.32 is the first release to include a new mode called Prometheus Agent. Once enabled via --enable-feature=agent, it disables some of the project’s features and allows Prometheus to serve as a remote write-only scraper and forwarder. Like this, it doesn’t need local storage and isn’t locally queryable, which is beneficial for resource restricted edge use cases and horizontal scalability for data ingestion amongst other things. 

New mode aside, Prometheus 2.23 comes with a reduced default maximal retry time of five seconds to prevent DDoS-like behaviour for some remote-write calls, and a search option in the metrics explorer drop down menu. It also lets admins use int and uint as datatypes for template formatting, so that humanize functions should no longer fail.

The latest iteration of Prometheus’s command line tool Promtool has been fitted with a command to check service discovery, some reworked under-the-hood computations to improve numerical stability, and easier to read test output. 

Time series database component TSDB meanwhile learned to work with arm64 architectures under Windows, and skips unneeded sorting to make certain queries a bit more efficient. Users should also see TSDB encountering less panics during WAL replay and in cases where the checkpoint directory is empty, and get notified about problems when writing individual index sections earlier thanks to additional tests.

The Prometheus update landed just a couple of days after data visualisation project Grafana, which is often used together with Prometheus, informed its user base about a high severity (CVSS 7.5) vulnerability in its software. 

The issue, which got assigned CVE-2021-43798, left Grafana versions 8.0.0-beta1 through 8.3.0 vulnerable to directory traversal attacks, that could be used to access local files. It has been mitigated in newly released versions 8.0.7, 8.1.8, 8.2.7, or 8.3.1, so users are advised to update their systems accordingly. 

Grafana Agent, the implementation of which was ported to the Prometheus codebase as a basis for the just mentioned Prometheus Agent, also got an update following the disclosure. The fixes in versions 0.20.1 and 0.21.2 are hoped to protect users from CVE-2021-41090, another CVSS 7.5 bug which could be used to access inline secrets defined within a metrics instance config.