Prometheus gets conformance program to certify compatibility

Rust release

The Prometheus project team will bring in the Prometheus Conformance Program with the aim of certifying compatibility between the open source monitoring tool and other software.

Announced on the Prometheus blog, the aim of the new program is to ensure interoperability, to protect users from potentially unwanted surprises, and to enable more parallel innovation in Prometheus.

With the Prometheus project’s already extensive and growing test suite, projects and vendors will be able to determine the level of compliance to Prometheus’ specifications and compatibility within the overall Prometheus ecosystem, according to the Prometheus team.

The intention is that for every component there will be a mark such as “OpenMetrics 2021-05 compliant” or “PromQL 2021-05 compliant”, and any project or vendor will be able to submit their compliance documentation. Upon reaching 100%, the mark will be granted.

For entire software packages, there will be a mark indicating compatibility with specific versions, such as “Prometheus 2.26 compatible”. Relevant component compliance scores are multiplied, and upon reaching 100 %, the mark is given.

As an example, the Prometheus Agent supports both OpenMetrics and Prometheus Remote Write, but not PromQL. As such, only compliance scores for OpenMetrics and Prometheus Remote Write are multiplied.

Both compliant and compatible marks are valid for two minor releases or 12 weeks, whichever is longer. This should allow all projects and vendors enough time to update while being current and useful for users, according to the Prometheus team.

At launch, compliance tests for three components are available, although these are not in a fully complete state. These are PromQL, Remote Read-Write, and OpenMetrics. More tests are planned.

The Prometheus Conformance Program is being introduced with the help of the Cloud Native Computing Foundation (CNCF). The CNCF Governing Board is expected to formally review and approve the program during its next meeting. Meanwhile, the Prometheus team is inviting the wider community to help improve the tests during this ramp-up phase.