Is the CUE language the future of the app delivery ecosystem? Maybe let’s understand it first

Is the CUE language the future of the app delivery ecosystem? Maybe let’s understand it first

If you’ve had an attentive eye on your favourite container tech person’s social media account over the last year, chances are you stumbled upon a mention of the CUE language at least once in 2021. From CNCF Toc chair Liz Rice and Kelsey Hightower, to Prometheus co-founder Julius Volz, everyone seems to have been at least somewhat intrigued by the data validation language. 

Docker co-founder Solomon Hykes even went as far as stating that “the entire app delivery ecosystem (CI, CD, cfg mgmt, infra mgmt, paas, observability) will reinvent itself around CUE” in December. So will CUE be on every infra team’s mind by the end of the year? We’ll see.

But chances are you’ll at least keep hearing about it some more, which is why a closer look seems to be in order.

The fact that most of the supporting voices for CUE stem from the world of Kubernetes and adjacent projects might be down to the project’s creator Marcel van Lohuizen. As a member of the initial team that created Kubernetes predecessor Borg, van Lohuizen was responsible for the development of BCL and borgcfg — the main way to interface with Borg.

While BCL went on to be generalised into the Google Config Language GCL, van Lohuizen moved into the Go team, where he started to think about ways to improve configuration. His background in natural language processing and “15 years of hindsight” led the way, and consequently to van Lohuizen developing CUE.

Cue CUE, or, Why yet another config project?

According to its creator, CUE is an associative, commutative and idempotent logic programming language that promises to be useful for configuration, declarative scripting, data validation and templating, policy checking, and the generation of code and tests. However, it’s not all the CUE project is made of, as a logic programming engine and a series of APIs including frameworks for describing workflows and the like, various tools, and adaptors for JSON, YAML, Protobuf, and JSON Schema are also part of the package.

When coming up with the project, van Lohuizen was guided by some dos and don’ts for choosing a configuration language. He found that having more than one level of inheritance made it hard to see where values were coming from, which tended to cause more complexity and problems when writing config language tooling. As a result, CUE stays away from override and file-overlay-style inheritance (which can be found in kustomize, for example).

Parameterization and non-trivial computations are other don’ts van Lohuizen identified. The problem with parameterization to him is that abstracting away from an underlying API leads to users wanting access to all the API’s features at some point, making the abstraction as complex as the API. Too much non-trivial computation in a configuration language would make it too hard to understand, which is against the goal of writing readable configs.

New, yet familiar — the CUE DSL

With that in mind, CUE’s design focuses on automation and being declarative, while offering detailed type checking. From a high-level point of view, the result can be seen as a “text-based spreadsheet for JSON-like trees of data” on top of “a logically sound data reasoning engine”. (The comparison to a spreadsheet is down to data trees allowing users to specify values in relation to other parts of the tree instead of having concrete values at all end nodes.)

The CUE DSL is meant to operate on all levels of configuration and give users the ability to specify data and types in a superset of JSON. Types are values in CUE, so the syntax is the same no matter if data or types are declared and allows for ordering everything in one value-lattice-formed hierarchy. Value lattices are partially ordered sets in which “every two elements have a unique least upper bound and greatest lower bound”, with the latter allowing for unambiguous merges of configs, independent of order.

To make sure the DSL adheres to the rules van Lohuizen formulated for CUE, it is constraint-based, aspect-oriented, and composable. Aspect orientation here is meant to ensure that the DSL doesn’t have to use inheritance, while composability makes things like validation and combination of values from different sources feasible since it again means the order of input values doesn’t matter. 

Of course, CUE isn’t the first language in the configuration space (looking at you, HCL), and especially the similarity to JSON might make potential users wonder why they shouldn’t just go with Jsonnet instead. Both indeed help to reduce complexity, which is why CUE tries to score by adding typing and the promise of ease of use in large scale setups on top of that.

State of things and next steps

Currently, the CUE team looks rather active in its endeavours to get v0.4.1 out of the door. This is a good sign for those taking the prolonged release break after the migration of the project into a neutral home as an omen of its demise. In an online meeting last autumn, van Lohuizen recognised the need for stability and a v1.0 to get adoption going, but the team isn’t quite there yet.

Before making the jump, not only does the language need to mature a little, there are also aspects such as modules and package management, documentation, performance and support for other encodings (XML and HCL for instance) that have to be moved forward first. Having van Lohuizen on the project full-time will surely help in that respect.

Though CUE is still at the beginning of its journey, the thought of an easy way of writing and validating very complex configurations has already led to interest at companies such as Grafana and Salesforce. Docker creator Solomon Hykes even made it an integral part of his new endeavour Dagger, a kit for building CI/CD pipelines, which might explain his full confidence in CUE’s future.

According to Hykes, deployments are one of the biggest pains at the moment, as application development teams have more and more disparate components to choose from. This makes for pretty custom constructs which lead to deployments being “an exercise in gluing things together” — mostly via scripts that tend to be slow to adapt to infrastructure changes. 

Dagger looks to provide an alternative to these scripts with a programming environment that uses delivery graphs to describe how the different components work together and is more flexible. The project builds on compute engine Buildkit and the CUE language, as they provide ways to write and work with graphs in a way that shell scripts, Python, and YAML don’t. Amongst other things, CUE helps here to write configurations, tie existing ones together via its package model, and catch configuration errors early.