Argo Workflows v3.0 has finally been pushed out this week, following nine release candidates since this version was announced to the world back in January.
This is a major update to the cloud-native workflow engine that adds 20K new lines of code, bringing features such as a new user interface, controller high availability, key-only artifacts to make it easier to perform map-reduce operations, and more.
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs using Kubernetes. It is capable of running tens of thousands of concurrent workflows, each with thousands of steps. The platform counts the CERN research organisation among its users.
Among the major new features in Argo Workflows v3.0 is controller high availability. This introduces a hot standby workflow controller capability by building on the built-in Kubernetes leader election mechanism. A default install enables leader election, with one pod selected as leader. If this controller pod should crash, Kubernetes will restart it. However, to reduce start-up time, users can now run two pods, the second of which will be a hot standby ready to take over instantly if the leader fails.
The new user interface for Argo Workflows v3.0 is claimed to be more robust and reliable. It also supports Argo Events, the Argo Project’s event-driven automation tool that can be used in conjunction with Argo Workflows.
An event-flow page in the UI allows users to understand how event sources and sensors are connected together, according to the Argo Project. It also shows workflows created by triggers and displays an animation whenever a message is seen.
The UI also features widgets that allow users to embed the status and progress of a workflow, or the latest workflow created by a template or Cron workflow. The log viewer has also been updated to help debug artifact issues by enabling users to view init and wait containers.
Argo Workflows v3.0 also introduces a default artifact repository reference and key-only artifacts. This allows users to move configuration of the artifact repository out of the workflow specification, so they can configure a default artifact repository for their namespace rather than having to define it explicitly for each workflow.
The key can be used to specify fan-in artifact patterns, simplifying map-reduce style workflows. It can also be used to reference artifacts globally without using parametrised inputs and outputs.
According to the Argo Project, one consequence is that there is no longer a need to replicate non-key elements in manifests, reducing the amount of storage space required for workflows.
With the release of Argo Workflows v3.0, the Argo Project said it will provide long-term support for v2.12, providing bug fixes for at least six months but no new features.
Looking ahead, Argo Workflows v3.1 is expected to feature enhancements to make it easier to write fan-in, fan-out workflows using artifacts and conditional artifacts.