Spring Cloud sees flowering of Data Flow 2.0….and Skipper too

Spring Cloud has released 2.0 versions of Data Flow and Spring Cloud Skipper, just about hitting the deadline the team behind the service set itself last month.

Spring describes Cloud Data Flow as a “toolkit for building data integration and real-time data processing pipelines”, while Skipper is a tool for discovering, and managing the lifecycle of, Spring Boot applications.

The new Data Flow Server, which is based on Spring Boot 2.1, brings a raft of changes, not least the consolidation of Local, Cloud Foundry and Kubernetes servers into a single server.

Skipper gets to play a bigger role. Data Flow Server 1.x allows for Streams to be deployed directly by the server, or to be delegated to Skipper, dubbed class and skipper respectively. With 2.x the only option is to deploy via Skipper (which technically means no option at all). This also providers “rolling upgrade and downgrade functionality for long lived stream applications”.

Users can now specify the platform where a task will execute – something that was previously only available for Stream and Application deployments – and you can configure multiple Kubernetes and Cloud Foundry task platforms.

The security options have been overhauled, with OAuth2 and OpenID the default security implementation, with “traditional security options….removed.”

2.0 encompasses a raft of UI improvements. It now supports launching a Task against a collection of back-end platforms, and should provide a consistent experience for both Stream and Task launches. You can now rollback to the previous version of a stream with a new button, while a job restart button has been added to the execution page for a job.

The analytics tab has been removed and replaced with a link to a micrometer-fed Grafana dashboard. This ties in with a revamped metrics and monitoring setup for deployed applications, which sees the removal of the Data Flow Metrics Calculator introduced in Data Flow 12.4. Instead, the Micrometer library in deployed applications is used to send metrics back to a “popular monitoring system” before they are visualised in Grafana.

Looking ahead, the focus for the time being is on documentation, user guides, and “overall user experience”. It’s also worth pointing out that maintenance of the 1.x series of Spring Cloud Data Flow will end a year from today.