Apache Ignite team grants users’ wish for better observability

Apache Ignite team grants users’ wish for better observability

Open source in-memory computing platform Apache Ignite has made the jump to version 2.8, with its development team adding an experimental observability subsystem and pushing forward machine learning capabilities.

The new subsystem is made up of a number of rather generic registries, which group metrics related to components like cache or service grid APIs. It is experimental for now, so users can take the API for a spin before it gets a default.

In the long run, however, the system is meant to help users deploying the tool in critical environments get more insight into the state of the platform. Initial exporters for monitoring interfaces cover log files, JMX and SQL views, as well as OpenCensus, so it should be pretty useful even in this early stage.

The Ignite team tried to improve the platform’s performance, taking care of some blocking caused by partition-map-exchange processes that kicked in when a node of the current baseline topology left the cluster or a thick client connected to it. It also revisited crash recovery for clusters on restarts and added a read-repair feature to manage data inconsistencies.

In terms of machine learning, the team has put some effort into finally adding ensemble methods to the tool. This approach is helpful to combine simpler machine learning models into more complex and, depending on the input and technique used, also more precise and less biased ones. On top of that, a new pipelining API has been included to let users build multi-step workflows and there are now ways to import models trained in Apache Spark or using XGBoost. 

Since it isn’t a strictly a Java project, Ignite learned partition-awareness for thin clients, which should lead those building apps in C++, Python, or other compatible languages to a bit of a performance boost, since queries can now be sent to the nodes with the needed data directly instead of being proxied through a single server node. More details about changes and improvements can be found in the release announcement.