Not bad in the grand schema of things: Elastic pushes out 7.12 with direct object storage search

Not bad in the grand schema of things: Elastic pushes out 7.12 with direct object storage search

Elastic has released 7.12, making a bunch of new capabilities available in its Elastic Enterprise Search, Observability, and Security solutions that form part of the Elastic Stack providing distributed search and analytics capabilities.

Perhaps the most significant of these is a new schema flexibility that allows users to have both performance and adaptability in Elasticsearch. Users can also unlock new value by making object stores fully searchable, and automatically scale deployments on the Elastic Cloud hosted service, according to the firm.

The schema flexibility comes through a feature known as runtime fields, which allows users the option to create a schema on the fly at query time – known as schema on read – in addition to using the conventional schema on write.

Schema on write helps make Elasticsearch fast, but requires prior planning and testing of an organised structure for how the data will be represented. This can cause difficulties if, some time later, the user needs to ingest new data or adopt a new use case for the data within a short timeframe.

According to Elastic, runtime fields have been implemented in such a way that users do not have to choose between the speed and scale of schema on write or the flexibility of schema on read. Both can be used at the same time, on the same Elastic Stack, and on the same data. The feature is claimed to greatly reduce the time-to-value for customer data by trading off some search performance.

The frozen tier feature is currently in technical preview, but adds the capability to directly search object stores such as Amazon’s S3, Google Cloud Storage and Microsoft Azure Storage. The benefit of this is cost, enabling users to search data while reducing the amount of dedicated resources needed for the search, albeit with a trade-off in performance.

Elastic claims that by fetching only the data needed to complete a query from the object store and caching this locally, the frozen tier offers the best search experience while enabling users to access an unlimited amount of data. This should make it cost-effective to store more analytics data for marketing analysis, or retain all log and security data for security teams to analyse.

Elastic Cloud now features autoscaling, which monitors both the storage utilisation for Elasticsearch data nodes and the available capacity for machine learning jobs and will automatically adjust resource capacity to maintain node performance.

Once autoscaling is enabled through API, command line, or the Elastic Cloud console, the user’s Elasticsearch data node capacity will grow automatically as more data is stored, while machine learning node memory and CPU capacity will grow or shrink based on the resource requirements of the machine learning jobs. Elastic says that users can set thresholds to prevent runaway cluster growth.

Elastic 7.12 is available now on Elastic Cloud, while customers may also download the Elastic Stack and cloud orchestration products, Elastic Cloud Enterprise and Elastic Cloud for Kubernetes, for a self-managed experience.

As we reported earlier this year, starting with Elasticsearch and Kibana 7.11, source code can be used either under the SSPL or Elastic License.