Cloudera's Latest Release Includes Hadoop Optimization Tool

Cloudera, which provides a complete data management and analytics platform built on Apache Hadoop and related open source technologies, has announced the availability of Cloudera Enterprise 5.5.

The new release focuses on improving the performance, security, and functionality of analytics on Hadoop and includes the limited beta release of Cloudera Navigator Optimizer--a tool for improved workload performance and efficiency.

Workload management is an issue for many Hadoop-focused organizations. Within any business, there can be countless workloads being run at any given time, across multiple systems, that change based on time of day and business need. Across many of these workloads, similar pain points have emerged, like breakdowns in ETL pipelines, long wait times for BI reports, increasing system pressure from ad hoc queries, and unnecessary query complexity. When adopting new systems based on Hadoop, it’s critical that customers understand their workloads, so they can address these inefficiencies and run the right workloads in the right systems for the best results.

In the end, managing workloads well can reduce the complexity of Hadoop, which is an issue.  Gartner, Inc.'s 2015 Hadoop Adoption Study, involving 284 Gartner Research Circle members, found that only 125 respondents who completed the whole survey had already invested in Hadoop or had plans to do so within the next two years. The study found that there are difficulties in managing Hadoop.

“Inefficiently allocated data workloads are all too common across enterprises—causing costly delays, errors, and complexity. Customers need visibility into their workloads and guidance to ensure the best results with Hadoop,” said Charles Zedlewski, vice president, Products at Cloudera. “The beta release of Cloudera Navigator Optimizer, as part of Cloudera Enterprise 5.5, provides customers with a path to success by providing visibility into their workloads, and helps customers understand the ones that are the best suited for deployment on Hadoop to reduce development time and improve performance.”

Cloudera Navigator Optimizer analyzes existing workloads, "providing visibility into which ones are the most critical, which data is accessed most, and how is it being used," according to Cloudera. It then turns this information into a full optimization strategy for use with Hadoop. Through a dashboard, customers get prioritization guidance on where to focus development efforts to achieve the biggest impact, centered around identifying duplication, exposing complexity, and leveraging compatibilities with ecosystem tools such as Impala and Apache Hive.

In addition to Cloudera Navigator Optimizer, Cloudera Enterprise 5.5 includes a number of other capabilities to empower analytics on Hadoop, including:

- Analytics on Complex Data Structures - Impala now supports nested data types, including JSON, for expanded data discovery and business intelligence.

- Secure Analytics Across the Entire Business - Impala now includes column-level security for fine-grained access controls for all users. Cloudera Enterprise now also includes credential management, including password redaction, for further protection.

- Automated Data Stewardship and Extensible Governance - Cloudera Navigator automatically enforces full data lifecycle workflows, including retention and archiving, so the right data is always available for consumption.

 “One of the most common questions is, 'how can Hadoop complement and optimize data warehouse environments?’ as shown in ESG research,” says Nik Rouda, senior analyst at ESG Research. “Cloudera Navigator Optimizer, built on the innovations of Xplain.io, looks incredibly useful in answering this question. It will help identify the best candidates for workloads that could be shifted to Hadoop, as well as any inefficiencies or redundancies in queries. The cost delta between traditional data warehouse environments and Hadoop is enormous, and this enables companies to realize those gains for better performance and efficiency.”

You can learn more on the Cloudera Engineering Blog.