The Top 10 Big Data Tools And Technologies Of 2018 (So Far)

As the volume of data that businesses try to collect, manage and analyze continues to explode, spending for big data and business analytics technologies is expected to reach $210 billion by 2020. Here are 10 big data products that caught our attention in the first half of 2018.

Managing The Data Deluge

See the latest entry: The 10 Hottest Big Data Tools of 2022

Ever-increasing volumes of data, new data-generation sources like Internet of Things networks, and the growing use of artificial intelligence and machine learning to manage and analyze all that data are just some of the drivers behind the continuing evolution of big data technology.

Spending on big data and business analytics products is expected to reach $210 billion by 2020, according to market researcher IDC, up from $150.8 billion in 2017. That growth is fueling a continuous stream of startups in the big data arena. But it's also pushing more established players to maintain a rapid pace of developing and delivering new and updated big data products.

Here are 10 big data products announced in the first half of 2018 that caught our attention.

(For more on the biggest news of 2018, check out "CRN's Tech Midyear In Review.").

Arcadia Data

Arcadia Data unleashed a significant new release of its Arcadia Enterprise and Arcadia Instant business analytics software in March, offering greater scalability to handle bigger volumes of data, a greater variety of data formats and more concurrent users. The software also provided new self-service analytics and data governance capabilities.

Arcadia Data's software works natively with data lakes -- massive repositories of raw data in original formats -- to help business users derive insights and value from all that data. Arcadia's software eliminates the need to move data into separate business intelligence servers or data warehouses for analysis.

In April the vendor debuted Arcadia Instant for KSQL, a product the company said makes it easier for business users to develop visual analysis apps for live data streaming through Apache Kafka systems.

Couchbase Managed Cloud

In Partnership with Rackspace, Couchbase in May debuted Couchbase Managed Cloud, a fully-managed data base service that offers the Couchbase NoSQL database technology along with Rackspace's managed cloud and application services.

Couchbase Managed Cloud provides database infrastructure as a cloud service deployed within an organization's public cloud environment alongside other cloud applications. That, according to Couchbase, reduces processing latency, increases performance and enhances security.

Couchbase Managed Cloud also provides a unified data platform across multi-cloud architectures by way of Couchbase's cross datacenter replication technology.

Hortonworks Data Platform 3.0

Hortonworks launched the latest edition of its big data platform in June, offering a range of new enterprise features with an emphasis on improving the speed of developing and deploying big data applications.

The Hadoop-based HDP 3.0 supports application deployment through containerization, which allows applications to be launched more quickly. The new release also provides improved query optimization -- the ability to process more data at a faster rate -- through support for the Apache Hive 3.0 data warehouse software.

HDP 3.0 also supports deep learning and machine learning applications and offers improved data governance and regulatory compliance functionality.

Incorta Version 3.0

Incorta's enterprise analytics platform aggregates complex data in real time, augmenting traditional data warehouses -- or eliminating the need for them entirely.

In February Incorta unveiled the 3.0 edition of its analytics software, offering query speed improvements of as much as 500 percent, a "Direct Data Mapping" engine that speeds deployments by ingesting data without traditional data modeling, and expanded capabilities for third-party data analysis by leveraging the Incorta SQL interface.

Maana Q

Maana develops the AI-based Maana Knowledge Platform, a system that uses the company's patented Computational Knowledge Graph technology and algorithms to provide a holistic view of assets or processes a company wants to optimize.

In February the company debuted Maana Q, which provides a way to capture and digitize the expertise of subject matter experts who make critical operational decisions.

Maana Q adds self-service capabilities to the Knowledge Platform, allowing subject matter experts to use the Computational Knowledge Graph to create a digital knowledge layer over operational and industrial data. That layer demonstrates the interdependencies between concepts such as equipment, people and activities, which businesses use to build AI-driven applications that optimize decision flows.

MapR Data Platform

In June MapR Technologies unveiled the latest release of its MapR Data Platform, providing a range of enhancements that improve access to cloud and edge-device data. The new release also simplifies the development and deployment of AI and analytical applications.

The new release, MapR Data Platform 6.1, provides native API access to Amazon Web Services S3 storage, allowing analytical applications running on the MapR system to access S3 data in place, speeding up analytical workloads.

The new release also supports working within hybrid cloud environments with Internet accessible storage. And AI and analytical application development get a boost through support for Spark 2.3, Hive 2.3 and Apache Drill 1.14.

MongoDB 4.0, MongoDB Stitch

In June MongoDB launched MongoDB 4.0, a new release of the company's database software with support for multi-document ACID (atomicity, consistency, isolation, durability) transactions. That provides a consistent view of data across replicated sets and enforce all-or-nothing execution to maintain transaction data integrity.

MongoDB also introduced MongoDB Stitch, a new serverless platform that facilitates the development of mobile and web applications. The system includes Stitch QueryAnywhere, which makes the MongoDB query language available to developers; Stitch Functions, allowing developers to run JavaScript functions; and Stitch Triggers, real-time notifications of changes in the database.

Also new is MongoDB Mobile, which extends MongoDB to iOS and Android devices and Internet of Things assets. And the new MongoDB Stitch Mobile Sync synchronizes data between MongoDB Mobile and the backend database.

SAS Viya

SAS announced at its annual SAS Global Forum in April that it had embedded artificial intelligence capabilities into its flagship SAS analytics platform and its Viya visual analytic software, part of a broader effort to leverage AI technology across the vendor's broad product portfolio.

The AI functionality in the core platform automates a number of data preparation tasks, including cleaning, formatting and transforming data for analysis. The AI capabilities in Viya automate a number of steps associated with building and deploying analytical models.

Splunk Industrial Asset Intelligence

In April Splunk took the wraps off of Splunk Industrial Asset Intelligence, the company's first product specifically for Internet of Things data collection and analysis.

Splunk IAI targets applications in the industrial IoT space, including manufacturing, transportation, oil and gas, and energy/utilities.

Splunk IAI is designed to bridge the gap between operational technology and traditional IT, including business analytics. The software provides capabilities for collecting, monitoring and analyzing industrial IoT machine data -- in real time -- generated by industrial controllers, sensors and operational applications.

Tableau Prep

Recent studies have shown that analysts spend as much as 80 percent of their time preparing data and only 20 percent analyzing it. In April data analytics software developer Tableau Software sought to reduce data preparation times with Tableau Prep, a new data preparation product that makes it easier for everyday workers to combine, shape and clean data for data analysis.

The software, which emphasizes data visualization functionality, includes features that automate complex data preparation tasks such as joins, pivots and aggregations. The new product also provides smart algorithms such as "fuzzy clustering" that automate repetitive tasks such as grouping by pronunciation or cleaning by punctuation.

Tableau Prep is integrated with Tableau's data analysis workflow and is offered as part of the company's Creator subscription offerings. Current users of Tableau Desktop can use Tableau Prep at no charge for two years.