Hadoop provider Cloudera has acquired big-data Relevant Products/Services security Relevant Products/Services firm Gazzang. The companies said their combined capabilities will provide end-to-end enterprise-grade security and compliance along with encryption and management.

Terms of the deal were not made public. Cloudera, which works with the Apache Hadoop framework, said its customers will still have the ability to choose from cross-platform protection Relevant Products/Services methods made available by its partners. Gazzang's products offer data encryption in Hadoop environments and access management for keys and tokens.

"We're entering a whole new era with the rise of the Industrial Internet and the Internet of Things," Cloudera CEO Tom Reilly said in a statement, "where there is vastly more data being streamed from billions of devices."

Center for Security Excellence

Currently, Cloudera can provide encryption for all data-at-rest inside a Hadoop cluster, which it said was transparent to programs using the data. The Gazzang team, according to Cloudera, will be directed toward offering additional security measures and will become the core of the company's Center for Security Excellence. The Gazzang technology can be a key part of Cloudera's appeal to customers in industries with high requirements for end-to-end security and compliance.

That center Relevant Products/Services, exclusively dedicated to Hadoop-based security, will work on technologies relating to cluster data and security, such as "follow the data" authorization and encryption policies.

It will also focus on security testing and configuration, including continuous vulnerability assessment, performance optimization, development of regulatory compliance playbooks, enabling security integration APIs and certifying partner products. Cloudera said it would continue working with Intel Relevant Products/Services on big-data security in the open-source Hadoop community.

Additionally, the Palo Alto, California-based Cloudera said it would continue investing in support of the open-source project Rhino, a security architecture that includes perimeter security, entitlements, access control and data protection. The Gazzang technology is expected to help with the Rhino initiative.

Cloudera Versus Hortonworks

In May, Cloudera competitor Hortonworks bought California-based XA Secure, a start-up involved in Hadoop-based security. Hadoop projects often implement their own security, or use third-party solutions.

Many regarded Cloudera and its distribution of Hadoop as the leading distribution and vendor until Yahoo's Hadoop engineering team created a new company, Hortonworks, in the summer of 2011. The Hortonworks Data Platform, which came out in fall of 2011, became the key competitor to Cloudera's version.

Hortonworks offers its distribution for free, with paid training and tech support. Cloudera also has a free distribution, with proprietary management console and support services sold separately.

Apache Hadoop is an open-source software Relevant Products/Services project and community that is designed for distributed processing of big data, using clusters of commodity servers that could number in the thousands. On Tuesday, the Hortonworks Hadoop Summit opened in San Jose, California. One of the key drivers for this year's conference is YARN, a cluster resource management layer whose acronym stands for Yet Another Resource Manager and which was released last October.