Hardware compression solution for Apache Hadoop

Exar and data management solutions have announced hardware-accelerated compression solution for Apache Hadoop. The solution called AltraHD enables all applications in the Hadoop stack to be transparently compressed using hardware acceleration. AltraHD integrates seamlessly into the Hadoop stack, and is able to transparently compress all files, including files that are stored using the file system as well as files stored locally as intermediate data outside of the file system. AltraHD is the only compression solution for Hadoop to offer the following key features, as per Exar: Application transparent file system filter that sits below the Hadoop Distributed File System (HDFS) to compress all files that are using HDFS. Compression codec that compresses intermediate data during the MapReduce phase of Hadoop processing. Exar's high performance PCIe-based hardware compression card that automatically accelerates all compression operations, maximizing performance while offloading the host CPU. A single card provides up to 3.2 gigabyte/sec of compression throughput. "Exar has taken big data analytics to the next level," stated Rob Reiner, Exar's director of marketing for Data Compression and Security products. "AltraHD typically improves Terasort performance by 50 – 100%, and typically provides a 3x - 5x savings in storage capacity, enabling unmatched Hadoop cluster p...