HPC Job Bank
HPC in the Cloud

Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Language Flags

Thank you for your interest in this content. Your download is available at the link below:

Xyratex Map/Reduce on Lustre®
Source: Xyratex
Release Date:August 30, 2011

This paper compares HDFS and Lustre architectural drivers and resulting system performance of Map/Reduce computations on HPC hardware. We evaluate theoretical and actual performance of Lustre and HDFS for a variety of workloads in both traditional and Map/Reduce-based applications. Further, we examine the additional benefits (cluster efficiency, flexibility, and cost) of using a general-purpose distributed file system, such as Lustre, on a Hadoop compute cluster.