HPC in the Cloud


Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Language Flags

Whamcloud, EMC Collaborate on PLFS and Lustre Integration


Extend existing relationship to continue to deliver Lustre enhancements

DANVILLE, Calif., April 9 — Whamcloud, a venture-backed company formed from a worldwide network of high-performance computing (HPC) storage industry veterans, today announced it is extending its working relationship with EMC Corporation (NYSE:EMC). The relationship between the two companies began over a year ago and promotes the open source availability of Lustre. Whamcloud and EMC, a fellow member of the OpenSFS consortium, are extending their collaboration for an additional year.

Whamcloud and EMC will continue working together to provide deeper integration between the Parallel Log-structured File System (PLFS) and Lustre. As part of their joint efforts, Whamcloud and EMC will continue augmenting Lustre's IO functionality, including the enhancement of small file IO and metadata performance. The two companies will look for multiple ways to contribute to the future feature development of Lustre.

PLFS is a parallel IO abstraction layer that rearranges unstructured, concurrent writes by many clients into sequential writes to unique files (N-1 into N-N) to improve the efficiency of the underlying parallel filesystem. PLFS can reduce checkpoint time by up to several orders of magnitude. Lustre is an open source massively parallel file system, generally used for large scale cluster computing. It is found in over 60% of the TOP100 supercomputing sites.

"This is a great example of the growing Lustre ecosystem. World-class companies like EMC are committing serious resources to expand Lustre into new areas," said Brent Gorda, CEO of Whamcloud. "Fundamentally, we see the Whamcloud – EMC relationship creating greater technology choices for EMC's customers. We think this is a very positive step forward for Lustre."

"Lustre already has a wide spread following in the technology and HPC communities. It helps address many scalability and performance issues that our HPC customers demand," said Percy Tzelnic, Senior Vice President Fast Data Group and EMC Fellow at EMC. "Lustre continues to grow in importance, and we are pleased to be working with Whamcloud to move the technology into new areas."

As part of the collaboration, Whamcloud and EMC will be working closely to modify some portions of the Lustre client and core code, with the goal of getting the Lustre client included in the upstream Linux kernel.

For additional info on PLFS, see: http://institutes.lanl.gov/plfs.

About Whamcloud

Whamcloud is composed of HPC storage veterans and well-known Lustre experts, and implements and supports Lustre solutions in HPC centers around the world. Whamcloud actively promotes the growth, stability and vendor neutrality of Lustre. Lustre is utilized in over 60% of the TOP100 supercomputing sites today and is the best technology for addressing many of the exascale issues of tomorrow. http://www.whamcloud.com.

-----

Source: Whamcloud

Most Read Blogs

Aspen

Feature Articles

CometCloud: Using a Federated HPC-Cloud to Understand Fluid Flow in Microchannels

The ever-growing complexity of scientific and engineering problems continues to pose new computational challenges. Thus, we present a novel federation model that enables end-users with the ability to aggregate heterogeneous resource scale problems. The feasibility of this federation model has been proven, in the context of the UberCloud HPC Experiment, by gathering the most comprehensive information to date on the effects of pillars on microfluid channel flow.
Read more...

CERN, Google, and the Future of Global Science Initiatives

Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Read more...

Avoiding Scientific Computing Bottlenecks in the Cloud

Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
Read more...

Short Takes

NASA Builds 'Climate in a Box'

May 23, 2013 | The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
Read more...

Running Computational Fluid Dynamics in the Cloud

May 16, 2013 | When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
Read more...

Sponsored Whitepapers

Best Practices in Big Data Storage

05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.

Exploring the Potential of Heterogeneous Computing

04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.

Sponsored Multimedias

Newsletters

Stay informed! Subscribe to HPC in the Cloud email Newsletters.

HPC in the Cloud Update
HPCwire Weekly Update
Digital Manufacturing Report
Datanami
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases



HPC Job Bank


Featured Events



  • June 16, 2013 - June 20, 2013
    ISC'13
    Leipzig,
    Germany

  • June 17, 2013 - June 18, 2013
    Forecast 2013
    San Francisco, CA
    United States




HPC in the Cloud Conferences & Events