HPC in the Cloud


Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Language Flags

Bright Computing Brings Data Aware Scheduling to Cloud Bursting into Amazon EC2


Bright Cluster Manager 6.0 increases productivity and ease of use for extending high performance computing into the cloud

SAN JOSE, Calif., May 1 — Bright Computing, a leading provider of cluster and cloud management software for high-performance computing (HPC), today announced that Bright Cluster Manager 6.0 is now shipping with comprehensive support for cloud bursting, including data aware scheduling. Bright's ability to easily extend on-site clusters and servers into the Amazon EC2 cloud, and to manage these resources as if local, significantly lowers the barrier, and cost, of dynamically adding compute capacity.

Bright Computing is the first to offer data aware scheduling for cloud bursting, a productivity boost that saves both manual effort and compute time. Bright Cluster Manager ensures that the data required for cloud computing is accessible just prior to the start of the job, initiates the job, and then transfers the results back as soon as the job is completed. This work is completed as a single step for the user, eliminating the need to monitor and manage the data.

"One important barrier to cloud computing adoption in HPC is data movement and data management," said Steve Conway, IDC research vice president for HPC. "Cloud latency issues can make it challenging to ensure that the right data are in position when applications need to run on the cloud, and that the data are returned in a timely way afterward.

"Bright addresses this challenge by designing data aware scheduling directly into the company's cloud bursting capability. This integrated approach could make it easier for mainstream HPC applications to exploit cloud computing."

Typically, to run a compute job without data aware scheduling, cloud computing users need to initiate the data transfer, monitor the completion of this task, and then submit the job into the queue. Upon completion, the user needs to retrieve the output data. This often-clumsy process consumes both user time and usage fees. By contrast, Bright Cluster Manager runs the process as a single step, dramatically improving productivity and cycle times.

Bright Cluster Manager 6.0 provides cloud-readiness as a standard feature, enabling system administrators to expand capacity as needed, or to evaluate the use of GPUs for applications. Bright supports all Amazon instance types, including Cluster GPU Instances.

Bright provides two alternatives for cloud bursting: extending on-site clusters into the EC2 cloud, and managing these nodes as if part of the local system; or creating entirely new clusters in EC2 and providing the full provisioning, scheduling, monitoring and management capabilities that Bright provides for local clusters. Either approach is achieved with a few mouse clicks.

The cluster extension scenario is particularly useful for users with varying workload demands on their clusters. Users can determine whether to run their applications locally, in the cloud, or for suitable parallel applications, split between the local cluster and the cloud. In all cases, the system administrator and user have complete visibility, as if the entire cluster was on-site.

Bright removes the complexity from using the cloud for HPC. Much of the complex functionality is performed without the need for user intervention, such as the data aware scheduling or automatic VPN creation. Furthermore, with Bright Cluster Manager, all instance types are initialized with equal ease, including cluster compute instances that utilize Hardware Virtual Machine (HVM) virtualization. The tasks are managed with Bright's intuitive graphical user interface, making it easy to get the most out of cloud bursting into EC2.

Aside from cloud bursting, Bright Cluster Manager 6.0 is shipping with dozens of enhancements to its system management capabilities:

  • Integration of openlava, the open source version of LSF, in addition to the existing integration of LSF, SLURM, PBS Professional, TORQUE/Moab, TORQUE/Maui, Univa Grid Engine, Open Grid Scheduler
  • Power-saving mode for use with SLURM and PBS Professional to lower operating costs
  • Full support for CUDA 4.1
  • Enhancements to automatic failover capabilities
  • JSON based API in addition to the existing SOAP API, for ease of JavaScript integration
  • Support for Mellanox and QLogic OFED stacks
  • Booting over IB added for QLogic
  • Head node installation on existing Linux installations, in addition to standard bare-metal installation
  • Dozens of other enhancements

"In building Bright 6.0, we listened closely to what our customers and prospects needed to benefit from cloud computing," said Martijn de Vries, CTO of Bright Computing. "They wanted to access cloud resources from their existing clusters, and cluster management solution— not as a separate entity. They wanted it to be straightforward— no learning curve, no process changes. They wanted to solve the data management issue— without jumping through hoops every time. They simply wanted to extend their local clusters, as needed, without becoming EC2 experts. I believe we have delivered."

About Bright Computing

Bright Computing specializes in management software for clusters, grids and clouds, including compute, storage, Hadoop and database systems. Bright's fundamental approach and intuitive interface makes cluster management easy, while providing powerful and complete management capabilities for increasing productivity. Bright Cluster Manager is the solution of choice for many research institutes, universities, and companies across the world, and is used to manage several Top500 installations. Bright Computing has its headquarters in San Jose, California. http://www.brightcomputing.com.

-----

Source: Bright Computing, Inc.

Most Read Blogs


Feature Articles

CERN, Google, and the Future of Global Science Initiatives

Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Read more...

Avoiding Scientific Computing Bottlenecks in the Cloud

Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
Read more...

Overcoming the Cloud Security Barrier for Financial Services

The private industry least likely to adopt public cloud services for data storage are financial institutions. Holding the most sensitive and heavily-regulated of data types, personal financial information, banks and similar institutions are mostly moving towards private cloud services – and doing so at great cost.
Read more...

Short Takes

Running Computational Fluid Dynamics in the Cloud

May 16, 2013 | When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
Read more...

In Support of Cloud-based Rendering

May 10, 2013 | Australian visual effects company, Animal Logic, is considering a move to the public cloud.
Read more...

Internet2 Awards Program Seeks Innovative Applications

May 10, 2013 | Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
Read more...

Sponsored Whitepapers

Best Practices in Big Data Storage

05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.

Exploring the Potential of Heterogeneous Computing

04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.

Sponsored Multimedias

Newsletters

Stay informed! Subscribe to HPC in the Cloud email Newsletters.

HPC in the Cloud Update
HPCwire Weekly Update
Digital Manufacturing Report
Datanami
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases



HPC Job Bank


Featured Events



  • June 16, 2013 - June 20, 2013
    ISC'13
    Leipzig,
    Germany

  • June 17, 2013 - June 18, 2013
    Forecast 2013
    San Francisco, CA
    United States




HPC in the Cloud Conferences & Events