November 02, 2011
SEATTLE, Nov. 1 — Scientists studying data or compute-intensive problems require high bandwidth and computational resources, often from heterogeneous systems at different sites.
But they don't need these resources all the time.
Ideally, a scientist studying the properties of new materials for producing solar energy, for example, would be able to grab a "slice" of a high-bandwidth pipeline, set their workflow in motion, grab compute resources in the cloud and then release those resources, so they could be used by other researchers in different configurations.
At the RENCI/North Carolina research exhibit at SC11, three demonstrations by the RENCI networking research group and Duke University will use ORCA, the Open Resource Control Architecture, to bring together cyber resources from multiple providers as needed to accommodate a scientific workflow.
ORCA was developed by Duke computer science professor Jeff Chase and his students with funding from the National Science Foundation. It is one of the experimental control frameworks for the NSF's Global Environments for Network Innovation (GENI) project. GENI is a virtual laboratory for networking experiments that will help researchers develop the tools and protocols that will define future internets. With funding from the Department of Energy Advanced Scientific Computing Research program and the NSF Software Development for Cyberinfrastructure program, researchers are adapting ORCA as an Infrastructure as a Service (IaaS) platform for serving the diverse needs of computational scientists.
The first demonstration will execute a scientific workflow by using ORCA to allocate a slice of computational resources from multiple cloud providers and bandwidth-provisioned network connections between provider sites. The workflow, managed by the Pegasus workflow management system, will use six serial applications, which will run on Condor clusters dynamically provisioned from clouds owned by RENCI in Chapel Hill, NC, and by Duke University in Durham, NC. The two clouds are connected by the Breakable Experimental Network (BEN), an experimental network that connects RENCI and its partner institutions at Duke, UNC-Chapel Hill and North Carolina State University.
A final large MPI application will run on several thousand processors on Hopper, a Cray Xe6 system at the National Energy Research Scientific Computing Center (NERSC) in Berkeley, CA.
ORCA will provision several network resources to move data across the continent, starting with BEN in North Carolina. From the southeastern U.S., the workflow will make its way to NERSC, first via the National Lambda Rail, then to the StarLight interconnect in Chicago, and finally via ESnet, the Energy Science Network, to NERSC.
"We will set up a collection of disparate resources in multiple clouds that never existed before and won't exist once the job is completed," said Ilia Baldine, director of the RENCI networking research group. "We plan to show that ORCA is an Infrastructure as a Service platform suitable for both GENI experimenters and computational scientists and that it is capable of provisioning resources as they are needed and then allowing them to return to their owners to be accessed by other users."
The science: new materials for solar energy
The scientific job will be a simplified version of a workflow used to apply effective forward design strategies to the discovery of new materials for solar energy. In inverse design, scientists start with a set of desired electronic properties for a material and then search for the best structure. A major step in the process is the calculation of a particular property that occurs as part of the forward chain. The workflow will examine the electronic structure of moieties of Ruthenium (Ru) molecules and attempt to determine their total energy. Ruthenium can absorb light in the visible spectrum, which makes it a good candidate for a material used in cost-effective solar energy cells. The work is supported under U.S-DOE SciDAC-e award DE-FC02-06ER25764, "Enhancing Productivity of Materials Discovery Computations for Solar Fuels and Next Generation Photovoltaics."
A related demonstration will use the ORCA framework to execute a Hadoop workflow on multiple clouds connected through bandwidth-provisioned network pipelines. Hadoop is a software framework for data-intensive distributed applications. A third demonstration will take a closer look at a part of the first demonstration: the on-demand provisioning of computational infrastructure to stand up a Condor cluster in a networked cloud environment.
The demonstrations will take place in the RENCI booth(2942). Demonstration times are:
· Monday, Nov. 14: 7 p.m. – 9 p.m.
· Tuesday, Nov. 15: 10:30 a.m. (demo 1), 11:30 a.m. (demo 2) and 1 p.m. (demo 3)
· Wednesday, Nov. 16: 10:30 a.m. (demo 1), 2 p.m. (demo 2) and 2:30 p.m. (demo 3)
· Thursday, Nov. 17: 10:30 a.m. – 12:30 p.m. (demos 1, 2 and 3)
ORCA was developed at the Duke University New Internet Computing Lab by computer science professor Jeff Chase and his students. RENCI and Duke are partners in a GENI project to evaluate ORCA as a future Internet control plane framework.
For more information:
The ever-growing complexity of scientific and engineering problems continues to pose new computational challenges. Thus, we present a novel federation model that enables end-users with the ability to aggregate heterogeneous resource scale problems. The feasibility of this federation model has been proven, in the context of the UberCloud HPC Experiment, by gathering the most comprehensive information to date on the effects of pillars on microfluid channel flow.
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.