November 27, 2006
The National Center for Data Mining at UIC has won the HPC Bandwidth Challenge at SC06 in Tampa, FL, sponsored by Qwest. Nine institutions participated in the competition. NCDM won by sustaining a data transfer rate of 8 Gb/s over a 10 Gb/s link, with a peak rate of 9.18 Gb/s during the competition window. NCDM uses its own open source software products, UDT and SECTOR, to transfer large datasets efficiently at high speeds on optical networks.
This year the Bandwidth Challenge focused on a specific facet of networking: End-to-End achievement. Competitors were asked to fully utilize one 10 Gig path, end-to-end, disk-to-disk, from SC06 in Tampa back to their home institution, using the actual production network back home. Participants were required to realize, demonstrate and publish all the configuration, troubleshooting, tuning and policies used. The SC06 show floor was connected with the major U.S. research networks, specifically: Abilene, ESnet, NLR PacketNet, NLR FrameNet, and HOPI. The U.S. research networks provided transit for the international networks with which they peer.
NCDM transferred Sloan Digital Sky Survey Data (SDSS) between the SC06 show floor in Tampa, Fl and its lab in 4223 SEL at the University of Illinois in Chicago. It used SECTOR, the newly developed distributed data space management system. SECTOR transparently manages the file locating and data moving, while the NCDM developed UDT software is used for the actual data transfer. The data transfer was disk to disk over one 10 Gb/s shared routed link between SC06 and UIC, via StarLight.
SDSS is systematically mapping a quarter of the entire sky to produce a detailed image of it and determine the positions and absolute brightness of more than 100 million celestial objects. When completed, it will provide a three-dimensional map of nearly a million galaxies and quasars. The total survey information is about 15 terabytes and rivals the information content of the Library of Congress. As the survey progresses, the data are released to the scientific community and the general public in annual increments.
Bandwidth Challenge Competitors: Winner
National Center for Data Mining (NCDM) at UIC, Northwestern Univ., Johns Hopkins Univ., "Transporting Sloan Digital Sky Survey Data Using SECTOR."
1) CalTech, CERN, Univ. of Florida, Univ. of Michigan, "High Speed Data Gathering, Distribution and Analysis for Physics Discoveries at the Large Hadron Collider."
2) Indiana Univ., Pittsburgh SuperComputing Center, Oak Ridge National Laboratory, "All in a Day's Work: Advancing Data Intensive Research with the Data Capacitor."
Japanese Aerospace Exploration Agency; Pacific Northwest National Laboratory; Purdue University; Internet2, Univ. of Washington; Texas A&M Univ., Univ. of Delaware; Univ. of Tokyo.
The ever-growing complexity of scientific and engineering problems continues to pose new computational challenges. Thus, we present a novel federation model that enables end-users with the ability to aggregate heterogeneous resource scale problems. The feasibility of this federation model has been proven, in the context of the UberCloud HPC Experiment, by gathering the most comprehensive information to date on the effects of pillars on microfluid channel flow.
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.