HPC Job Bank
HPC in the Cloud


Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Language Flags
Translation Disclaimer

Subscribe | Sign In

November 27, 2006

NCDM Wins Bandwidth Challenge at SC06

The National Center for Data Mining at UIC has won the HPC Bandwidth Challenge at SC06 in Tampa, FL, sponsored by Qwest. Nine institutions participated in the competition. NCDM won by sustaining a data transfer rate of 8 Gb/s over a 10 Gb/s link, with a peak rate of 9.18 Gb/s during the competition window. NCDM uses its own open source software products, UDT and SECTOR, to transfer large datasets efficiently at high speeds on optical networks.

This year the Bandwidth Challenge focused on a specific facet of networking: End-to-End achievement. Competitors were asked to fully utilize one 10 Gig path, end-to-end, disk-to-disk, from SC06 in Tampa back to their home institution, using the actual production network back home. Participants were required to realize, demonstrate and publish all the configuration, troubleshooting, tuning and policies used. The SC06 show floor was connected with the major U.S. research networks, specifically: Abilene, ESnet, NLR PacketNet, NLR FrameNet, and HOPI. The U.S. research networks provided transit for the international networks with which they peer.

NCDM transferred Sloan Digital Sky Survey Data (SDSS) between the SC06 show floor in Tampa, Fl and its lab in 4223 SEL at the University of Illinois in Chicago. It used SECTOR, the newly developed distributed data space management system. SECTOR transparently manages the file locating and data moving, while the NCDM developed UDT software is used for the actual data transfer. The data transfer was disk to disk over one 10 Gb/s shared routed link between SC06 and UIC, via StarLight.

SDSS is systematically mapping a quarter of the entire sky to produce a detailed image of it and determine the positions and absolute brightness of more than 100 million celestial objects. When completed, it will provide a three-dimensional map of nearly a million galaxies and quasars. The total survey information is about 15 terabytes and rivals the information content of the Library of Congress. As the survey progresses, the data are released to the scientific community and the general public in annual increments.

Bandwidth Challenge Competitors: Winner

National Center for Data Mining (NCDM) at UIC, Northwestern Univ., Johns Hopkins Univ., "Transporting Sloan Digital Sky Survey Data Using SECTOR."

Honorable Mention

1) CalTech, CERN, Univ. of Florida, Univ. of Michigan, "High Speed Data Gathering, Distribution and Analysis for Physics Discoveries at the Large Hadron Collider."

2) Indiana Univ., Pittsburgh SuperComputing Center, Oak Ridge National Laboratory, "All in a Day's Work: Advancing Data Intensive Research with the Data Capacitor."

Additional teams

Japanese Aerospace Exploration Agency; Pacific Northwest National Laboratory; Purdue University; Internet2, Univ. of Washington; Texas A&M Univ., Univ. of Delaware; Univ. of Tokyo.

RSS Feeds

Feeds by Topic

Feeds by Content Type


Subscribe to All Content

Most Read Blogs

Aspen

Feature Articles

CometCloud: Using a Federated HPC-Cloud to Understand Fluid Flow in Microchannels

The ever-growing complexity of scientific and engineering problems continues to pose new computational challenges. Thus, we present a novel federation model that enables end-users with the ability to aggregate heterogeneous resource scale problems. The feasibility of this federation model has been proven, in the context of the UberCloud HPC Experiment, by gathering the most comprehensive information to date on the effects of pillars on microfluid channel flow.
Read more...

CERN, Google, and the Future of Global Science Initiatives

Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Read more...

Avoiding Scientific Computing Bottlenecks in the Cloud

Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
Read more...

Short Takes

NASA Builds 'Climate in a Box'

May 23, 2013 | The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
Read more...

Running Computational Fluid Dynamics in the Cloud

May 16, 2013 | When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
Read more...

Sponsored Whitepapers

Best Practices in Big Data Storage

05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.

Exploring the Potential of Heterogeneous Computing

04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.

Sponsored Multimedias

Newsletters

Stay informed! Subscribe to HPC in the Cloud email Newsletters.

HPC in the Cloud Update
HPCwire Weekly Update
Digital Manufacturing Report
Datanami
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases



HPC Job Bank


Featured Events



  • June 16, 2013 - June 20, 2013
    ISC'13
    Leipzig,
    Germany

  • June 17, 2013 - June 18, 2013
    Forecast 2013
    San Francisco, CA
    United States




HPC in the Cloud Conferences & Events
Home | Features | Short Takes | This Just In | Special Features | Public Cloud | Private Cloud | Hybrid Cloud | IaaS | PaaS | SaaS | Cloud Management | Open source | Virtualization | Networking | Datacenters | Applications | Hardware | Storage | Security | Compliance | Standards | Research | Use Cases | Event News | Vendor News | Academia & Research | Media & Entertainment | Financial Services | Government | Life Sciences | Manufacturing | Energy | Retail | Market Watch | Events | Multimedia | Whitepapers | Job Bank | About HPC in the Cloud | Contact Us | Contributors | Site Map | Subscribe | Write for Us
HPC in the Cloud
Tabor Communications |Tabor Publications & Events
Tabor Communications