December 07, 2011
Dec. 5 — In a large, loud computer equipment room at NASA's Goddard Space Flight Center, amidst the humming of fans and trilling of transistors, is a gadget about the size of a small paperback. Network engineers call them "pluggables." These devices can pump data into a fiber optic line at rates up to 100 gigabits per second (100 Gbps).
That's "gigabit" as in "a billion bits." It is 10,000 times faster than a typical broadband cable modem connection, which operates at a mere 10 million bits per second, or 10 Mbps. 100 Gbps is fast enough to transfer a 25 GB Blu-ray (HD) movie over the Internet in 2 seconds flat.
A data superhighway as speedy as this one doesn't come cheap. The pluggable across the hall costs nearly as much as a luxury sports car. It converts electronic signals into pulses of laser light that travel down fiber optic wires and zip out onto the Internet at near-light speed.
A team of Goddard network engineers borrowed two of the super-fast 100 Gbps pluggables and a router in preparation for a major technology demonstration in Seattle at the Supercomputing 2011 (SC11) conference, November 12-18. The demo gave the high-performance computing world a glimpse of how the Internet will be used in the future to conduct research involving extraordinarily large transfers of data.
Today's leading edge is known as petascale computing. That's "peta" as in "petabyte," or 1,000 terabytes. One terabyte is 1,000 gigabytes, and a gigabyte is 1,000 megabytes, so a petabyte is a LOT of data. A petabyte of storage could hold 40,000 Blu-ray movies.
Now computer scientists are even talking about the coming era of exascale computing: "exa" as in one quintillion bytes, or one billion gigabytes.
Connected to the MAX
Quite a bit of network "plumbing" is required to move data at this scale. Various pieces of equipment and fiber-optic connections had to be put in place to create a 100 Gbps network connection from Goddard to Seattle.
The everyday Internet can't transfer data anywhere near 100 Gbps. The Goddard team had to make special arrangements with various entities on the Internet to create a big enough data pipeline. This required connecting multiple high-speed, fiber-optic research and education networks.
In Goddard's local neighborhood, a key partner in the demo was the Mid-Atlantic Crossroads — affectionately known in the network world as "the MAX," which is based in College Park, Md. The MAX provides Goddard access to Internet2 and other high-speed networks, such as ESnet, SCinet, and Starlight.
Specialized equipment, some of it not yet released on the open market, is also required to switch and route data at such blisteringly high speeds. NASA's vendor partners loaned leading-edge network and server technologies worth around $2 million for the SC11 demonstrations. These companies include Acadia Optronics, Alcatel-Lucent, Brocade, Ciena, cPacket, Force10, Fujitsu, Intel, Juniper, and Supermicro.
After much pre-meeting preparation, network engineers Bill Fink, Paul Lang, and Jeff Martz, members of Goddard's High End Computer Networking (HECN) team, flew out to Seattle to demonstrate technology that could be used to transfer exceedingly large sets of data across a 100 Gbps network. Crates of computer workstations and networking equipment were shipped to the site. The team used the equipment to build a local network at the convention center, connecting the booths of NASA and its various demo partners.
The demos consisted of transferring data between a computer workstation at Goddard and the NASA booth at SC11, and also among the booths of NASA and its partner organizations at SC11: Ciena, the SCinet Network Operations Center, and the University of Illinois at Chicago Laboratory for Advanced Computing/Northwestern University International Center for Advanced Internet Research.
One of the toughest tricks in high-performance networking is a so-called "disk to disk" (DTD) transfer of large data sets. In a DTD transfer, the speed limit is determined by how fast the data moves between two hard drives.
Hard drives are a notorious bottleneck in high-performance networks. In a typical personal computer, the fastest hard drives can transfer data at about 130 megabytes per second, or roughly 1,000 Mbps.
But the Goddard team has developed innovative ways to set up the hardware, software, and networking paths to allow the fastest possible data flow. One important dimension of this is using solid-state memory drives, which are scaled up versions of the small thumb drives people use to transport files between computing devices.
The computer workstation at Goddard used for disk-to-disk networking experiments contains some 40 Vertex3 120-GB solid-state hard drives. Each drive can spew data at a maximum rate of 550 MB per second — fast enough to download hundreds of iTunes songs per second. You would need more than 100 conventional spinning hard drives to match the performance of 40 solid-state drives.
Last year, at Supercomputing 2010, the team demonstrated disk-to-disk transfers up to 30 Gbps. This year they doubled it to 60 Gbps, transferring data between the NASA booth in Seattle and the workstation at Goddard. The top-speed disk-to-disk demo this year transferred data between two booths in Seattle at 72 Gbps (9 GB per second). That is equivalent to a Blu-ray movie download every 3 seconds.
The NASA networking research has its immediate applications in high-performance computing for understanding and predicting climate change, as well as other computer modeling tasks, carried out at the NASA Center for Climate Simulation and other organizations. Ultimately, networking technology enhances our ability to use computers to solve pressing social problems.
The ever-growing complexity of scientific and engineering problems continues to pose new computational challenges. Thus, we present a novel federation model that enables end-users with the ability to aggregate heterogeneous resource scale problems. The feasibility of this federation model has been proven, in the context of the UberCloud HPC Experiment, by gathering the most comprehensive information to date on the effects of pillars on microfluid channel flow.
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Frank Ding, engineering analysis & technical computing manager at Simpson Strong-Tie, discussed the advantages of utilizing the cloud for occasional scientific computing, identified the obstacles to doing so, and proposed workarounds to some of those obstacles.
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.