November 14, 2011
SEATTLE, Nov. 14 — Today at Supercomputing 2011 (SC11), the international conference for high performance computing, networking, storage and analysis, researchers from Orange Silicon Valley, The Department of Energy's Energy Sciences Network (ESnet), InfiniBand Trade Association (IBTA) and OpenFabrics Alliance (OFA) join forces with the SC11 conference network SCinet to showcase the world's first demonstration of a Remote Direct Memory Access (RDMA)-enabled data transfer at 40 Gigabits per second (Gbps) over wide area network links spanning approximately 7,000 miles.
This experiment marks the first time that RDMA has been demonstrated at 40Gbps data rates for such a long distance, showcasing the technology's ability to improve performance for bulk data transfer in the wide area. The demonstration illustrates three usage models of RDMA at such a distance: the ability to move multiple simultaneous data streams; the ability for a user to access a remote data set as though it were locally mounted; and the ability to move large file transfers at high bandwidths. The RDMA protocols in the wide area enable the three usage models to achieve up to 96 percent utilization of the available bandwidth in the network.
During the demonstration, SC11 attendees will see firsthand how RDMA enables servers in the local nodes to transfer data to or from application memory, eliminating the overhead of copying data between application memory and the data buffers in the operating system. When an application performs an RDMA operation, the application data is delivered directly to the network, eliminating the additional latency the transfer would incur if the data were to be copied into the operating system memory buffers. This dramatically increases the application-messaging rate and the application processing efficiency of the servers from approximately 20 percent up to 80-90 percent.
"RDMA communication opens new possibilities for collaboration among researchers by enabling quick and efficient data sharing," said Jim Ryan, chair of the Open Fabrics Alliance. "By eliminating many of the typical network performance bottlenecks, users make more efficient use of their WAN, cloud and data center infrastructures."
The demonstration was conceived this past summer when Orange Silicon Valley, a subsidiary of France Telecom-Orange, an international operator for mobile and broadband Internet services and one of the world leaders in providing telecommunication services to multinational companies, partnered with ESnet and Bay Microsystems to perform the first field trial of this technology demonstrating RDMA protocols over a 40Gbps WAN over a distance of 230 miles. The success of that field trial encouraged Orange Silicon Valley and the collaborators to consider longer distance trials at SC11, leveraging the SC11 SCinet Research Sandbox (SRS), a conference program, which provides a venue for network researchers to showcase disruptive innovation in an experimental network environment.
Orange Silicon Valley successfully submitted a proposal to the SCinet SRS to showcase this technology at SC11, choosing the OFA and IBTA booth based on both consortiums' vision of embracing open standards. The SC11 demonstration – including technology from members of both the IBTA and OFA - leverages ESnet's nationwide ANI Testbed which is being made available to researchers both in the public and private sector as a first of its kind platform to test experimental approaches to new network protocols and architectures in a greater than 10 Gbps network environment.
"This collaboration at SC11 between Orange Silicon Valley, ESnet and members of the IBTA and OFA enables a unique environment for demonstrating the benefits of globally scaling out and deploying RDMA-enabled applications," said Brian Sparks, co-chair of the marketing working groups of the IBTA and OFA.
The following member companies provided hardware, software, architectural and/or operational expertise in this demonstration: 3M, Bay Microsystems, Chelsio Communications, DataDirect Networks, Finisar, HP, Mellanox Technologies, NetApp, Software Forge, System Fabric Works and Volex.
About the InfiniBand Trade Association
The InfiniBand Trade Association was founded in 1999 and is chartered with maintaining and furthering the InfiniBand specification. The IBTA is led by a distinguished steering committee that includes HP, IBM, Intel, Mellanox Technologies, Oracle, QLogic Corporation and System Fabric Works. Other members of the IBTA represent leading enterprise IT vendors who are actively contributing to the advancement of the InfiniBand specification. The IBTA markets and promotes InfiniBand from an industry perspective through online, marketing and public relations engagements, and unites the industry through IBTA-sponsored technical events and resources. For more information on the IBTA, visit http://www.infinibandta.org.
About the OpenFabrics Alliance
The OpenFabrics Alliance (OFA) is a 501c) (6) non-profit association that develops, tests, licenses and distributes the OpenFabrics Enterprise Distribution (OFED) – multi-platform, open-source software for high-performance, low-latency and energy-efficient computing. OFED is used in business, research and scientific environments that require fast and efficient networks, storage connectivity and parallel computing. OFED is free and is included in major Linux distributions, as well as Microsoft Windows. In addition to distributing OFED, the OFA conducts training, workshops and interoperability testing to ensure all releases meet multivendor enterprise requirements for security, usability and reliability. For more information about the OFA, visit http://www.openfabrics.org.
France Telecom-Orange is one of the world's leading telecommunications operators with 170,000 employees worldwide, including 102,000 employees in France, and sales of 33.8 billion euros in the first nine months of 2011. Present in 35 countries, the Group had a customer base of 221 million customers at 30 September 2011, including 145 million customers under the Orange brand, the Group's single brand for Internet, television and mobile services in the majority of countries where the company operates. On 30 September 2011, the Group had 162 million mobile customers and 14 million broadband internet (ADSL, fibre) customers worldwide. Orange is one of the main European operators for mobile and broadband Internet services and, under the brand Orange Business Services, is one of the world leaders in providing telecommunication services to multinational companies.
With its industrial project, "conquests 2015," Orange is simultaneously addressing its employees, customers and shareholders, as well as the society in which the company operates, through a concrete set of action plans. These commitments are expressed through a new vision of human resources for employees; through the deployment of a network infrastructure upon which the Group will build its future growth; through the Group's ambition to offer a superior customer experience thanks in particular to improved quality of service; and through the acceleration of international development. For more information, visit http://www.orange.com.
ESnet provides the high-bandwidth, reliable connections that link scientists at national laboratories, universities and other research institutions, enabling them to work together on some of the world's most important scientific challenges including energy, climate science, and the origins of the universe. Funded by the U.S. Department of Energy's Office of Science, and managed and operated by the ESnet team at Lawrence Berkeley National Laboratory (Berkeley Lab), ESnet provides scientists with access to unique DOE research facilities and computing resources, as well as to scientific collaborators including research and education networks around the world.
Source: ESnet and partners
Researchers from the Suddhananda Engineering and Research Centre in Bhubaneswar, India developed a job scheduling system, which they call Service Level Agreement (SLA) scheduling, that is meant to achieve acceptable methods of resource provisioning similar to that of potential in-house systems. They combined that with an on-demand resource provisioner to ensure utilization optimization of virtual machines.
Experimental scientific HPC applications are continually being moved to the cloud, as covered here in several capacities over the last couple of weeks. Included in that rundown, Co-founder and CEO of CloudSigma Robert Jenkins penned an article for HPC in the Cloud where he discussed the emergence of cloud technologies to supplement research capabilities of big scientific initiatives like CERN and ESA (the European Space Agency)...
When considering moving excess or experimental HPC applications to a cloud environment, there will always be obstacles. Were that not the case, the cost effectiveness of cloud-based HPC would rule the high performance landscape. Jonathan Stewart Ward and Adam Barker of the University of St. Andrews produced an intriguing report on the state of cloud computing, paying a significant amount of attention to the problems facing cloud computing.
Jun 19, 2013 |
Ruan Pethiyagoda, Cameron Boehmer, John S. Dvorak, and Tim Sze, trained at San Francisco’s Hack Reactor, an institute designed for intense fast paced learning of programming, put together a program based on the N-Queens algorithm designed by the University of Cambridge’s Martin Richards, and modified it to run in parallel across multiple machines.
Jun 17, 2013 |
With that in mind, Datapipe hopes to establish themselves as a green-savvy HPC cloud provider with their recently announced Stratosphere platform. Datapipe markets Stratosphere as a green HPC cloud service and in doing so partnering with Verne Global and their Icelandic datacenter, which is known for its propensity in green computing.
Jun 12, 2013 |
Cloud computing is gaining ground in utilization by mid-sized institutions who are looking to expand their experimental high performance computing resources. As such, IBM released what they call Redbooks, in part to assist institutions’ movement of high performance computing applications to the cloud.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.