October 29, 2010
This week, Dan Reed, Corporate Vice President of Microsoft’s eXtreme Computing Group and Technology Strategy and Policy reiterated the growing demands for more effective ways for scientists to manage the vast influx of data that is streaming in from the ever-growing number of sensors and instruments. He stated that, “In all domains, scientists and researchers are drowning in data. They’ve gone from scarcity to an incredible richness, necessitating a sea change in how they extract insight from all this data.”
In addition to the mounting data, there are also concerns about how data can more seamlessly intersect across disciplines. Since scientists have work that usually overlaps with research from other fields, making the integration of various projects and models as intuitive as possible should be a primary concern.
To highlight this point, consider the case of the Gulf oil spill where scientists were tasked not only with looking for solutions, but what the spill could mean for oil distribution in the water and the subsequent impact on localized ecosystems. This goes, of course, far beyond using computational fluid dynamics to examine the problem myopically—it involves thousands of researchers in many areas all working together to solve the same problem, but in compartmentalized ways.
When a problem like an oil spill or other major event arises, the moment for easy-to-deploy solutions is more critical than ever, however, as Reed notes, “unfortunately, [this requires] many researchers to assume additional systems administrator roles. These researchers often spend inordinate amounts of time maintaining the computing systems they require to do their research rather than devoting their time and talents to the research itself.”
With the rising cost, both in terms of scientist efforts to maintain the needed systems as well as absorbing the costs of general maintenance of massive machines, there is no doubt that a more streamlined system for scientists that removes the complexity of use and allows them to shed the operational expense burden would be of incredible value.
Reed states that, “fortunately, the emergence of cloud computing, coupled with powerful software on clients, such as local desktop computer, offers a solution to this conundrum.” Since the cloud is maintained off-site and kept up to speed with the latest updates and fixes as needed, scientists are freed from the burden and furthermore, the cost issue is mitigated as scientific users can use a “pay as you go” paradigm to maximize efficiency (power and cost) for peak loads and easily “shut down” or drastically reduce use when demand is lessened.
What this means in the context of a major event that requires scientists across disciplines to quickly build or deploy models to collaborate on solutions is that “organizations can buy just-in-time services to process and exploit data, rather than on infrastructure” which lowers the cost barrier of entry, speeds the time to solution, and allows scientists to focus on science rather than the systems they require.
The number of devices that scientists use as instruments of one kind or another, including mobile applications they use as they gather data, iPads, or standard laptops, are growing in number, adding to the data deluge but refining the way scientists gather data in near real-time. As Reed sees it, “the cloud offers unique opportunities to support a global, multi-party and neutral type of collaboration—allowing a diverse set of experts scattered across multiple contents to bring their expertise to bear…By extending the capabilities of powerful, easy to use PC, Web, and mobile applications through on-demand cloud services, the capabilities of the entire research community will be broadened significantly, accelerating the pace of engineering and scientific discovery.”
As we often try to communicate here by highlighting use cases of cloud for HPC and as Reed echoes “the net effect will be the democratization of research capabilities that are now available only to the most elite scientists.”
Full story at TechNet
Researchers from the Suddhananda Engineering and Research Centre in Bhubaneswar, India developed a job scheduling system, which they call Service Level Agreement (SLA) scheduling, that is meant to achieve acceptable methods of resource provisioning similar to that of potential in-house systems. They combined that with an on-demand resource provisioner to ensure utilization optimization of virtual machines.
Experimental scientific HPC applications are continually being moved to the cloud, as covered here in several capacities over the last couple of weeks. Included in that rundown, Co-founder and CEO of CloudSigma Robert Jenkins penned an article for HPC in the Cloud where he discussed the emergence of cloud technologies to supplement research capabilities of big scientific initiatives like CERN and ESA (the European Space Agency)...
When considering moving excess or experimental HPC applications to a cloud environment, there will always be obstacles. Were that not the case, the cost effectiveness of cloud-based HPC would rule the high performance landscape. Jonathan Stewart Ward and Adam Barker of the University of St. Andrews produced an intriguing report on the state of cloud computing, paying a significant amount of attention to the problems facing cloud computing.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.