September 22, 2011
This week at the EGI Technology Forum in Lyon France, leaders from around Europe gathered to discuss the creation of sustainable, broad-reaching research objectives that would address existing user communities, encourage new ones to develop, and find ways to leverage the influx of data to increase scientific discovery and European competitiveness.
Among the many elements on the agenda for the week, the GRDI2020 Vision was the topic of a great deal of conversation. This effort proposes a ten-year plan to create global research infrastructures that can address the needs of data-intensive scientific projects while remaining sustainable. While this was announced last year, the EGI Technical Forum was the locale for a more comprehensive roadmap as Europe looks ahead to 2020.
The roadmap that was presented this week laid out Europe’s vision for a global research data infrastructure that can be the “enabler of an open, extensible and evolvable digital science ecosystem. This will be maintained and created through advances in grid and cloud computing via the use of science gateways (community-specific sets of tools, application and data collections that can be accessed via a portal or application suite) and virtual research environments (VREs). VREs will be the technological framework behind virtual working environments and communities that collaborate via cloud or grid-based portals.
By creating these environments that take advantage of distributed resources, hardware, software and knowledge-wise, GRDI2020 hopes to create an “interoperable science ecosystem” that will reduce data fragmentation and speed access and use of data stores.
This, of course, involves a great deal of coordination of every layer of the technical and scientific community in Europe. From those providing tools and support to create and maintain the grid or cloud-based ecosystem to those governing—and all researchers in between.
The group sees a large number of challenges, however. Issues like international collaboration, laws and policies at odds certainly tops the list, but other problems, including the technical challenges of creating interoperable tools, authentication layers, data movement issues, and more general aspects of distributed computing are present as well.
Despite the practical concerns that could limit progress, GRDI2020 sees cloud computing as one of the leading forces in the march toward their goals. The group claims that they “envision that the future Digital Data Libraries (Science Data Centers) will be based on cloud philosophy and technology” with each community having its own cloud. This is where federation (an issue that EGI lead Steven Newhouse discussed at length this week) comes into play. By federating these clouds, GRDI2020 can approach the vision with increased collaboration to enable multidisciplinary research.
At the heart of this anticipation of the era of data-intensive scientific computing are a number of concerns that the organization hopes to address, including the need to build realistic, scalable research infrastructures to support data-intensive research. This is a program that the European Commission is backing, and that many attendees at the EGI Technology Forum seemed to feel was a critical first step in advancing European research.
As part of these far-reaching efforts, GRDI2020 seeks to spend the next decade creating “a framework for obtaining technological, organizational and policy recommendations guiding the development of ecosystems of global research data infrastructures.” This means leveraging existing user communities, experts, leaders behind large projects and policy makers to help lead to this vision of sustainable global research systems.
The experts behind this initiative say that Europe has entered the “new science paradigm” in which many areas of research are facing a hundred, if not a thousand-fold increase in the amount of data they contend with compared to just ten years ago. This is due to an explosion in the number of sensors and scientific instruments, not to mention the fact that storing the data gathered is now far more affordable than it was ten years ago.
Officials from the GRDI2020 project say “this data deluge can revolutionize the way research is carried out and lead to the emergence of a new fourth paradigm of science based on data-intensive computing.” They say that this new era will lead to a “data-centric” way of thinking about research and solving problems—but that there is a severe lack of infrastructure available to support these opportunities.
GRDI2020’s vision of research data infrastructures requires a great deal of cross-disciplinary collaboration. They define this new way of thinking about infrastructure in the following categories:
• Tools and services that support the whole research cycle
• The movement of scientific data across scientific disciplines (which was an issue that was addressed in detail by the representatives who spoke about GlobusEUROPE and GlobusOnline)
• The creation of open linked data spaces by connecting data sets from diverse disciplines
• The management of scientific workflows
• The interoperation between scientific data and literature
• The development of an integrated science policy framework.
This week nearly every session addressed one or more of these issues—but the actual GDRI statement on the vision for 2020 broke down these generalities—and shed light on progress toward goals.
The efforts from GRDI2020 might serve as ample motivation to other localized areas to create similar policies. For instance, this type of federated vision could easily extend to multiple universities sharing individual disciplinary clouds that are gathered under one roof. However, if there was one thing that became clear this week, it’s that Europe has its act together, organizationally speaking. By creating research infrastructures that filter down from a policy hierarchy and extend into hundreds of sub-branches, there is cohesion—an essential element of any distributed computing or resource-sharing effort.
The GRDI2020 program is funded by the European Commission under the 7th Framework Programme, which designed to boost Europe’s competitiveness through key technology and research investments. It combines all of the research-driven initiatives in the EU together under one organization, spitting the focus between four areas, including cooperation, ideas, people and capacities. As one can imagine this creates a rather complex set of hierarchies under each classification, leading to a wide range of activities that receive funding and support from the program.
To put the work of GRDI2020 in context and see how they are enabling researchers to have better access to needed tools and infrastructure, a good example is below. While data from fisheries might not be every scientist’s cup of tea, his challenges are similar to those in nearly every discipline—needing to contend with massive data sets in a way that promotes quick access, collaboration and thorough host of tools to manage and solve problems.
Jun 17, 2013 |
With that in mind, Datapipe hopes to establish themselves as a green-savvy HPC cloud provider with their recently announced Stratosphere platform. Datapipe markets Stratosphere as a green HPC cloud service and in doing so partnering with Verne Global and their Icelandic datacenter, which is known for its propensity in green computing.
Read more...
Jun 12, 2013 |
Cloud computing is gaining ground in utilization by mid-sized institutions who are looking to expand their experimental high performance computing resources. As such, IBM released what they call Redbooks, in part to assist institutions’ movement of high performance computing applications to the cloud.
Read more...
Jun 06, 2013 |
The San Diego Supercomputer Center launched a public cloud system for universities in the area designed specifically to run on commodity hardware with high performance solid-state drives. The center, which currently holds 5.5 PB of raw storage, is open to educational and research users in the University of California.
Read more...
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.