Key Research Challenges in Cloud Computing

I am writing this post to summarize the research challenges that, in my view, need to be tackled to realize the full potential of Cloud Computing. These are the open research issues on cloud computing that should be addressed to create better clouds and so to bridge the existing technology gap in order to foster the adoption of cloud computing.

This is an extract from a post that I wrote in the DSA-Research's blog to describe a keynote at the 3rd EU-Japan Symposium on Future Internet and New Generation Networks.

Cloud Aggregation. Research challenges in the aggregation of resources from diverse cloud providers adding additional layers of service management.

• Novel architectural models for aggregation of cloud providers
• Brokering algorithms for high availability, performance, proximity, legal domains, price, or energy efficiency
• Sharing of resources between cloud providers
• Networking in the deployment of services across multiple cloud providers
• SLA negotiation and management between cloud providers
• Additional privacy, security and trust management layers atop providers
• Support of context-aware applications
• Automatic management of service elasticity

Cloud Management. Research challenges in delivering infrastructure resources on-demand in a multi-tenant, secure, elastic and scalable environment.

• Scalable management of network, computing and storage capacity
• Scalable orchestration of virtualized resources and data
• Placement optimization algorithms for energy efficiency, load balancing, high availability and QoS
• Accounting, billing, monitoring and pricing models
• Security, privacy and trust issues in the cloud
• Energy efficiency models, metrics and tools at system and datacenter levels

Cloud Enablement. Research challenges in enhancing platform infrastructure to support cloud management requirements.

• Technologies for virtualization of infrastructure resources
• Virtualization of high performance infrastructure components
• Autonomic and intelligent management of resources
• Implications of Cloud paradigm on networking and storage systems
• Support for vertical elasticity
• Provision of service related metrics

Cloud Interoperability. Challenges to ensure that the available cloud services can work together and interoperate successfully.

• Common and standard interfaces for cloud computing
• Portability of virtual appliances across diverse clouds providers

SIENA European Roadmap on Grid and Cloud Standards for e-Science and Beyond

This is the third, and last (for the moment), report that I would like to recommend. It nicely complements the other two reports (The Future of Cloud Computing and Priorities to Advance Cloud Computing). SIENA European Roadmap on Grid and Cloud Standards for e-Science and Beyond addresses requirements, technologies, and interoperability and standards for e-infrastructure. I am part of the Editorial Board of this roadmap that in this first version includes use cases and position papers collected for the Cloudscape III event.

Cloudscape III – Driving Europe’s Cloud computing Strategy - took place on the 15 and 16 of March in Brussels offering the perfect opportunity for experts, developers and end-users in the cloud computing space to become part of the cloud standards process. The event, hosted by the EC-funded SIENA initiative, brought a 360° overview on the Cloud computing landscape, covering benefits for enterprise, research and government alongside challenges spanning scalability, data movement, ownership and privacy, security and legal issues, openness and interoperability.

There Will Be a Grid of Clouds

Since 2007, when the term cloud computing came into popularity, we have a vision about the future of Grid and Cloud Computing as fully complementary technologies that will coexist and cooperate at different levels of abstraction in e-infrastructures. Pioneering blog posts like Grid and Clouds are Complementary Technologies, A Virtual Infrastructure Layer for Cluster and Grid Computing or Moving Jobs or Moving the Nodes anticipate the full virtualization of Grid sites, the benefits of hybrid Cloud computing for Grid sites or the use of Grid services to federate Cloud sites. Presentations like From Grids to Cloud Services predict the future of compute Grid infrastructures, from infrastructures for sharing basic Grid services to infrastructures for sharing raw resources. I think that this perspective, controversial at that time, was, in part, result of our participation in pioneering initiatives in Cloud computing, such as RESERVOIR, flagship of the EU-funded projects in Cloud computing, or OpenNebula, widely used open-source technology to build clouds.

Unfortunately, at that time, a large part of the Grid computing community saw Cloud as a threat and not as an opportunity, and tried to undervalue the advantages of bringing Cloud to Grid infrastructures, probably because they were not able to understand the potential of the Cloud, or just "scared of the unknown". However things are now changing. Last week, EGI (European Grid Infrastructure) organized the User Virtualization Workshop to discuss the future of virtualization and cloud computing in the European production infrastructure. The aim of the workshop was to evaluate the challenges to move towards an Infrastructure as a Service model with federation of resource providers. It is only a very first step, but we are very happy to see that EGI is now moving in the right direction and is planning to incorporate virtualization and Cloud computing to enhance Grid computing and to allow other communities to take advantage of the infrastructure. I think this is a first step to adapt to technology and an opportunity to define a leading position in the new infrastructure arena. After all, Grid will be one of the many services that could be executed on the new European Cloud Infrastructure.

World Economic Forum Report on Priorities for Industry and Governments to Advance Cloud Computing

In my last post, I recommended the European Commission report on The Future of Cloud Computing to people defining strategies, developing research lines or exploring market opportunities in Cloud Computing. This time I would like to share a new reading recommendation, the World Economic Forum report on Advancing Cloud Computing: What to Do Now?. This report was the main outcome of a two-year study on the Future of Cloud Computing conducted by World Economic Forum and Accenture.

As part of this study, last December I participated in a Cloud Computing Workshop organized in London by World Economic Forum with senior government representatives, industry experts and academics in Europe to capture perspectives on the development, adoption and deployment of Cloud Computing. This private workshop was the final in-person discussion prior to the launch of the recommendations of the study at the Forum's Annual Meeting in Davos in January 2011. This report is one of the inputs of the consultation process on a European Cloud Computing strategy initiated in Davos by Neelie Kroes some months ago.

The report highlights eight action areas that industry and governments need to address:

• Explore and Facilitate the Realization of the Benefits of Cloud
• Advance the Understanding and Management of Cloud-related Risks
• Promote Service Transparency
• Enhance Accountability across all Relevant Parties
• Ensure Data Portability
• Facilitate Interoperability
• Accelerate the Adaptation and Harmonization of Regulatory Frameworks Related to Cloud
• Provide Sufficient Network Connectivity to Cloud Services

I invite anyone who has an interest in Cloud Computing to read this report. It is worth reading.

European Commission Report on The Future of Cloud Computing

Last year, I participated in a group of experts to work on a report that outlines the future directions of Cloud Computing research. This expert group on Cloud Computing was convened by the European Commission Directorate-General Information Society and Media, Software and Services. The group was moderated by Burkhard Neidecker-Lutz of SAP Research and Keith Jeffery of ERCIM, with Lutz Schubert of HLRS as rapporteur and Maria Tsakali as the responsible EC official.

Although the report was published one year ago, it contains an up-to-date evaluation of the state-of-the-art, the current research and the main technical gaps. The report is available for download at the repository of the Software & Service Architectures and Infrastructures Unit. It is a must-read for people defining Cloud Computing strategies, developing innovative research lines, or exploring emerging market opportunities beyond today’s Clouds.

Call for Contributions to the Cloud Track of ServiceWave 2011

We look forward to your contributions to the Cloud Computing track at ServiceWave 2011, which will be held in Poznan on October 26th-28th. The aim of the track will be to discuss the potential of emerging techniques and technologies to contribute to a European Cloud Computing Strategy. The deadline for contributions is May 16th, 2011. Concrete topics comprise but are not limited to:

• Cloud management and enablement technologies
• Interoperability and portability
• Federation and aggregation of clouds
• Security for clouds
• Efficient energy usage of resources

I am very happy to chair this scientific track with the help and advice of the following recognized experts in the field:

• Rosa M. Badia, Barcelona Supercomputing Center and CSIC, Spain
• Brian Coghlan, Trinity College Dublin, Ireland
• Frederic Desprez, INRIA, France
• Erik Elmroth, Umeå University, Sweden
• Wolfgang Gentzsch, HPC Consultant, DEISA, Germany
• Christian Geuer-Pollmann, Microsoft, Germany
• Seif Haridi, SICS, Sweden
• Dieter Kranzlmüller, LMU & LRZ Munich, Germany
• Marcel Kunze, Karlsruhe Institute of Technology, Germany
• Lucio Grandinetti, University of Calabria, Italy
• Philippe Massonet, CETIC, Belgium
• Ruben S. Montero, Complutense University of Madrid, Spain
• Christine Morin, INRIA, France
• Burkhard Neidecker-Lutz, SAP AG, Germany
• Karsten Oberle, Alcatel-Lucent, Bell-Labs, Germany
• Dana Petcu, West University of Timisoara, Romania
• Antonio Puliafito, University of Messina, Italy
• Omer Rana, Cardiff University, UK
• Domenico Talia, ICAR-CNR & University of Calabria, Italy
• Stefan Wesner, HLRS, Germany
• Yaron Wolfsthal, IBM Haifa Research Laboratory, Israel
• Ramin Yahyapour, TU Dortmund University, Germany

Identifying the Elements of a European Cloud Strategy

The 1st Digital Agenda Assembly, which will be held in Brussels on June 16-17 2011, will include the workshop Towards a Cloud Computing Strategy for Europe: Matching Supply and Demand to help identify the main elements of a European Cloud Strategy and the possible need for public-policy intervention, considering demand and supply side concerns. The workshop, organized by Maria Tsakali and chaired by Megan Richards, is composed of three different panels:

• Cloud Computing: what will Europe do?
• Cloud computing fostering innovation
• Towards cloud interoperability

I will participate in the panel about fostering innovation with cloud computing, where we will discuss the opportunities presented by the cloud and the potential for innovation. Other panelists are senior-management representatives of European companies, organizations and networks, and the European Parliament.

The workshop will build on and broaden the consultation process on a European

Cloud Computing strategy launched by Vice-President Neelie Kroes in the World Economic Forum Davos, 27 January 2011. The strategy includes three main areas:

• First, the legal framework. This concerns data protection and privacy, including the international dimension. It also concerns laws and other rules that have a bearing on the deployment of cloud computing in public and private organisations. And it concerns users' rights insofar as they are provided for by law.
• Second, technical and commercial fundamentals. We want to extend our research support and focus on critical issues such as security and availability of cloud services. As a mediator, the Commission can also play a stronger role in the technical standardisation of APIs and data formats, as well as in the development of template contracts and service level agreements.
• Third, the market. We will support pilot projects aiming at cloud deployment. To really harness the power of public procurement we want to engage with our public sector partners on Member State and regional levels to work on common approaches to cloud computing.

Registration is now open.

Where Will Cloud Computing Take Us?

I am starting this new blog as a medium to share observations, thoughts and opinions about Cloud Computing innovation, research, and strategy. In the recent years I have collaborated with a variety of IT companies, telecoms and governments in the definition of Cloud Computing research programs, strategies, open architectures, and migration plans. I have also participated as speaker in different Cloud Computing events where I have tried to communicate my vision about the data centre as the computer of the future, the IT resources as the next utility, and cloud computing as a source of innovation and competitive advantage. My aim is to share my professional experiences and lessons learned from these collaborations through this blog.

Regarding research in cloud computing, we are still at the beginning of the road, there are still many technology challenges to be researched and adoption barriers to be overcome. Regarding innovation, we are starting to understand the revolutionary impact Cloud Computing could have on the economy and the markets. Cloud computing will play a major role in tomorrow’s world. I like the analogy between the advent of the car a hundred years ago and the increasing importance of cloud computing described by Rolf Harms and Michael Yamartino, head corporate strategists at Microsoft, in their paper The Economics of the Cloud.

"When cars emerged in the early 20th century, they were initially called ―horseless carriages‖. Understandably, people were skeptical at first, and they viewed the invention through the lens of the paradigm that had been dominant for centuries: the horse and carriage. The first cars also looked very similar to the horse and carriage (just without the horse), as engineers initially failed to understand the new possibilities of the new paradigm, such as building for higher speeds, or greater safety. Incredibly, engineers kept designing the whip holder into the early models before realizing that it wasn‘t necessary anymore.

Initially there was a broad failure to fully comprehend the new paradigm. Banks claimed that, ―The horse is here to stay but the automobile is only a novelty, a fad. Even the early pioneers of the car didn‘t fully grasp the potential impact their work could have on the world. When Daimler, arguably the inventor of the automobile, attempted to estimate the long-term auto market opportunity, he concluded there could never be more than 1 million cars, because of their high cost and the shortage of capable chauffeurs."