Opportunities for HPC in the Cloud

Author: 
Philippe Olivier Alexandre Navaux, professor at the Universidade Federal of Rio Grande do Sul and Computer Science Area Director of CAPES
Focus Area: 

High Performance Computing (HPC) in Cloud environments

Who stands to benefit and how : 

The benefit is directly to the HPC community, because the use of Cloud for HPC in an easy way will enable that more users have access to the computational power needed by their research

Position Paper: 

High-Performance Computing (HPC) researchers found in Cloud Computing a viable alternative to execute their workloads instead, or expanded the capability, of a cluster or a supercomputer. Several applications were successful migrated to the Cloud, such as weather prediction numerical models, genomics, seismic simulation, among others. The main benefits of Cloud Computing for HPC are the absence of up-front costs and the elasticity of the environments, allowing that the user adapts the environment, adding more machines and deleting them when they are not useful, and pays just for the amount of resources that were actually used. However, normally the applications are migrated to the Cloud "as is" without modifications that can take advantages of the cloud model.

One example of adaptation that can be made is the input data manipulation and sharing, in an application that simulates several situations depending of the input parameters, it is possible to configure the data to execute all the simulations at the same time with little variation of the application and this could benefit the user with faster results. Another aspect that needs more research is the cloud heterogeneity and the application needs. Cloud providers has different instance types, with different characteristics among them, such as processing and interconnection speeds, memory amount, as well as varying prices. Furthermore, the tasks of high-performance parallel applications show different computational needs, because they are not completely homogeneous. These differences can be exploited for improving the cost efficiency of high-performance parallel applications in cloud environments by mapping the application requirements to specific instance types.

Our position is that the new HPC applications need to be build with the Cloud model in mind, in this way the user can benefit from the all potential of modern environments. However the actual HPC applications need to be adapted to the Cloud model and a way that this could be made is by adapting the Input and Output methods of the application, in this way it is possible to benefit from almost all of the relevant aspects of Cloud model without major changes in the core of the application. Moreover, there is a lack of tools and methodologies that enables the migration of HPC application to the Cloud model. There is no libraries that help the user to convert the file format from the legacy application to the new cloud file capabilities, the automatic scalability that some cloud providers offer is not explored without big changes in the application as well. We think that there is room for improvements that will enable massive cloud adoption by the HPC community, at the same time there are some interesting research themes that will help both the cloud providers and the community, such as tools to migrate applications, models of execution, file format converters and HPC schedulers to enable scalability in an automatic way for HPC in the cloud.

EU EUBrasilCloudFORUM is funded by the European Commission under the Cooperation Programme, Horizon 2020 grant agreement No 689495. Este projeto é resultante da 3a Chamada Coordenada BR-UE em Tecnologias da Informação e Comunicação (TIC), anunciada pelo Ministério da Ciência, Tecnologia e Inovação (MCTI), no âmbito to acordo de subvenção Número 689495. Privacy Policy | Disclaimer Terms of use