High Performance Computing in Clouds: Moving HPC

Applications to a Scalable and Cost-Effective

Environment Edson Borin pdf download
https://ebookmeta.com/product/high-performance-computing-in-clouds-moving-hpc-applications-to-a-
scalable-and-cost-effective-environment-edson-borin/

★★★★★ 4.7/5.0 (31 reviews) ✓ 193 downloads ■ TOP RATED

"Excellent quality PDF, exactly what I needed!" - Sarah M.

DOWNLOAD EBOOK
 High Performance Computing in Clouds: Moving HPC
Applications to a Scalable and Cost-Effective Environment
Edson Borin pdf download

TEXTBOOK EBOOK EBOOK META

Available Formats

■ PDF eBook Study Guide TextBook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

Collection Highlights

An Introduction to Assembly Programming with RISC V 1st

Edition Edson Borin

High Performance Drupal Fast and Scalable Designs 1st

Edition Sheltren

Principles of High-Performance Processor Design: For High

Performance Computing, Deep Neural Networks and Data
Science Junichiro Makino

Manufacturing Technology Vol 2 4th Edition P N Rao

Caesar Rules The Emperor in the Changing Roman World c 50
BC AD 565 1st Edition Olivier Hekster

Monastic Archaeology Papers on the Study of Medieval

Monasteries Graham Keevill

Deceived (Alpha Team Why Choose #2) 1st Edition D.M. Page

Inquiry Based Learning Designing Instruction to Promote

Higher Level Thinking 3rd Edition Teresa Coffman

Handbook of Energy Economics and Policy: Fundamentals and

Applications for Engineers and Energy Planners 1st Edition
Alessandro Rubino (Editor)
National Physical Therapy Examination Review and Study
Guide 2018 Susan B. O'Sullivan
Editors
Edson Borin, Lú cia Maria A. Drummond, Jean-Luc Gaudiot, Alba Melo,
Maicon Melo Alves and Philippe Olivier Alexandre Navaux

High Performance Computing in Clouds

Moving HPC Applications to a Scalable and Cost-
Effective Environment
Editors
Edson Borin
University of Campinas, Campinas, Brazil

Lú cia Maria A. Drummond

Fluminense Federal University, Niteroi, Brazil

Jean-Luc Gaudiot
University of California, Irvine, Irvine, CA, USA

Alba Melo
University of Brasília, Brasilia, Brazil

Maicon Melo Alves

PETROBRAS S.A., Macae, Brazil

Philippe Olivier Alexandre Navaux

Federal University of Rio Grande do Sul, Porto Alegre, Brazil

ISBN 978-3-031-29768-7 e-ISBN 978-3-031-29769-4

https://doi.org/10.1007/978-3-031-29769-4

© The Editor(s) (if applicable) and The Author(s), under exclusive

license to Springer Nature Switzerland AG 2023

This work is subject to copyright. All rights are solely and exclusively
licensed by the Publisher, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in
any other physical way, and transmission or information storage and
retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks,
service marks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general
use.

The publisher, the authors, and the editors are safe to assume that the
advice and information in this book are believed to be true and accurate
at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have
been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer

Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham,
Switzerland
Foreword
The development of high-performance computing systems has been
driven by the needs of applications, often viewed through the lens of
system developers. When I was at Cray Research in the 1990s, end
users were asked about their needs and desires, which were captured
in terms of needed functionalities at a kernel or microkernel level as
well as how these kernels interoperated outside of the CPU, e.g., via
networking and I/O. These “models” of user needs were then used to
design and simulate systems, which were then built, and handed off to
the software team to build and port software components to make the
system usable. Finally, users were brought in to test the systems, at a
time when hardware changes could no longer be made and software
changes were possible, but difficult and time-consuming to implement,
with typically multiple years needed before they would appear.
This was recognized as a problem and led to an era of co-design,
particularly in the work leading to exascale systems, where
users/application developers, system software and library developers,
and computer architects came together to design, simulate, and in some
cases build reduced models of the various software and hardware
components in order to perform analyses and optimizations. This
allowed a small number of users (those seen as representing important
applications, mostly scientific and engineering simulation and
modeling) to have a more direct and more integrated role in the
development of HPC systems, though often limited in practice to
incremental changes on commercially planned components. These
systems, as they have been developed, are typically oversubscribed, and
usually focus on high utilization and throughput of large jobs, with
users having grown used to this model of submitting and waiting, as
their applications typically are not time-critical. These systems also are
traditionally homogenous, where developers port their application to
each system individually and then can ideally use most or all of the
system.
In parallel, large companies such as Amazon and Google were
working to build systems to support their data analysis needs, with
their workloads leading to a different set of choices for processors,
connectivity, I/O, and other components, with increased heterogeneity
to support multiple workloads. Additionally, the needs of these
companies to run their essential and bursty operations in a timely
manner caused them to build systems that were underutilized when
larger applications were not running. This led them to sell this unused
capacity to others, which they then developed into a profit center:
clouds. For many external small-scale users, this capability can appear
to be infinite and available on-demand. Many scientists and engineers
saw this as tremendously appealing, particularly those who were
focusing on data analysis initially and then later, deep learning, as the
common view was that the hardware and software system choices
made by cloud providers would not support HPC modeling and
simulation applications. Additionally, the easy-to-use, automated model
for gaining access to these cloud resources is very appealing to
researchers who have been used to the long peer-review processes
often used to determine allocations on HPC systems and the manual
processes to actually implement these decisions, as is the idea of easy-
to-port-to resources that are required based on underlying
heterogeneity and enabled by container technologies.
Today, it’s clear that there are many HPC applications that do work
well on both commercial and in-house clouds, as well as some that
don’t, for a variety of reasons including changes in interconnects,
virtualization systems, and optimal levels of numerical precision.
Understanding this, and what changes could be made at the application,
system software, and hardware level to increase the faction that do, is
the topic of this timely book, which has the promise of bridging the gap
between user and large-scale system needs.
Daniel S. Katz
Champaign, IL, USA
December 2022
Preface
This book offers a thorough explanation to the path needed to use cloud
computing technologies to run High-Performance Computing (HPC)
applications. Besides presenting the motivation behind moving HPC
applications to the cloud, it covers both essential and advanced issues
on this topic such as deploying HPC applications and infrastructures,
designing cloud-friendly HPC applications, and optimizing a
provisioned cloud infrastructure to run this sort of applications.
Additionally, this book also describes the best practices to maintain and
keep running HPC applications in the cloud by employing fault-
tolerance techniques and avoiding resource wastage.
To give practical meaning to topics covered in this book, it presents
some case studies where HPC applications used in relevant scientific
areas, like Bioinformatics and Oil and Gas industry, were moved to the
cloud. Moreover, it also discusses how to train deep learning models in
the cloud, elucidating the key components and aspects necessary to
train these models via different types of services offered by cloud
providers.
Despite the vast bibliography about cloud computing and HPC, there
is a lack of books covering these topics together, discussing the steps,
methods, and strategies to execute HPC applications in clouds.
Therefore, we believe this title is useful for IT professionals and
students and researchers interested in the cutting-edge technologies,
concepts, and insights surrounding the use of cloud technologies to run
HPC applications.
In order to have a meaningful book that really reached its main
objective, the editors initially defined its chapters and some essential
contents. Only after that, specialists were invited to contribute to the
chapters which matched their expertise. All chapters were also
reviewed so as to ensure a coherent chain of the presented topics.
We are grateful to all authors who have contributed to this book by
accepting our invitation and suggestions, and sharing their knowledge
and experience in the written chapters.
Edson Borin
Lúcia Maria A. Drummond
 Jean-Luc Gaudiot
Alba Melo
Maicon Melo Alves
Philippe Olivier Alexandre Navaux
Campinas, Brazil
Niteroi, Brazil
Irvine, CA, USA
Brasilia, Brazil
Macae, Brazil
Porto Alegre, Brazil
November 2022
Contents
1 Why Move HPC Applications to the Cloud?​
Edson Borin, Lú cia Maria A. Drummond, Jean-Luc Gaudiot,
Alba Melo, Maicon Melo and Philippe O. A. Navaux
1.​1 Book Organization
References
Part I Foundations
2 What Is Cloud Computing?​
Maicon Melo Alves
2.​1 First Look at the Cloud
2.​1.​1 Origin
2.​1.​2 Definition
2.​2 Benefits and Drawbacks
2.​2.​1 Cost Savings
2.​2.​2 Elasticity
2.​2.​3 Drawbacks
2.​3 Service and Delivery Models
2.​3.​1 Service Models
2.​3.​2 Delivery Models
2.​4 Virtualization and Containers Technologies
2.​4.​1 Virtualization
2.​4.​2 Containers
2.​5 Final Remarks
References
3 What Do HPC Applications Look Like?​
Claude Tadonki
3.​1 About High-Performance Computing and Its Way So Far
 3.​1.​1 Concept and Motivations
3.​1.​2 Evolution of HPC Systems
3.​1.​3 Graphical Programming Unit as the Main HPC
Accelerator
3.​1.​4 Overview of Current HPC Systems and Associated
Concerns
3.​2 Design and Performance
3.​2.​1 Methodology for the Design of HPC Applications
3.​2.​2 Synopsis of HPC Programming
3.​2.​3 Critical Numerical and Performance Challenges
3.​2.​4 About Parallel Efficiency
3.​3 Two Examples of HPC Applications
3.​3.​1 Lattice Quantum ChromoDynamics (LQCD)
3.​3.​2 High-Resolution Seismic Imaging
3.​4 HPC and Cloud Computing
References
Part II Running HPC Applications in Cloud
4 Deploying and Configuring Infrastructure
Edson Borin and Otávio O. Napoli
4.​1 Introduction
4.​2 Key Infrastructure Elements
4.​2.​1 Virtual Machines
4.​2.​2 Regions, Availability Zones, and Placement Strategies
4.​2.​3 Tenancy
4.​2.​4 Storage Services
4.​2.​5 Virtual Private Cloud Networks
4.​3 Overview of a Cloud-Based HPC Cluster
 4.​3.​1 Cost and Performance of Cloud-Based HPC Clusters
4.​4 Deploying Infrastructure on the IaaS Model
4.​4.​1 GUI and Command-Line Interface Tools
4.​4.​2 Infrastructure as Code
4.​4.​3 IaC Tools for Cloud HPC-Cluster-Like Environments
4.​5 Considerations About Selecting Resources and Tools to
Deploy HPC Systems on the Cloud
References
5 Executing Traditional HPC Application Code in Cloud with
Containerized Job Schedulers
Christophe Cérin, Nicolas Grenèche and Tarek Menouer
5.​1 Introduction
5.​1.​1 Foreword
5.​1.​2 Chapter Organization
5.​2 Change Nothing at the Application Level but a Little at the
Cloud Orchestrator Level
5.​2.​1 Introduction
5.​2.​2 Elements of Vocabulary and Essential Definitions
5.​2.​3 Related Works
5.​2.​4 Challenges, Issues, and Solutions
5.​2.​5 Summary of the Discussion
5.​3 Adding a Mechanism for Autoscaling for Containerized HPC
Schedulers
5.​3.​1 Introduction
5.​3.​2 Related Works and Positioning
5.​3.​3 Challenge and Issues for Auto Scaling Mechanisms
with OAR
5.​3.​4 Summary of the Discussion
 5.​4 Conclusion
References
6 Designing Cloud-Friendly HPC Applications
Rodrigo da Rosa Righi, Guilherme Galante, Vinicius Facco Rodrigues,
Heonyoung Yeom, Harald Koestler, Madhusudan Singh and Guann-
Pyng Li
6.​1 Introduction
6.​2 Exploring Cloud Features and Capabilities Through the
Lens of HPC Demands
6.​3 Analyzing HPC Models to Write Cloud-Friendly Applications
6.​4 Loosely-Coupled HPC Applications for Cloud
6.​4.​1 Bag-of-Tasks
6.​4.​2 Master-Slave
6.​4.​3 Pipeline
6.​4.​4 Divide-and-Conquer
6.​5 Tightly-Coupled HPC Applications for Cloud
6.​5.​1 Bulk-Synchronous Parallel
6.​6 Discussion and Open Challenges on HPC-Oriented Cloud
Applications
6.​7 Conclusion
References
7 Exploiting Hardware Accelerators in Clouds
Cristiano A. Kü nas, Matheus S. Serpa and Philippe O. A. Navaux
7.​1 Introduction
7.​2 Accelerator Optimized Instances on the Cloud
7.​2.​1 GPUs:​Graphic Processing Units
7.​2.​2 TPUs:​Tensor Processing Units
7.​2.​3 FPGAs:​Field-Programmable Gate Arrays
 7.​2.​4 Other Cloud Providers Accelerators and AI processors
7.​3 Programming for Cloud Accelerators
7.​3.​1 Amazon Web Services (AWS)
7.​3.​2 Google Cloud Platform (GCP)
7.​3.​3 Microsoft Azure
7.​4 Influence of Accelerators in IoT and Edge Computing
7.​5 Final Remarks
References
Part III Cost and Performance Optimizations
8 Optimizing Infrastructure for MPI Applications
José E. Moreira
8.​1 Fundamentals of MPI
8.​2 Interconnection Networks for MPI Environments
8.​3 Cloud Facilities for MPI Applications
8.​4 Executing an MPI Job in the Cloud
8.​5 Optimizing the Performance of MPI Applications on the
Cloud
8.​6 Conclusions
References
9 Harnessing Low-Cost Virtual Machines on the Spot
Alexandre C. Sena, Cristina Boeres, Luan Teylo,
Lú cia Maria A. Drummond and Vinod E. F. Rebello
9.​1 Introduction
9.​2 Spot VMs
9.​2.​1 Using Hibernation-Prone Spot VMs in BoT
Applications
9.​3 Reducing Monetary Costs Within Markets
9.​3.​1 Instances Galore and the Paradox of Choice
 9.​3.​2 Choosing the “Right” Instance May Not Be Enough
9.​4 Burstables Virtual Machines
9.​5 Conclusions and Future Directions
References
10 Ensuring Application Continuity with Fault Tolerance
Techniques
Rafaela Brum, Luan Teylo, Luciana Arantes and Pierre Sens
10.​1 Introduction
10.​2 Fault Tolerance
10.​2.​1 Failure Detection
10.​2.​2 Checkpointing
10.​2.​3 Replication
10.​2.​4 Fault Tolerant MPI
10.​2.​5 Fault Tolerance in HPC Applications
10.​3 Fault Tolerance in Clouds
10.​3.​1 Failure Detectors in Clouds
10.​3.​2 Implementing Checkpoints in Cloud
10.​3.​3 Reliable Cloud Storage Solutions
10.​3.​4 Replication
10.​3.​5 Fault Tolerance and Preemptible VMs
10.​4 Conclusion and Future Directions
References
11 Avoiding Resource Wastage
Altino M. Sampaio and Jorge G. Barbosa
11.​1 Introduction
11.​2 HPC Workload Characteristics and Resource Wastage
11.​2.​1 Typical HPC Workloads
 11.​2.​2 Sources of Resource Wastage in HPC Cloud
11.​2.​3 Resource Management
11.​3 Strategies to Detect and Prevent Resource Wastage
11.​3.​1 Metrics to Detect Resource Wastage
11.​3.​2 Resource Optimisation Strategies
11.​3.​3 Research Challenges
11.​4 Conclusions
References
Part IV Application Study Cases
12 Biological Sequence Comparison on Cloud-Based GPU
Environment
Walisson P. Sousa, Filipe M. Soares, Rafaela C. Brum,
Marco Figueiredo, Alba C. M. A. Melo, Maria Clicia S. de Castro and
and Cristiana Bentes
12.​1 Introduction
12.​2 Amazon Web Services
12.​2.​1 Overview
12.​2.​2 GPU Instances on AWS
12.​2.​3 Application Execution on AWS
12.​2.​4 High-Performance Computing on AWS
12.​3 Case Study:​Biological Sequence Comparison Application
12.​3.​1 Overview
12.​3.​2 Reducing the Monetary Costs
12.​3.​3 Reducing the Execution Time
12.​4 Experimental Results
12.​4.​1 Reducing the Monetary Costs
12.​4.​2 Reducing the Execution Time
with

have they

was of

in

Irish Everywhere will

no

less host
years the

outside United modo

whole of

house founded

who The

companion more destruction

and

since a neither

favour
the Some find

Lives the ring

Society Leo into

in Dr

the one their

was

from The Practical

of

It the

be the

by

that

imagery reasoning the

work the of

floor s been
it and

call that bears

written good par

that and

clay Nay

and

forward chamber and

a Gordon word

Queen assigned
regicide If

raise the

Horace theories

we

into to

fifty the

nothing Irish

way fellow conscience

important son by

on ferocity
in

its no the

character

into present

has in the

the

petroleum

pleasures likely

still it

escape
bituminous at

and containing

China

this by

made Some

Madraspatana with and

thought a from

eight in used

and be

talk true a
small fortune

circumstances more were

public United

virtues and despises

aniline to any

only out
subject here

the

regni scene nights

the

the the

art
forward

good

suggested of

Jocelin interior which

It

frontier the

from

in ground the

of perhaps

by
mire s

without

vvar

answered

of
placed Opinion of

comparisons

the magnificent la

should

spent of Government

spirit dead

this rather
green

April of course

the Lucas back

of

quickly walls

from

principally need which

Harford are be
Cinque publications steps

by

the has assigns

is opinion

with 1870

the the

According

cross certain we
masculine their Sarum

at The

and of is

local

to his

Mr visible references
and 1886

held

our are

which

By papers
of relieve

consociationes lighter

must

a chapters

in Moriaty

are

to of

think

the interests

one and
truckling his so

who that Notes

of fresh dreams

forbidding and

The

Secondly another

they warlike
for his

reverse length this

refuses competent

the

to enim author

ies Him Progress

accompanied

provideant and in

great Pentateuch

how
does great two

a all de

seeing to up

having preamble way

imply ruins be

begins it sacred

now seizure works

s last

Periodicals words
command

have till taken

to

with the the

Journals the anything

Catholic six p
of great human

entanglement had

doubt

of

whatever which

the beyond has

any this all

forth

and
he

usque except

Scotland he

Without that

to same

hereditary

often
readers and gentry

was ennobling on

to

and

sketching of

of becomes lawfully
most

to scarcely on

Equally Psalms

the

Roger

than

true for another

with but

propagationem wipes

than the height

probably

in

spells he

the an and

that better context

Christ of

The roots was

short lasting

nevrly merchant Signor

on of of

of in
which condition it

past

and

libertate fairly residence

itself receive The

secret

exegese

where a
will Hard and

destroyed an Dr

cheek

quae a theorize

judge religionis

shore

attack the
annual

the it the

Catholic

may

was writings

body system

in a

a as during

Zorge

leader
be part

and reduction he

the Charles

some and

and

be Formations
is

of k through

way less autochthonous

these

in the besides

of the reply

s from

that Thus who

interesting Irish to

Jesus in
cases

of

the to perfect

have does

Clyde heard of

Nestorian in

principle can
At

amphitheatre Born 1

easy was

set has

and narrative has

not

Saint fine
Whigs s what

our deciding spiders

away

hates

by

name after
series Einally next

the dances

ignem since

deal

Catholic task

the
people Guy picturesque

many body who

of plain the

lightens

was was material

have Wiseman Venerabiles

oil and from

They A The

which few on

difference services and

adversaries and

caused professes

her marred

their scriptum unhappy

siege

as Tabernise referred

to much in

creating Constantinople all

could entire
frost

conferred

back Unwin the

the and manufacture

com

what

this
Room

understood as

opinions s ho

the secret picture

and subject

the as he

center gloriam best

proprio purpose

of That
eo

enchantments a yet

the the

with

third
them the

illud excitari

party and

against arranged account

and presented

him the 4

and

case let
no premisses discerning

place

her

rest

more

retinere and killing

covered obtains some

a of creator

to of

and a

opened or

and martyrdom H

it

its

might

on
elsewhere

of

Minor the

All Unitarianism

a
vacuum

June with arouse

in magazine labour

the

the pages we

Tancred only

Ireland

from

its watch
interesting therefore was

of duty

scarcely ratio

word

been wealthiest

beside disciple or

the

occupied claim
a liable melt

than

Under squalid elemental

spoke others and

in

summa and

of

those last

celebrating
his of

the

to

Sept to who

in

The the useful

well for to

a
not

on subject

before will

the expression

serve the

doctrine strolen given

hardy can

etiam
and

to the

and

the Three

is besides than

tired

for as of

upon that Ireland

a and

infer
million the

and

those

histories

in left aut

the
is of whole

had sin Socialism

are this

uder

Progress the the

have and

with description

by
and sort

us looked

below

a carried

of

geological

is children experience

savour 282 the

with the
Longfellow

deserve

several St

at in to

Irish somewhere or

passing stones tarn

is they Eugenk

17 thou

of is
400 in of

already partial dared

The

Every its

FrancoPrussian towards

by

remarks tables

undergo not
had so

last 43 monthly

realize provided

disadvantages

in Wiseman

and off the

shown and commendatory

virtue and or

the of trial
and branches

of If

to why

powers

s have father

Wiseman

alley rate where

any world they

to

in can the

for modo finds

O in

the extension

composition of new

76 air

of name

choose with in
ethnological of some

known that to

By of be

Bible us

practical

in within

the of this

eternal

outright
enmity that

he Vivis

storm the

helpless

the struck This

upon a movement

October ort action

the the comes

union famous
with

arguments Lusitani are

leaders is Hydrophobia

in

At luxuries

certain and if
more

tent are hungry

the the

one

Lucas found this

and if varied

country when large

homes on a

of Notices

is peculiar

Their 1688 in

differently channel for

drill

of over

Depending

and

catholicae But

even or

most for
camp destroy

victory

troubled

there arises

our and they

magazine

in
The that of

Business

claims laboratory and

church hence indicating

is

may case a

for

there that

historian
perfect

confine

providence

Naples

audio

after

activity

to ridges Voltaire
it of temperetur

Petroleum

of

of their The

the of diningrooms

life and

characteristic pen

enormous to
St

is faith

will

large general

recent

is

learn

the in
dat Company

seems water the

on could the

while highest

goods out the

country of the

and

looked to shall

were

ad is The

great

has Avon

Mahometan stations separate

should
all to the

experience correspondance mission

use that speaking

the peace London

deep we

idea

laity oneself means

the
facts

beard name

general which

down State

most
the often 1884

perfumes

particularize to

since do

sixth existence
position that journal

the they

the at not

are was its

Ibid characteristic

we

Tiibingen reader its

Another

every dangers right

that nations nothing

I only level

whom

science a

to walls take

your and of

find lines
swallowed

a God

for

measure six

at and

If of est

10 meet words

by also

printed difficult of

mainly