OpenStack Compute Management

Foreword

⚫ This course describes the positioning and functions of the OpenStack

Dashboard service (Horizon), its interactions with other services,
architecture, and GUIs. It also describes the working principles and
processes of Nova and typical operations.

2 Huawei Confidential
 Objectives

⚫ Upon completion of this course, you will understand:

 The positioning and functions of Nova in OpenStack and its interactions with
other services.
 The architecture, components, and working principles of Nova.
 The typical operations in Nova.

3 Huawei Confidential
 Contents

1. Nova Overview

2. Nova Architecture

3. Nova Working Principles and Processes

4. Nova Typical Operations

4 Huawei Confidential
 Compute Service: Nova

• Provides massively • First appeared in the

scalable, on-demand, and OpenStack Austin release.
self-service access to • Depends on Keystone
compute resources.
• Manages bare metal
Nova (OpenStack Identity service),
Neutron (OpenStack Network
servers, VMs, and service), and Glance
containers. (OpenStack Image service).

5 Huawei Confidential

• Nova provides compute, storage, and network services in the initial OpenStack
releases.

• Now, Nova provides only the compute service, depending on Keystone

(OpenStack Identity service), Neutron (OpenStack Network service), and Glance
(OpenStack Image service).
 Positioning of Nova in OpenStack

Nova
• Nova is a core module of
OpenStack and provisions
compute resources.
• Nova does not include
virtualization software.
Instead, it defines drivers
that interact with
underlying virtualization
mechanisms that run on
your host operating system,
and exposes functionality
over a web-based API.

6 Huawei Confidential

• As shown in the figure, Nova belongs to the compute service layer. Users can use
Horizon, Nova clients, APIs, or CLIs to create and manage compute instances.
 Mission and Functions of Nova
Mission: Implement services and associated libraries to provide massively scalable, on demand, self-
service access to compute resources.

What does Nova • VM lifecycle management

• Lifecycle management of other compute
do? resources

What does Nova • Managing the physical hosts VMs run on

not do? • Comprehensive system status monitoring

7 Huawei Confidential

• Nova is the core of OpenStack and has the following characteristics:

▫ It is one of the first two OpenStack projects.

▫ It provides the most complex functions and has the largest code size among
OpenStack projects.

▫ Most integration projects work with Nova.

▫ Its contributors have the greatest influence in the community.

Interactions with Other Services

• Invokes the image services

provided by Glance. Users
create compute instances
• Works with Keystone using Nova.
to provide
authentication services.
Nova • When a compute instance
created using Nova is
started, Neutron connects
the instance to a virtual or
physical network.

8 Huawei Confidential
 Contents

1. Nova Overview

2. Nova Architecture

3. Nova Working Principles and Processes

4. Nova Typical Operations

9 Huawei Confidential
 Nova System Architecture

⚫ Nova is composed of multiple

server processes, each
performing different functions.
⚫ The user-facing interface is a
REST API, while internally Nova
components communicate via an
RPC message system.

10 Huawei Confidential

• DB: SQL database for data storage.

• API: Component that receives HTTP requests, converts commands and

communicates with other components via the [Link] queue or HTTP.

• Scheduler: Decides which host gets each instance.

• Compute: Manages communication with hypervisor and virtual machines.

• Conductor: Handles requests that need coordination (build/resize), acts as a

database proxy, or handles object conversions.

• Placement: Tracks resource provider inventories and usages.

• RPC: Remote Procedure Call (RPC) is a computer communication protocol that

allows programs running on a computer to invoke sub-programs on another
computer. The programmer does not need to program for the interaction.

• The API servers process REST requests, which typically involve database
reads/writes, optionally sending RPC messages to other Nova services, and
generating responses to the REST calls.

• RPC messaging is done via the [Link] library, an abstraction on top of

message queues.

• Nova uses a messaging-based, "shared nothing" architecture and most of the

major Nova components can be run on multiple servers, and have a manager
that is listening for RPC messages.
Physical Deployment of Instances
HTTP Load Balancer
⚫ The architecture is decentralized.

⚫ Components are not locally persistent.

Controller Controller Controller
Nova-API
Nova-Scheduler
Nova-API
Nova-Scheduler ... Nova-API
Nova-Scheduler
⚫ Horizontal scalability is supported.
Nova-Conductor Nova-Conductor Nova-Conductor
... ... ... ⚫ Generally, Nova-API, Nova-Scheduler, and
Nova-Conductor components are deployed
on the controller nodes.
MySQL RabbitMQ ⚫ Multiple controller nodes are deployed for
HA and load balancing.

⚫ The system capacity can be expanded by

Compute Compute Compute adding controller nodes and compute nodes.
Nova- Nova- ... Nova-
Compute Compute Compute

11 Huawei Confidential
 Nova Service Architecture
Nova-API
API Layer
Nova-API Nova-API-Cell Nova-API-EC2

Conductor and Nova-Scheduler

AMQP Queue
Scheduler Layer Service Database
Nova-Conductor

Hypervisor Layer Nova-Compute Nova-Compute Nova-Compute

Nova-Compute Nova-Compute Nova-Compute
...
Virtual Infrastructure VMware VC driver Xen driver KVM libvirt driver

vCenter

Host Aggregate/Group Host Aggregate/Group Host Aggregate/Group

12 Huawei Confidential

• The Nova components can be distributively deployed and can be connected to

different virtualization platforms using virtDriver.
 Nova Resource Pool Management
⚫ Region > Availability Zone > Host Aggregate

Region VM m VM m
VM 1 VM n VM x VM 1 VM n VM x

Nova-Compute Nova-Compute Nova-Compute Nova-Compute

Controller Node Host Aggregate 1 Group 1
Host Aggregate 2
Availability Zone 1
Nova-API
Nova-Scheduler
Nova-Conductor
...
...
MySQL
RabbitMQ VM 1 VM n VM m VM x VM 1 VM n VM m VM x

Nova-Compute Nova-Compute Nova-Compute Nova-Compute

Host Aggregate 1 Group 1
Host Aggregate 2
Availability Zone 2

13 Huawei Confidential

• Region > Availability Zone > Host Aggregate

 Nova Components - API
WSGI WSGI
Middleware Application

Request

WSGI WSGI
Client WSGI Server
Middleware Application

Response

WSGI WSGI
Middleware Application

⚫ Nova-API:  Verifies requested resource quotas and reserves

resources accordingly.
 Receives and processes requests from external
systems over RESTful APIs.  Creates, updates, deletes, and queries resources.
 Validates and restrains the transferred parameters.  Provides the entry for VM lifecycle management.

14 Huawei Confidential

• WSGI: Web Server Gateway Interface

 Nova Components - Conductor
Nova-API Nova-Conductor Nova-Scheduler Nova-Compute

Database operations

AMQP Queue
Database Service

⚫ Nova-Conductor:
 Provides database operations and decouples Nova-Compute and the database.
 Provides complex process control, such as creation, cold migration, live migration, VM flavor
adjustment, and VM rebuilding.
 Provides dependencies of other components. For example, Nova-Compute can be started only after
Nova-Conductor is successfully started.
 Periodically writes the heartbeats of other components.
15 Huawei Confidential

• Nova-Conductor has the following advantages:

▫ Security: In the past, each nova-compute component directly accessed

databases. Once a compute node is attacked, the attacker can obtain all
permissions to access the database and wantonly operate it.

▫ Easy upgrade: nova-compute is decoupled from the database. In this case,

nova-compute does not need to be upgraded even if the database mode
changes.

▫ Performance: In the past, nova-compute directly accessed the database,

blocking database access. Because nova-compute has only one OS thread,
the access of a green thread to the database will block other green threads.
As a result, green threads cannot concurrently access the database.
However, nova-conductor uses remote procedure calls (RPCs) that are
green-thread friendly. Before an RPC returns the result, it does not block
the execution of other green threads. This improves the access concurrency.
 Nova Components - Scheduler
⚫ Nova-Scheduler:

Host 1 Host 1
 Filters physical machines and
The memory is
weighed by default. then selects where the VM
Host 2 Host 2 Host 5
instance will be deployed.
Filtering Weighting  Assigning a VM instance
Host 3 Host 3 Host 3

Filters Weighting includes two steps:

Host 4 Host 4 Host 1 ◼ Use a filter to select the
compute nodes meeting the
Host 5 Host 5 Host 6
criteria.

Host 6 Host 6
◼ Select the best node based on
Hosts chosen after
filtering and sorted after the weight.
weighting (here the best
variant is Host 5)

16 Huawei Confidential

• Nova-Scheduler determines the physical machine where the VM is to be

assigned. This process mainly includes two steps: filtering and weighting. When
creating a VM, users raise their requirements on resources, such as CPUs,
memory, and disks. OpenStack then defines these requirements in the flavor, and
users only need to specify the flavor they want when creating a VM.

• The scheduling process is as follows:

▫ Use a filter to select the compute nodes meeting the criteria.

▫ Select the optimal node based on the weight.

 Nova Components - Compute
Operations on VMs Periodic tasks Driver management  Nova-Compute framework
VM lifecycle Resource APIs of other ◼ Manager
(creation, deletion, and management components
migration) ◼ Driver
Network
information update
Other VM operations
(mounting disks and VM status
 Virtualization platforms
NICs) synchronization
connected with Nova-Compute
Nova-Compute (Manager)
...
◼ KVM
◼ VMware
Hyper-V Xen VMware Libvirt
◼ Xen
VM VM VM
KVM LXC QEMU
◼ LXC
VM VM VM
VM VM VM ◼ QEMU
VM VM VM
VM VM VM ◼ ...
VM VM VM

17 Huawei Confidential

• The executor of VM lifecycle operations invokes the driver of the corresponding

hypervisor.

• The underlying layer connects to different virtualization platforms, such as KVM,

VMware, Xen, and Ironic.

• Built-in periodic tasks are used to update resources and synchronize the VM
status.

• The resource management module (resource_tracker) works with the plug-in

mechanism to collect statistics on resources.
 Contents

1. Nova Overview

2. Nova Architecture

3. Nova Working Principles and Processes

4. Nova Typical Operations

18 Huawei Confidential
VM States

⚫ VM states: ⚫ Relationships between states:

 vm_state: indicates the VM state recorded in the  The system records only vm_state, task_state, and
database. power_state.
 task_state: indicates the current state of a VM task.  The result of status is generated based on both
The value can be Intermediate or None. vm_state and task_state.
 power_state: indicates the VM state obtained from ⚫ Example:
the hypervisor.  If the value of vm_state is active and that of
 status: indicates the VM state displayed externally. task_state is rebooting, REBOOT will be displayed
for status.
 If the value of vm_state is building, BUILD will be
displayed for status.
19 Huawei Confidential
VM States
vm_state task_state status vm_state task_state status
rebooting REBOOT resize_prep RESIZE
reboot_pending REBOOT
resize_migrating RESIZE
reboot_started REBOOT
stopped resize_migrated RESIZE
rebooting_hard HARD_REBOOT
reboot_pending_hard HARD_REBOOT resize_finish RESIZE
reboot_started_hard HARD_REBOOT default SHUTOFF
rebuild_block_device_mapping REBUILD
active rebuilding REBUILD
rebuild_spawning REBUILD
migrating MIGRATING
resize_prep RESIZE
resize_migrating RESIZE
resize_migrated RESIZE
resize_finish RESIZE
default ACTIVE

20 Huawei Confidential
 VM State Transitions
VM state transitions allowed by OpenStack

21 Huawei Confidential

• For details about the VM states and task states for various commands issued by
users, see [Link]
[Link]?highlight=vm.
 Nova VM Creation Process
1
Horizon/CLI 2
Keystone
keystone db
3 4 21
5
Glance
Nova nova-api
glance-api glance-registry
6 20
10
Image store db
7 8 11
9 22
nova-
db MQ scheduler
15 Neutron neutron-server
12 24
17 16 18
13 14/19 MQ
23 neutron-plugins neutron-agent
nova-conductor
25 db
nova-compute

Cinder cinder-api
29 27
26 28
Hypervisor cinder-volume MQ cinder-scheduler

volume-backend db

22 Huawei Confidential

• Step 1: The Horizon Dashboard or OpenStack CLI gets user credentials and
authenticates with Keystone via the RESTful API.
• Step 2: Keystone authenticates the user with the user credentials and then
generates and sends back an auth-token.
• Step 3: The Horizon or CLI sends a boot instance request, which carries the auth-
token, to nova-api over the RESTful API.
• Step 4: nova-api gets the request and sends that request to Keystone for
validation of the auth-token and access permission.
• Step 5: Keystone validates the token and sends the updated authentication
headers with roles along with the permissions. (Note: Some operations require
role permissions.)
• Step 6: After getting the response from Keystone, nova-api interacts with nova-
database.
• Step 7: nova-api creates initial database entry for the new instance or VM.
• Step 8: nova-api sends the [Link] request to nova-scheduler expecting to get
updated instance entry with Host ID specified.
• Step 9: nova-scheduler picks the request from the queue.
• Step 10: nova-scheduler talks to nova-database to locate an appropriate host
using the filtering and weighting mechanism.
• Step 11: nova-scheduler returns the updated instance entry with the appropriate
host ID after filtering and weighting.
• Step 12: nova-scheduler sends the [Link] request to nova-compute for
launching an instance on the appropriate host.

• Step 13: nova-compute picks the request from the queue.

• Step 14: nova-compute sends the [Link] request to nova-conductor to get the
VM or instance information.

• Step 15: nova-conductor picks the request from the queue.

• Step 16: nova-conductor interacts with nova-database based on the request

message.

• Step 17: nova-conductor gets the instance information from nova-database.

• Step 18: nova-conductor sends the instance information to the queue.

• Step 19: nova-compute picks the instance information from the queue.

• Step 20: nova-compute sends an HTTP request using the auth-token obtained
from the Keystone RESTful API to glance-api to get the image required for
creating the VM.

• Step 21: glance-api validates the auth-token with Keystone.

• Step 22: After the token is verified, nova-compute gets the image URL.

• Step 23: nova-compute sends an HTTP request using the auth-token obtained
from the Keystone RESTful API to neutron-server to get the network required for
creating the VM.
• Step 24: neutron-server validates the auth-token with Keystone.

• Step 25: After the token is verified, nova-compute gets the network information.

• Step 26: nova-compute sends an HTTP request using the auth-token obtained
from the Keystone RESTful API to cinder-api to get the persistent storage
required for creating the VM.

• Step 27: cinder-api validates the auth-token with Keystone.

• Step 28: After the token is verified, nova-compute gets the block storage
information.

• Step 29: nova-compute generates data for the hypervisor driver and executes the
request on the hypervisor.
Nova Scheduling Process

Conductor Scheduler Placement

Create RequestSpec object

Submit RequestSpec to Submit resource requirements Query which RPs will represent compute
scheduler to placement nodes to satisfy requirements

Create a HostState object for

Return list of resource providers and their
each RP returned from
corresponding allocations to scheduler
Placement

Create allocations against selected compute

Filter and weigh results
node

Return a list of selected host &

Submit list of suitable
alternates, along with their
hosts to target cell allocations, to the conductor

25 Huawei Confidential
Nova Filter Scheduler

Host 1 Cost Cost Host 4 ⚫ Hosts are allocated

Cost Weight 1=12 based on their weights.
The weights are sorted
Host 2 Cost Cost Host 1 based on available
Cost Cost Weight 2=87
resource space by
default.
Host 3 Cost Cost Weight 3=23 Host 3
Cost Cost Cost ⚫ All weights are stored
in the
Host 4 Cost
Cost Weight 4=10 Host 6 nova/scheduler/weights
directory. Currently, the
Cost default weigher is
Host 5 Cost Cost Weight 5=56 Host 5 RAMWeigher. The
weights are calculated
CostCost based on the idle
Host 6 Cost Cost Weight 6=40 Host 2 memory of the
compute nodes. The
more the idle memory,
the higher the weight.
Hosts from Weights- The instance will be
Costs of the hosts capabilities Sorted list of deployed on the
the pool of sums of
relative to the request specifications hosts compute node with the
hosts costs
most idle memory.

26 Huawei Confidential
 Live Migration Principles
Conductor Source Destination
call
check can live migrate destination
call
check can live migrate source

cast
live migrate
call
pre live migration (set up dest)

driver.live_migration (success)
post live migration (clean up source)
call post live migration at destination
(finish dest)

driver.live_migration (failure)
rollback live migration remove volume connections
call
cast rollback live migration at destination

27 Huawei Confidential

• After the migration is successful, information about the source node will be
deleted.

• If the migration fails, a rollback is performed and the information about the
destination node is cleared.
 Contents

1. Nova Overview

2. Nova Architecture

3. Nova Working Principles and Processes

4. Nova Typical Operations

28 Huawei Confidential
Nova Typical Operations
Category Description
VM creation, deletion, startup, shutdown, restart, rebuilding, flavor change, pause, pause
VM lifecycle cancellation, suspension, resume, migration, online migration, locking, unlocking,
management evacuation, rescue, unrescue, shelving, shelving deletion, shelving restoration, backup, VM
image export, as well as list, details, and information query and password change.
Volume and snapshot These operations are essentially the encapsulations of Cinder APIs and include creating or
management deleting a volume or snapshot, listing volumes or snapshots, and querying their details.
Attaching or detaching a volume to or from a VM, and querying VM the volume list and
Operations on volumes
details.
Operations on These operations are essentially the encapsulations of Neutron APIs and include creating
networks or deleting a virtual network, and querying the virtual network list and details.

Operations on NICs Attaching or detaching a NIC to or from a VM, and querying the VM NIC list.

These operations are essentially the encapsulations of Glance APIs, and include creating or
Operations on images
deleting an image, as well as querying the image list and details.
Operations on other
Operations on flavors, host aggregates, key pairs, and quotas.
resources

29 Huawei Confidential
Major Operation Objects of Nova (1)
Name Description Remarks

Server VM The most important data object in Nova.

It is used to add additional description information in key-value
Server metadata VM metadata
format to a VM.
It is used to define the VM type, for example, a VM with two
vCPUs, 4 GB memory, and 40 GB local storage space. A flavor is
Flavor Flavor template of a VM
created by the system administrator and used by common users
for VM creation.
It is used to specify the limit of the logical resources that can be
Quota Resource quota
used by a tenant.
For virtualization technologies such as KVM and Xen, a node
Hypervisor/node Node corresponds to a physical host. For vCenter, a node corresponds
to a cluster.
For virtualization technologies such as KVM and Xen, a host
Host Host corresponds to a physical host and also a node. For vCenter, a
host corresponds to a set of vCenter environment.
A host aggregate contains multiple hosts. Physical hosts in a
Host aggregate Host aggregate host aggregate have the same physical resource features, for
example, the CPU model.

30 Huawei Confidential
Major Operation Objects of Nova (2)

Name Description Remarks

VMs in the same affinity group are scheduled to the same
A VM affinity or anti-affinity
Server group physical host during creation. VMs in the same anti-affinity
group
group are scheduled to different physical hosts during creation.
It manages the status of Nova-related services, including nova-
Service A Nova service compute, nova-conductor, nova-scheduler, nova-novncproxy,
nova-consoleauth, and nova-console.
It is a block storage device used to describe the information
BDM Block device mapping
about the storage device of a VM.
Image Image file It contains the operating system and is used to create VMs.

31 Huawei Confidential
 Quiz

1. (Single-answer question) Which of the following Nova processes is responsible for

managing the VM lifecycle?
A. Nova-API

B. Nova-Compute

C. Nova-Conductor

D. Nova-Scheduler

32 Huawei Confidential

• 1. B
 Summary

⚫ This course described the positioning, functions, architecture, and working

principles of the OpenStack Compute service (Nova), its interactions with
other services, and typical operations, helping you further understand
instance provisioning in OpenStack.

33 Huawei Confidential
 More Information

⚫ OpenStack Community
 [Link]

34 Huawei Confidential
Acronyms
⚫ API: Application Programming Interface (API) is a particular set of rules and specifications
that are used for communication between software programs.
⚫ CLI: Command-Line Interface (CLI) is a means of communication between a program and its
user, based solely on textual input and output. Commands are input with the help of a
keyboard or similar device and are interpreted and executed by applications. Results are
output as text or graphics to the interface.
⚫ EC2: Elastic Compute Cloud (EC2) is a web service system developed by Amazon, which
allows users to rent applications to run their own VMs.
⚫ HTTP: Hypertext Transfer Protocol (HTTP) is an application-layer protocol used for
communication between web servers and browsers or other programs.

35 Huawei Confidential
Acronyms
⚫ KVM: Kernel-based Virtual Machine (KVM) is an open-source virtualization technology built
in Linux. Specifically, KVM helps users turn Linux into a hypervisor that enables a host
computer to run multiple isolated virtual environments, that is virtual clients or VMs.
⚫ MQ: A distributed message queue (MQ) is used for message transmission for each service
and service management. The MQ service can transmit messages inside a zone or across
zones and must be scaled up horizontally to support large-capacity concurrent message
requests and provide software development kits (SDKs) for other services.
⚫ QEMU: The Quick Emulator (QEMU) is an emulation processor, written by Fabrice Bellard,
that distributes source code under a general public license (GPL).
⚫ Web: World Wide Web (Web) is a global, interactive, dynamic, cross-platform, distributed,
graphical information system based on the hypertext and HTTP.
36 Huawei Confidential
Acronyms
⚫ WSGI: Web Server Gateway Interface (WSGI) is an interface that specifies how web
servers should forward requests to Python applications or frameworks.
⚫ Xen: A Xen hypervisor is an open source VMM that adopts the paravirtualization
technology and implements CPU scheduling and memory allocation among VMs.
The Xen hypervisor virtualizes a hardware layer and controls VM operations, but
does not deal with the network, storage device, video, and other input/output (I/O).

37 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2022 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.