CentOS Dojo 2013




KVM Optimizations and Best Practices
    From the Desktop to the Datacenter


    Jaime Melis (opennebula.org)
Summary
●   Tools
●   Drivers
●   Image Backends
●   CPU
●   Memory
●   Networking
●   Desktop Sharing
●   Cloud point-of-view Challenges
Tools
●   Libvirtd !!!!!!
●   virt-manager
●   virt-install
●   qemu-img
●   virt-alignment-scan (libguestfs)
●   lstopo
●   OpenNebula
Drivers
●   Easy... use virtio!!!
●   Huge performance gain
●   No emulation
●   Cooperation with the hypervisor
●   Integrated into the linux kernel
Image Backends
●   Regular file
●   Qcow2
    –   snapshots (beware of the sync!)
    –   additonal layer, less performance
    –   compressed
●   LVM → block device (san storage)
●   Virt-alignment-scan
●   Cache
    –   Writethrough (host page on, guest disk write cache off)
    –   Writeback (host page on, guest disk write cache on)
        ●   Good overall I/O Performance
    –   None (host page off, guest disk write cache on)
        ●   Good write performance
CPU - pinning
●   NUMA (Non-Uniform Memory Access)
●   Shared cache ~ 15% improvement
●   Bad for different tasks
●   Processor availability
●   virsh capabilities
●   lstopo
CPU - pinning

<cputune>
  <vcpupin   vcpu="0"   cpuset="1-4,^2"/>
  <vcpupin   vcpu="1"   cpuset="0,1"/>
  <vcpupin   vcpu="2"   cpuset="2,3"/>
  <vcpupin   vcpu="3"   cpuset="0,4"/>
  ...
</cputune>
CPU - cgroups
●   Limit, account and
●   Bad for different tasks
●   Processor availability
●   virsh capabilities
●   lstopo
CPU - cgroups
●   Limit, account and isolate resource usage



     <cputune>
       <shares>2048</shares>
       ...
     </cputune>
CPU - model
● Subset of features
● Increase performance

● Nested virtualization

● /usr/share/libvirt/cpu_map.xml




    <cpu match='exact'>
       <model fallback='allow'>core2duo</model>
       <vendor>Intel</vendor>
       <topology sockets='1' cores='2' threads='1'/>
       <feature policy='disable' name='lahf_lm'/>
     </cpu>
Memory - KSM
●   Kernel Samepage Merging
●   Combines memory private pages
●   Very useful for VMs !!!!
●   Increases VM density
●   Enable by default
Networking7
●   MacVTap (direct)... NO!
●   Bridged networking... way to go!
●   Disable STP
●   Optimal configuration
●   NAT (masquerading)
●   iptables
Desktop Sharing
●   VNC
●   SPICE
    –   qlx driver
    –   Redirect printers
    –   Usb (mass-storage)
    –   Audio
Cloud Challenges
●   Virt-manager + LVM + DHCP => 10 hosts
●   Live-migration to any host
●   Image layout – don't waste space
●   Contextualization: install once and deploy
    many
●   Network Isolation
●   Mac/IP spoofing
●   Multi-tenancy
Cloud Challenges - ebtables
●   Isolate
●   Prevent mac spoofing

    # Drop packets that don't match the network's MAC Address
    -s ! <mac_address>/ff:ff:ff:ff:ff:0 -o <tap_device> -j DROP

    # Prevent MAC spoofing
    -s ! <mac_address> -i <tap_device> -j DROP
Cloud Challenges – What storage?
●   No Storage Holy Grail

                        LVM2   iSCSI   Qcow2 + DFS

    Provisioning time   +      ++      +++

    I/O                 ++     ++      +

    Network             ++     +++     +

    Snapshotting        -      -       +

    Livemigration       +      -       +
Thanks for attending!

jmelis - #opennebula @Freenode

More Related Content

PDF
Optimization_of_Virtual_Machines_for_High_Performance
ODP
LSA2 - 01 Virtualization with KVM
PDF
Achieving the Ultimate Performance with KVM
PDF
Look Into Libvirt Osier Yang
PDF
LCE13: Virtualization Forum
PPTX
Optimizing VM images for OpenStack with KVM/QEMU
PDF
TechDay - Toronto 2016 - Hyperconvergence and OpenNebula
PPTX
RHEVM - Live Storage Migration
Optimization_of_Virtual_Machines_for_High_Performance
LSA2 - 01 Virtualization with KVM
Achieving the Ultimate Performance with KVM
Look Into Libvirt Osier Yang
LCE13: Virtualization Forum
Optimizing VM images for OpenStack with KVM/QEMU
TechDay - Toronto 2016 - Hyperconvergence and OpenNebula
RHEVM - Live Storage Migration

What's hot (20)

PDF
Achieving the ultimate performance with KVM
ODP
Disk Performance Comparison Xen v.s. KVM
PDF
Achieving the ultimate performance with KVM
PDF
SiteGround Tech TeamBuilding
PPTX
Cinder Live Migration and Replication - OpenStack Summit Austin
ODP
Gluster volume snapshot
ODP
Libvirt and bhyve under FreeBSD
PDF
Control your service resources with systemd
PDF
Dave Gilbert - KVM and QEMU
PDF
OSv at Usenix ATC 2014
PDF
XPDS14: libvirt support for libxenlight - James Fehlig, SUSE
PDF
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
PDF
KVM tools and enterprise usage
PDF
XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems
PDF
Improve your storage with bcachefs
PDF
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto García
PPTX
PDF
s6196-chris-huybregts-microsoft-new-gpu-virtualization-technologies
PDF
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
PDF
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBIT
Achieving the ultimate performance with KVM
Disk Performance Comparison Xen v.s. KVM
Achieving the ultimate performance with KVM
SiteGround Tech TeamBuilding
Cinder Live Migration and Replication - OpenStack Summit Austin
Gluster volume snapshot
Libvirt and bhyve under FreeBSD
Control your service resources with systemd
Dave Gilbert - KVM and QEMU
OSv at Usenix ATC 2014
XPDS14: libvirt support for libxenlight - James Fehlig, SUSE
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
KVM tools and enterprise usage
XPDS14 - OSv - A Modern Semi-POSIX LibraryOS - Glauber Costa, Cloudius Systems
Improve your storage with bcachefs
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto García
s6196-chris-huybregts-microsoft-new-gpu-virtualization-technologies
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBIT
Ad

Similar to Kvm optimizations (20)

PDF
Libvirt/KVM Driver Update (Kilo)
PDF
RMLL / LSM 2009
PDF
Rmll Virtualization As Is Tool 20090707 V1.0
PDF
VMware compute driver for OpenStack
PDF
Virtualization Primer for Java Developers
PPTX
Xen Project Update LinuxCon Brazil
PDF
Aplura virtualization slides
PDF
Storage-Performance-Tuning-for-FAST-Virtual-Machines_Fam-Zheng.pdf
PDF
RunningFreeBSDonLinuxKVM
PDF
OpenVZ Linux Containers
PDF
S4 xen hypervisor_20080622
PDF
3. configuring a compute node for nfv
PDF
Joyent's Bryan Cantrill: Experiences Porting KVM to SmartOS at KVM Forum, Aug...
PDF
Experiences porting KVM to SmartOS
PDF
LCA 2013 - Baremetal Provisioning with Openstack
ODP
UDS 2012 Xen
PPTX
virtualization and hypervisors
PDF
Virtualization with KVM (Kernel-based Virtual Machine)
PDF
CCNA17 KVM and CloudStack
PDF
Advancedperformancetroubleshootingusingesxtop 101110131727-phpapp02
Libvirt/KVM Driver Update (Kilo)
RMLL / LSM 2009
Rmll Virtualization As Is Tool 20090707 V1.0
VMware compute driver for OpenStack
Virtualization Primer for Java Developers
Xen Project Update LinuxCon Brazil
Aplura virtualization slides
Storage-Performance-Tuning-for-FAST-Virtual-Machines_Fam-Zheng.pdf
RunningFreeBSDonLinuxKVM
OpenVZ Linux Containers
S4 xen hypervisor_20080622
3. configuring a compute node for nfv
Joyent's Bryan Cantrill: Experiences Porting KVM to SmartOS at KVM Forum, Aug...
Experiences porting KVM to SmartOS
LCA 2013 - Baremetal Provisioning with Openstack
UDS 2012 Xen
virtualization and hypervisors
Virtualization with KVM (Kernel-based Virtual Machine)
CCNA17 KVM and CloudStack
Advancedperformancetroubleshootingusingesxtop 101110131727-phpapp02
Ad

More from OpenNebula Project (20)

PDF
OpenNebulaConf2019 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...
PDF
OpenNebulaConf2019 - Building Virtual Environments for Security Analyses of C...
PDF
OpenNebulaConf2019 - CORD and Edge computing with OpenNebula - Alfonso Aureli...
PDF
OpenNebulaConf2019 - 6 years (+) OpenNebula - Lessons learned - Sebastian Man...
PDF
OpenNebulaConf2019 - Performant and Resilient Storage the Open Source & Linux...
PDF
OpenNebulaConf2019 - Image Backups in OpenNebula - Momčilo Medić - ITAF
PDF
OpenNebulaConf2019 - How We Use GOCA to Manage our OpenNebula Cloud - Jean-Ph...
PDF
OpenNebulaConf2019 - Crytek: A Video gaming Edge Implementation "on the shoul...
PDF
Replacing vCloud with OpenNebula
PDF
NTS: What We Do With OpenNebula - and Why We Do It
PDF
OpenNebula from the Perspective of an ISP
PDF
NTS CAPTAIN / OpenNebula at Julius Blum GmbH
PDF
Performant and Resilient Storage: The Open Source & Linux Way
PDF
NetApp Hybrid Cloud with OpenNebula
PPTX
NSX with OpenNebula - upcoming 5.10
PDF
Security for Private Cloud Environments
PDF
CheckPoint R80.30 Installation on OpenNebula
PDF
DE-CIX: CloudConnectivity
PDF
PDF
Cloud Disaggregation with OpenNebula
OpenNebulaConf2019 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...
OpenNebulaConf2019 - Building Virtual Environments for Security Analyses of C...
OpenNebulaConf2019 - CORD and Edge computing with OpenNebula - Alfonso Aureli...
OpenNebulaConf2019 - 6 years (+) OpenNebula - Lessons learned - Sebastian Man...
OpenNebulaConf2019 - Performant and Resilient Storage the Open Source & Linux...
OpenNebulaConf2019 - Image Backups in OpenNebula - Momčilo Medić - ITAF
OpenNebulaConf2019 - How We Use GOCA to Manage our OpenNebula Cloud - Jean-Ph...
OpenNebulaConf2019 - Crytek: A Video gaming Edge Implementation "on the shoul...
Replacing vCloud with OpenNebula
NTS: What We Do With OpenNebula - and Why We Do It
OpenNebula from the Perspective of an ISP
NTS CAPTAIN / OpenNebula at Julius Blum GmbH
Performant and Resilient Storage: The Open Source & Linux Way
NetApp Hybrid Cloud with OpenNebula
NSX with OpenNebula - upcoming 5.10
Security for Private Cloud Environments
CheckPoint R80.30 Installation on OpenNebula
DE-CIX: CloudConnectivity
Cloud Disaggregation with OpenNebula

Kvm optimizations

  • 1. CentOS Dojo 2013 KVM Optimizations and Best Practices From the Desktop to the Datacenter Jaime Melis (opennebula.org)
  • 2. Summary ● Tools ● Drivers ● Image Backends ● CPU ● Memory ● Networking ● Desktop Sharing ● Cloud point-of-view Challenges
  • 3. Tools ● Libvirtd !!!!!! ● virt-manager ● virt-install ● qemu-img ● virt-alignment-scan (libguestfs) ● lstopo ● OpenNebula
  • 4. Drivers ● Easy... use virtio!!! ● Huge performance gain ● No emulation ● Cooperation with the hypervisor ● Integrated into the linux kernel
  • 5. Image Backends ● Regular file ● Qcow2 – snapshots (beware of the sync!) – additonal layer, less performance – compressed ● LVM → block device (san storage) ● Virt-alignment-scan ● Cache – Writethrough (host page on, guest disk write cache off) – Writeback (host page on, guest disk write cache on) ● Good overall I/O Performance – None (host page off, guest disk write cache on) ● Good write performance
  • 6. CPU - pinning ● NUMA (Non-Uniform Memory Access) ● Shared cache ~ 15% improvement ● Bad for different tasks ● Processor availability ● virsh capabilities ● lstopo
  • 7. CPU - pinning <cputune> <vcpupin vcpu="0" cpuset="1-4,^2"/> <vcpupin vcpu="1" cpuset="0,1"/> <vcpupin vcpu="2" cpuset="2,3"/> <vcpupin vcpu="3" cpuset="0,4"/> ... </cputune>
  • 8. CPU - cgroups ● Limit, account and ● Bad for different tasks ● Processor availability ● virsh capabilities ● lstopo
  • 9. CPU - cgroups ● Limit, account and isolate resource usage <cputune> <shares>2048</shares> ... </cputune>
  • 10. CPU - model ● Subset of features ● Increase performance ● Nested virtualization ● /usr/share/libvirt/cpu_map.xml <cpu match='exact'> <model fallback='allow'>core2duo</model> <vendor>Intel</vendor> <topology sockets='1' cores='2' threads='1'/> <feature policy='disable' name='lahf_lm'/> </cpu>
  • 11. Memory - KSM ● Kernel Samepage Merging ● Combines memory private pages ● Very useful for VMs !!!! ● Increases VM density ● Enable by default
  • 12. Networking7 ● MacVTap (direct)... NO! ● Bridged networking... way to go! ● Disable STP ● Optimal configuration ● NAT (masquerading) ● iptables
  • 13. Desktop Sharing ● VNC ● SPICE – qlx driver – Redirect printers – Usb (mass-storage) – Audio
  • 14. Cloud Challenges ● Virt-manager + LVM + DHCP => 10 hosts ● Live-migration to any host ● Image layout – don't waste space ● Contextualization: install once and deploy many ● Network Isolation ● Mac/IP spoofing ● Multi-tenancy
  • 15. Cloud Challenges - ebtables ● Isolate ● Prevent mac spoofing # Drop packets that don't match the network's MAC Address -s ! <mac_address>/ff:ff:ff:ff:ff:0 -o <tap_device> -j DROP # Prevent MAC spoofing -s ! <mac_address> -i <tap_device> -j DROP
  • 16. Cloud Challenges – What storage? ● No Storage Holy Grail LVM2 iSCSI Qcow2 + DFS Provisioning time + ++ +++ I/O ++ ++ + Network ++ +++ + Snapshotting - - + Livemigration + - +
  • 17. Thanks for attending! jmelis - #opennebula @Freenode