[jailhouse article] performance measurements for hypervisors on embedded ARM processors

S. Toumassian, R. Werner and A. Sikora, “Performance measurements for hypervisors on embedded ARM processors,” 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2016, pp. 851-855, doi: 10.1109/ICACCI.2016.7732152.

primary coverage ：

1、 It introduces Xen and Jailhouse The architecture of .

2、Xen and Jailhouse Verification method and measurement results .

With the Internet of things (IoT) And Industry 4.0 And other modern development , More and more computing services need to be hosted on embedded devices . In the past , Different services usually run on different hardware , This increases costs and reduces flexibility . With the continuous enhancement of computing device functions , Running multiple services on the same hardware becomes desirable . however , It has aroused people's concern about safety . The concept of virtualization allows isolated instances of multiple services to run on a single machine . This concept has been used in traditional server computers for decades , But now it is also increasingly used in embedded microcontrollers . therefore , Different concepts need to be revisited from the perspective of embedded requirements , For example, more extreme costs and resource constraints 、 Hardware access and real-time capabilities .

1. Full virtualization 、 Introduction to semi virtualization and hardware assisted virtualization

Full virtualization （ Do not modify the customer system ,VMM Use dynamic binary conversion to simulate privileged instructions ,VMWare、Qemu）： In full virtualization , The hypervisor simulates hardware resources （CPU、 Memory and PCI equipment ）, So that the unmodified client system can be run [2]. The client system has defined permissions （ Usually very limited ） The domain of , for example Windows Server 2003 and Windows XP. Full virtualization uses binary conversion to operate privileges “ Convert to an equivalent non privileged instruction block ”[2], To be directly in CPU Up operation , Produces exactly the same effect as the original instruction [2] [3].VMware [4] It is the first time to combine dynamic binary conversion with direct execution technology [2]. In full virtualization , operating system (OS) And hardware do not need any modification [2]. except VMware, also Qemu [5]、Virtual PC [6] etc. .

Semi virtualization （ Modify the customer system , Clients use super calls to execute privileged instructions ,Xen）： The virtualization cost of this virtualization technology is lower than that of full virtualization [2]. In semi virtualization , The client system was modified to use so-called super calls instead of privileged instructions . With system calls (syscall) equally , It is a software trap from the application to the operating system kernel , Super calls are software traps from the client domain to the hypervisor . Semi virtualization also allows you to replace multiple privileged instructions with a super call . This helps reduce the number of context switches between privileged and non privileged modes , So as to reduce the cost . The first semi virtualization concept is made up of Denali [7] And then Xen [8] Introduced . However , The disadvantage of semi virtualization is that it needs to modify the operating system [2].

Hardware assisted virtualization （ Do not modify the customer system , With virtual expansion ,VT-x,AMD-V）： Some hardware vendors have developed virtualization extensions , So that the unmodified client operating system can run on the hypervisor , for example Intel VT-x [9]、AMD-V [10] and ARM Virtualization extension [11]. They added a new privilege level to the processor . Highest privilege level “Ring-1” Used to manage programs , User application in “Ring-3” Run in [2]. The advantages of the new privilege level are “Ring-1”“ Give Way VMM Use safely and transparently VM Direct execution of , So it simplifies VMM Design ”[2].

The benefits of virtualization are manifold , For example, security 、 Flexibility and reduced cost and space . Besides , Virtualization makes system management and configuration easier . however , All the virtualization technologies mentioned above will increase the performance overhead .

2. intel virtualization VT-d VT-x VT-c What's the difference? , What do you mean , What's the role ？

https://www.d3tt.com/view/111

3. Hypervisor Introduce

3.1 Xen on ARM

Pictured ,Xen Hypervisor or Xen Virtual Machine Monitor (VMM) Is a bare metal open source management program , It runs directly at the physical level , And integrates low-level communication between hardware and virtual machines . stay Xen Virtual machines running above the hypervisor layer are called domains . Domain 0 or dom0 Refers to the privileged virtual machine （ Privileged domain ）, It contains all drivers of the device .Xen Create at boot time Dom0.Dom0 Contains all unmodified Linux Device driver , This domain has direct access to hardware , Deal with the system I/O All access to functions , control 、 establish 、 Pause 、 transfer 、 Termination and with other domains [1] signal communication .Dom0 Provide access to hardware for non privileged domains through semi virtualized back-end drivers . Besides , The modified non privileged domain can use the semi virtualized front-end driver to access a group of virtual devices .（Xen Have special Dom0 Be responsible for connecting the preceding and the following , Use the front and rear driver interfaces to complete the access of non privileged domain to hardware resources ）

Corresponding ARM Three virtualization extension execution levels （ User mode 、 Kernel mode 、 Management program mode ）,Xen Run in hypervisor mode ,VM Kernel mode runs in kernel mode ,VM User mode runs in user mode .

Xen version3 have Credit Scheduler ,Xen version4.5 Introduced RTDS（ Real time delayable scheduler ）, Provide customers with real-time performance , To schedule VCPU,RTDS The scheduler follows the preemptive global earliest deadline first (EDF) Theory to choose priorities , This means VCPU Get a higher priority . stay Xen In the management process , Scheduling and sorting... In queues VCPU Each iteration of will add additional load , This leads to additional execution time . Besides ,Xen Both schedulers in follow some algorithm to VCPU Sort , So it increases Xen Complexity of management procedures .

3.2 Jailhouse

Jailhouse yes Siemens [14] Open source partition manager .Jailhouse The hypervisor does not follow the methods of other hypervisors , Because there is no simulation of non-existent devices , Nor does it share anything between customers CPU, therefore , No scheduler is required .Jailhouse The main idea behind this is to make the hypervisor as simple as possible . In addition to running bare metal applications as customers ,Jailhouse It can also run the modified Linux Kernel image （ Semi virtualized features ）.Jailhose Divide a system into two parts called “cell” Isolated area . Every cell All with others cell Well isolated , And provide its own hardware resources for its customers .

And Xen Others, such as VMM Compared with many other hypervisors ,Jailhouse It can be considered as a very lightweight hypervisor .

3.3 KVM

Virtual machine based on Kernel (KVM) It's a Linux Subsystem , It uses hardware virtualization extensions to Linux OS [15] Provides management program functions . From face to face x86 The kernel version of the platform 2.6.20 Start , It has been included in Linux in . at present , It also supports ARM Cortex-A15 and Cortex-A7 platform . KVM Expanded Linux Traditional kernel and user mode , And created a new process mode , be called “guest”. Guests have their own kernel and user mode [1]. For the mainframe ,Guest Display as normal Linux process .KVM.ko It's a kernel module , It acts as a device driver , Provide the ability to manage virtual hardware [1]. By opening the device node “/dev/kvm” To create a virtual machine ; “ Each virtual machine can have a virtual machine created by Linux The scheduler allocates its own address space ”[1].

3.4 Xvisor

Xviso Is an open source lightweight virtual machine management program . It is not ARM Virtualization extends ARM The processor provides a hypervisor for virtualization solutions [16].Xvisor It is a bare metal management program that follows the full virtualization method . This means that it can run the unmodified operating system as a guest .

4. Performance Measurements

The purpose of these measurements is to investigate embedded ARM Overhead of different hypervisors in virtualized environments on processors , In order to know which candidate is more suitable for real-time applications . The approach in this article is to use a heavily loaded Application -Cpuburn-a8, When not in use Hypervisor Under the circumstances 、 Use Xen as well as Jailhouse In case of , Create a kernel module to access physical counters and a user application to measure the number of switches between privileged and non privileged domains / Time .

Experimental environment ：

• banana-pi, Allwinner A20(Cortex-A7 CPU), 8GB SD-Card
• Xen hypervisor version 4.6 including a (RTDS) scheduler.
• Jailhouse hypervisor in its current master branch version (as of May 2016).
• Linux kernel version 4.1, as it contains all the necessary configurations to enable virtualization on ARM processors and additionally has long time support (until September 2017).

The kernel module . Physical count register (CNTPCT) Is a general timer register , preservation 64 Bit physical counter value . however , It can only be accessed from kernel mode . To enable user mode to access CNTPCT register , Timer control register CNTKCTL The first bit of must be set to 1. Again ,CNTKCTL Can only be accessed from kernel mode . To solve this problem , We created a kernel module , It allows user mode access to physical timers .

User programs . Keep executing user programs Cpuburn-a8,Xen in , Consider two different schedulers （Credit、RTDS）,Jailhouse in , Use exactly the same platform and test program .

5. Results

Repeat four times to get the average .

According to the results listed in the table ,Xen The management program is in use Credit Scheduling programs has significantly higher overhead , Use RTDS The scheduler can reduce the overhead . in general ,Jailhouse It has the lowest and very attractive overhead on the system . as a result of Jailhouse Neither virtualize nor share hardware resources among customers , Therefore, no scheduler is required , All this reduces overhead and keeps the hypervisor lightweight .

Last , Even though Xen Provides more management functions , But the cost is still relatively high . On the other hand ,Hypervisor Compare with Xen Not rich in functions , But it provides a virtualization system with almost zero overhead .