Symptoms
- Every once in a while, randomly, KVM VPS running with SolusVM virtual machine is getting suspended or stopped.
- The following message can be found in
dmesg
or in/var/log/messages
:Nov 2 13:54:33 server kvm: 25 guests now active
server kernel: qemu-kvm invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
server kernel: Task in /machine.slice/machine-qemu\x2d291\x2dkvm1309.scope killed as a result of limit of /machine.slice/machine-qemu\x2d291\x2dkvm1309.scope
server kernel: memory: usage 9437184kB, limit 9437184kB, failcnt 155666
server kernel: memory+swap: usage 9621056kB, limit 9007199254740988kB, failcnt 0
server kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
server kernel: Memory cgroup stats for /machine.slice/machine-qemu\x2d291\x2dkvm1309.scope: cache:20KB rss:9437164KB rss_huge:0KB mapped_file:20KB swap:183872KB inactive_anon:1048736KB active_anon:8388452KB inactive_file:0KB active_file:0KB unevictable:0KB
server kernel: Memory cgroup out of memory: Kill process 67178 (CPU 5/KVM) score 541 or sacrifice child - The same issue might occur during installing of Windows 2016 guest from ISO image.
Cause
Insufficient value for hard_limits parameters in the configuration file of the virtual machine
Resolution
Increase hard_limit
value using SolusVM tuning. The information below is also available on the following documentation page
- Connect to the slave KVM server via SSH
- Check if the
/usr/local/solusvm/data/config.ini
exist and create a copy of example configuration file if it is not:# cp -a /usr/local/solusvm/data/config.ini.example /usr/local/solusvm/data/config.ini
- In this file
/usr/local/solusvm/data/config.ini
setdomain_config_exclude_memtune
as false:;; Don’t write memtune settings to domains configuration file (true|false)
domain_config_exclude_memtune = false - And tune
domain_config_memtune_hard_limit_percent_memory
parameter to required value:;; If memtune hard_limit is used in the domains configuration file set
domain_config_memtune_hard_limit_percent_memory = 110
domain_config_memtune_soft_limit_percent_memory = 110
domain_config_memtune_swap_hard_limit_percent_memory = 125
for example 200 will set a doubled value from RAM value assigned to the virtual server
- For newly created virtual servers limits will be applied automatically. To apply the changes for already existing virtual servers, reboot them.