To implement the CI pipeline for the Android project, I need to run Gitlab CI, in which KVM is enabled.
TL; TR
- Enable VTx virtualization technology in your server BIOS
- Pass
/dev/kvm
to Gitlab runner
Sample .gitlab-ci.yml
Test - E2E:
stage: Testing
image: reactnativecommunity/react-native-android:latest
before_script:
- apt update && apt install -y cpu-checker && apt auto-remove
- kvm-ok
- npm i -g envinfo detox-cli && envinfo
# Increase file watcher limit, see more here: https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers#the-technical-details
- echo fs.inotify.max_user_watches=524288 | tee -a /etc/sysctl.conf && sysctl -p
- mkdir -p /root/.android && touch /root/.android/repositories.cfg
- echo yes | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager --channel=0 --verbose "system-images;android-28;default;x86_64"
- echo no | $ANDROID_HOME/cmdline-tools/latest/bin/avdmanager --verbose create avd --force --name "Pixel_API_28" --device "Pixel_API_28_AOSP" -d pixel --package 'system-images;android-28;default;x86_64'
However, the pipeline was failing with the following error.
$ kvm-ok
INFO: /dev/kvm does not exist
HINT: sudo modprobe kvm_intel
/usr/sbin/kvm-ok: 89: modprobe: not found
Enable VTx virtualization technology
First, I had to configure my server to enable VTx
Virtualization Technology. To enable VTx
, I followed the below steps.
- Turn off/on the server
- Access to BIOS (In my case, F10)
- Go to Security -> System Security
- Enable
Virtualization technology (VTx)
- Click F10 to accept the change
- Save the change and hard power off and power it back on
Once you reboot the server, make sure you run kvm-ok
to confirm the KVM virtualization is enabled.
Update Gitlab runner
Now, to allow Gitlab runner to use KVM, I made some changes on my docker-compose.yml
for Gitlab runner.
services:
... other services ...
gitlab-runner-1:
container_name: gitlab-runner-1
image: gitlab/gitlab-runner:latest
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/kvm:/dev/kvm
gitlab-runner-1-register:
container_name: gitlab-runner-1-register
image: gitlab/gitlab-runner:latest
restart: "no"
volumes:
- gitlab_runner_1_config:/etc/gitlab-runner
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- gitlab-runner-1
command: >
register
... other parameters ...
--docker-devices=/dev/kvm
--docker-volumes=/var/run/docker.sock:/var/run/docker.sock
volumes:
gitlab_runner_1_config:
Or if you are using config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner-1"
url = "<url>"
token = "<token>"
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:stable"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
devices = ["/dev/kvm"]
volumes = ["/var/run/docker.sock","/cache"]
shm_size = 0
Note that make sure you deregister existing runners from Gitlab before adding new runners with /dev/kvm
.
And Voila!
Once you restart the CI pipeline, the kvm-ok
will show the result as below:
$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used