Using Jenkins libvirt-slave-plugin with bhyve

I've played with libvirt-slave-plugin today to make it work with libvirt/bhyve and decided to document my steps in case it would be useful for somebody.

libvirt-slave-plugin

Assuming that you already have Jenkins up and running, installation of libvirt-slave-plugin is as follows. As we need a slightly modified version, we need to build it ourselves. I've made a fork which contains a required modification which could be cloned like that:

git clone -b bhyve git@github.com:jenkinsci/libvirt-slave-plugin.git

The only change I made is adding a single line with 'BHYVE' hypervisor type, you could find the pull request here. When that would be merged, this step will be not required.

So, getting back to the build. You'll need maven that could be installed from ports:

cd /usr/ports/devel/maven2 && make install clean

When it's installed, go back to the plugin we cloned and do:

mvn package -DskipTests=true

When done, login to the Jenkins web interface, go to Manage Jenkins -> Manage Plugins -> Advanced -> Upload Pluging. It'll ask to provide a path to the plugin. It would be target/libvirt-slave.hpi in our plugin directory.

After plugin is installed, please follow to Manage Jenkins -> Configure System -> Add new cloud. Then you'll need to specify hypervisor type BHYVE and configure credentials so Jenkins could reach your libvirtd using SSH. There's a handy 'Test Connection' you could use your configuration.

Once done with that, we can go to Manage Jenkins -> Manage Nodes -> New Node and choose 'libvirt' node type. Then you'll need to choose a libvirt domain to use for the node. From now on, node configuration is pretty straightforward, expect, probably an IP address of the slave. To find out an IP address, you'd need to find out its MAC address (just run virsh dumpxml and you'll find it there) and then find the corresponding file in dnsmasq/default.leases file.

Guest Preparation

The only thing guest OS needs is to have jdk installed. I preferred to download a package with java/openjdk7, but I had to configure network first. My VMs use bridged networking on virbr0, so NAT config looks like that in /etc/pf.conf:

ext_if="re0"
int_if="virbr0"

virt_net="192.168.122.0/24"

scrub all

nat on $ext_if from $virt_net to any -> ($ext_if)

Now openjdk could be installed from the guest using:

pkg install java/openjdk7

Finally, find the nodes in node management menu and press 'Launch slave agent' button. It should be ready for the builds now.

PS It might be useful to sync clock on both guest and host systems using ntpdate.

PPS libvirt version should be at least 1.2.2.

Bhyve in libvirt

I continue my activities on improving libvirt FreeBSD support and I have some good news. Recent libvirt release, 1.2.2, is the first version to include the bhyve support!

Currently it's in its early stage and doesn't support some of the features and doesn't provide good flexibility, it's just a basic stuff at this point. I'll not provide a detailed description and instead will point you to the document: Libvirt: Bhyve driver. You'll find a sample domain XML which covers all the features currently supported by the driver.

TODO list

While there are lots and lots of things to be done, there are some specific ones I'm focusing on:

• Console support through nmdm(4). This is very important feature for debugging and checking what's going on in the guest.
• Domains autostart support. There's a patch already kindly provided by David Shane Holden that just needs review and testing.
• A little more flexible slot ids allocation / device configuration.

Qemu/FreeBSD status

As a side note, I'll give an update what's changed since my previous blog post about qemu libvirt driver on FreeBSD. So, here's what's new:

• Proper TAP interfaces cleanup
• CPU affinity configuration support, check http://libvirt.org/formatdomain.html#elementsCPUAllocation for details
• virsh console should now work if you run it from freebsd host and connect to libvirtd on Linux
• Node status support (such as virsh nodecpustats, virsh nodememstats)

Some of these are available in already released versions, some are only in git version.