A new way to develop on Linux

Continuing our collaboration with the GNOME Foundation through the Sovereign Tech Fund (STF), we have been busy on multiple fronts since our last update. This time, we'd like to share how we're improving the developer experience for building and testing software.

The problem

Flatpak is the recommended way to develop, test and iterate on applications. However systems components, as well as default system applications are not meant to run in a sandbox and need tighter integration with the host.

Building and iterating on system components has always been a risky and/or difficult endeavor. Iterating in a VM or container is tedious and interacting with the hardware can be challenging. Building on the host directly puts the system's integrity at risk and is not an option on immutable operating systems such as GNOME OS.

The solution

Inspired by Lennart Poettering and Jordan Petridis; we developed sysext-utils, a lean collection of tools built on top of systemd-sysext and build systems such as BuildStream and Meson.

The tools enable a developer workflow based on system extensions.

These extensions are "layered" at runtime on top of the operating system files tree. In the case of sysext-utils, they are applied temporarily and will be automatically reverted on reboot. If anything goes wrong that's all it will take to get back to a functional operating system.

This approach makes iterating on system components safer and enables it on immutable OSes. Additionally it makes it trivial to build artefacts on CI that anyone can download and test.

If you are working on system components we invite you to test this solution and share your feedback.

Here is a quick way to get started

$ git clone https://gitlab.gnome.org/GNOME/gtk.git && cd gtk
# Make your changes
$ meson setup build --prefix=/usr --libdir=/usr/lib/x86_64-linux-gnu
$ sysext-install extension build
# Test your changes
$ sysext-remove extension # or systemctl reboot

sysext-utils is available on the development tree of GNOME OS or you can install it on your system.

Note that, besides building the extension, sysext-utils will run common integration steps such as updating the glib schemas or updating the icons cache.

Limitations

In order to test the limitations of this approach, we tested and benchmarked it against some of the most demanding components of GNOME; its compositor and desktop namely Mutter and Shell. See our proposal Towards a better way to hack and test your system components.

Through our conversations with the GNOME community, we identified and clarified what can and cannot be covered with this approach:

• It’s not a package manager. If the changes to a system component require pulling other components, then these other components must also be provided through system extensions.
• It’s not a virtual environment. You either test your extension on your actual host, which might require interrupting your session, or you test it on a virtual machine.

More work will be needed to make this the perfect developer experience for the most demanding use cases but we are already able to generate system extensions for GNOME Shell on CI which will make it trivial and safe for anyone to test the changes made by a merge request.

Future work

There are already a few nice-to-haves on our wishlist, but based on our conversations with the community and the limitations described above, we identified that it might be worth exploring the use of systemd-vmspawn (another contribution from Codethink) to address some of their use cases and current limitations.

For GNOME, we hope one day to offer a unified way to develop, build and test changes.

Being able to build and test components individually has already enabled us to begin exploring the use of system extensions to speed up end-to-end testing pipelines in GNOME, but more on that in an upcoming follow-up.

If you’d like to learn more about our work with GNOME or anything discussed in this article, please reach out to us via sales@codethink.co.uk.