Linux ecosystem contributions from SteamOS

The SteamOS Linux distribution is focused on gaming, naturally, but the effort to build it has resulted in contributions to multiple areas in the Linux ecosystem. Alberto Garcia has been working on SteamOS and came to Bilbao, Spain to describe some of those contributions at Open Source Summit Europe 2023. There are some obvious areas where a gaming-focused OS might contribute upstream, such as graphics, but the talk showed contributions in several other areas as well.

García introduced himself as an employee of Igalia, which happened to turn 22-years old on the day of his talk. He is also a longtime FOSS and Debian developer; he has previously worked on projects like QEMU and Maemo. Lately, he has been working on SteamOS and getting Linux working on the Steam Deck consumer hardware. One of the things he likes about free-software development is the way that new projects end up contributing to older projects that are not necessarily directly connected; SteamOS fits that model well.

Steam Deck

The Steam Deck is a handheld gaming system that was released by Valve, the company behind the Steam game-distribution service, in 2022. The current version of the operating system for the device is SteamOS 3; there were two previous versions, but those do not work on the Steam Deck and have been discontinued. He finds the Steam Deck interesting because it is an example of "a successful consumer device with standard Linux components".

His talk would cover a bunch of different contributions from SteamOS, many of them from his colleagues at Igalia, but the talk was not meant to be an exhaustive list. In addition, there are contributions from other companies and developers, as well as from regular users. García said that he tried to give credit to the contributors in the talk and in his slides, but he apologized in advance to anyone that he had missed.

He showed a picture of the SteamOS user interface, but said that, for the purposes of the talk, it was an uninteresting full-screen interface that allows buying and playing games. Under the hood, though, SteamOS is a Linux distribution that is based on Arch Linux. It uses standard Linux components, with a few customizations, on a filesystem that is based on the Filesystem Hierarchy Standard (FHS). It has a user space with GNU tools and utilities, systemd as its init, and D-Bus for inter-process communication. Unlike many other devices, though, access to the underlying OS is completely unlocked by default; "the user has complete access to the OS, there's nothing hidden". Users can run a shell as root on the Steam Deck.

There are two modes in SteamOS: gaming and desktop. Most users never leave gaming mode, which gives them access to their Steam games, allows configuring things like the network, and more. But the desktop mode is available; he showed a screen shot and said "if this looks like a regular KDE Plasma session, it's because that's what it is exactly". In desktop mode, users can install anything they want, including web browsers, tools, and non-Steam games; in fact, the Steam client allows running non-Steam games that are installed on the Deck. Meanwhile, adding a keyboard, mouse, and display allows using the Steam Deck in desktop mode as a regular desktop computer.

Unlike, say, Android's developer mode, desktop mode is not hidden away at all. The regular power menu has a "Switch to Desktop" option alongside things like "Shutdown" and "Restart". That mode can be accessed by anyone.

Proton and Wine

The use case for the device is to play games on Linux, but there is an inherent problem with that: most Steam games only run on Windows. Those games are never going to get Linux versions, so the Proton compatibility layer is used to allow the games to run on SteamOS. Proton was developed by Valve in 2018 using a collection of different open-source tools. The main components of it are Wine, translators to convert Direct3D to Vulkan, and GStreamer. It is an actively developed project that is available in the Proton GitHub repository.

Wine is a project to run Windows applications on Linux, but it is not an emulator; instead the Windows APIs are implemented using POSIX and/or Linux APIs. It is developed by the community, but the main driver behind the project is a company called CodeWeavers, which works in a partnership with Valve on Wine for Proton. All of the components of Proton are customized for SteamOS and the Steam Deck, but all of those changes are contributed back to the upstream projects.

When implementing Windows APIs on Linux, there are two possibilities; the first is that the API is similar to that of Linux, which means there is no problem. Those APIs can be directly provided in Wine; "there's maybe a little overhead, but it works fine". If there is no equivalent Linux API, Wine needs to implement it, which can result in high overhead; there are some APIs that cannot be implemented in user space as well. Some developers have been looking at which APIs are problematic to implement, but are used frequently in games, in order to add features to Linux to help bridge the gap.

Kernel features

For example, Windows has a synchronization function that is used by a lot of games, WaitForMultipleObjects(). There is no direct equivalent in Linux, so it had been implemented using futex(), but futex() is restricted to a single object. The solution is a new futex() API, which was proposed because the existing API was hard to use and extend. That proposal led to the addition of the futex_waitv() system call in 5.16. The full new API is still a work in progress, but futex_waitv(), which was developed by André Almeida and others, is sufficient to emulate the Windows API call.

A well-known feature of Windows filesystems is that they are case-insensitive so applications expect that "data.bin" and "DATA.BIN" refer to the same file—which is not generally true for Linux filesystems. The "slow solution" to this problem was to implement the case-insensitivity in Wine; when a file lookup failed, Wine would have to look for case-variant versions of the name. The faster solution was to make ext4 case-insensitive, which was developed by Gabriel Krisman Bertazi "from our friends at Collabora". Daniel Rosenberg later added the feature to F2FS for Android, García said.

He quickly went through a few other kernel features, some of which are still in progress, that came from SteamOS, including the user-space adaptive spinlock work by Almeida and Mathieu Desnoyers. Spinlocks are difficult to implement reliably in user space, but changes to restartable sequences have provided a path to do so. Another change was to support mounting two Btrfs filesystems with the same fsid; this was done so that systems with A/B partitioning schemes can mount both filesystems at once when that is needed. Guilherme Piccoli did the work on the Btrfs feature, as well as working on the handling of split locks so that the kernel slowdown applied to those operations can be disabled. Games frequently use split locks, so slowing them down in gaming-centric workloads only leads to unhappy users.

Graphics

There are many more kernel improvements, García said, but then turned to contributions in another critical part of a gaming device: graphics. The RADV Vulkan driver for AMD GPUs was developed by Valve and others; it is an open-source driver that is part of the Mesa project. It is an alternative to the official AMD drivers; RADV is what most Linux distributions are shipping, thus what most people are using, he said.

A related contribution is the ACO shader compiler for AMD graphics, which was developed by Valve and released in 2019. It reduces stuttering and increases the frames-per-second (FPS) rate. There are also some Vulkan extensions for hardware-accelerated video, which needs encoding and decoding support in the graphics drivers and multimedia frameworks, such as for GStreamer; the work on that is still in progress.

The Linux Direct Rendering Manager (DRM) subsystem exposes a small set of color properties; proposals to extend the color API have been made, but none has been adopted. AMD GPUs have additional color capabilities, so a driver-specific color API for AMD devices has been developed; among other things, it will allow displaying high-dynamic-range (HDR) content. This capability, which was developed by Melissa Wen, Joshua Ashton, and Harry Wentland, will be part of the upcoming SteamOS 3.5 release.

GPUs are complex beasts; they can crash, but there is no standard way to report a problem of that sort to user space. There is work in progress to address that lack by standardizing GPU-hang reporting and handling, as well as creating a standard for what compositors do after a GPU reset. Almeida presented on his work in that area at Open Source Summit North America in May.

Valve has developed an open-source micro-compositor, gamescope, in order to solve problems with games trying to use resolutions that are different than the native resolution of the display. "Instead of changing resolutions every time for every game, gamescope tricks the game into thinking it is running in the resolution that the game requested." It then converts the output to conform to the native resolution; it can also do frame-rate limiting for reducing power consumption. Gamescope runs the Steam Deck game user interface and it is available in most Linux distributions.

OS and Flatpak

Part of the work on SteamOS is the same as with any distribution: ensuring that all of the components work well together, handling updates seamlessly, and so on. But there are parts of what SteamOS is doing that are of interest to the rest of the community. SteamOS 3, which is what García has personally been working on, is a customization layer on top of Arch Linux; almost all of the packages come directly from Arch, without being changed or even rebuilt. The Arch Linux philosophy is for packages to be as close to upstream as possible; downstream patches are not applied "unless it is really necessary". SteamOS has adopted the same philosophy; when there is a problem with a package, it is fixed upstream whenever possible.

Unlike Arch, SteamOS has an immutable, read-only root filesystem; it has an A/B partitioning scheme, so there are two versions of the root filesystem available, one of which is the active root. When an update is done, the inactive partition is updated; if all goes well, the update partition becomes the new active root. Users are not expected to do anything that changes the root filesystem, though they have the ability to do so since they have the privileges to remount the root filesystem as read-write; any changes will be lost at the next OS update, however. "So what is the way to install new software?"

Steam games are obtained from the store, "so Steam takes care of everything". For desktop applications, the suggested installation mechanism is to use Flatpak. It is also used by other distributions with read-only root filesystems, such as Fedora Silverblue and Endless OS.

Flatpak is a framework for sandboxed desktop applications that are isolated from the host OS; the isolation level can be adjusted depending on the needs of the application. Flatpak is also a distribution-independent packaging system; it allows the application developers to create a binary package that can be installed on any distribution that supports the framework. Flatpak applications can be distributed from anywhere, but the Flathub repository is "the Linux app store"; there are more than 2000 applications available there and it has become the primary distribution site for some applications (e.g. Bottles). SteamOS developers are working as part of the Flatpak community to fix bugs and add features.

He noted that using Flatpak as the primary distribution mechanism for desktop mode in SteamOS was something of a Trojan horse, as shown in a Mastodon post (and in the image on the left). When application developers add support for the Steam Deck, they are indirectly supporting the Linux desktop. Any effort that is put into improving the SteamOS version is immediately usable on the Linux desktop as well.

Portals are the mechanism used by Flatpak applications to interact with the host, though Portals are not Flatpak-specific. They provide a D-Bus interface to access things outside of the Flatpak sandbox for opening files and URLs, taking screen shots, changing settings, and more.

The Portal API is implemented by a desktop-specific backend, but not all of the backends work on all of the desktop environments, which can cause a variety of different problems. The way to choose the backend was fairly limited and that affected the Steam Deck because it has two separate desktop sessions active, one for each mode. Some work by Emmanuele Bassi gives a better way to choose Portals and backends; it was recently released in xdg-desktop-portal 1.18.0 and will start appearing in major distributions soon.

There is also work on bug fixes and new features for KDE that have been contributed by the SteamOS developers. That includes handling updates with many packages, improving detection of new icons, and better udev-event handling. Beyond that, there are lots of other contributions to things like NetworkManager, Pipewire, ALSA, and more.

García wrapped up by reiterating that SteamOS is making lots of contributions to make desktop Linux better and it is also providing an incentive for desktop-application developers to target Linux with their software. In the Q&A, he was asked about installing other types of packages; Flatpak is only for desktop applications, not for command-line tools. He said that there is no official support for installing those tools, yet, but that people are experimenting with various ways to do so.

[I would like to thank LWN's travel sponsor, the Linux Foundation, for travel assistance to Bilbao for OSSEU.]