Pinning: Apt's Best Kept Secret

I have recently discovered a feature of the apt package manager that smooths over almost all of the annoyances that I’ve had with point-release distributions (Debian, Ubuntu, etc.). apt provides a simple and effective way to install newer packages than are included in the release version that you are running. And it isn’t even hard to get set up.¹

First, a general observation about package versions: For a lot of the packages that are installed on a given Linux system, the fact that they aren’t the latest version is not very important. Ubuntu and Debian backport security and bug fixes to the supported versions of software for their main releases. So you will miss out on some new features, but theoretically shouldn’t be forced to live with any serious bugs or security vulnerabilities.

It is only a relatively small number of packages for which it is important to get the latest and greatest version. What those packages are is going to be pretty personal to you and your workflow. For me, it’s mpv/yt-dlp and neovim. If you’re into gaming, graphics drivers will probably make the list.

With this in mind, pinning in apt is a feature that lets you control what repository apt uses to install specific packages. There are two configuration files that we will be managing here, /etc/apt/sources.list and /etc/apt/preferences. The first of these files lists the repositories that apt will pull packages from, and the second allows you to set priorities for these repositories, either on a package by package basis, or in general.

Let’s consider as a specific example getting an up-to-date version of yt-dlp on Ubuntu 22.04 LTS. Recently, YouTube pushed out an API update that broke this program, and the authors of yt-dlp pushed an update to unbreak it. However, this updated package is not (or at least was not) available in the standard repositories for 22.04.² However, it is available in the repositories for the upcoming Ubuntu 23.04: Lunar Lobster. So let’s set apt up to install Lunar Lobster’s version of yt-dlp onto a machine running Jammy Jellyfish.

Adding new Repositories

First, make sure that you do a full update and upgrade of your system. We’re doing to attempt to do an update later to verify that we have configured our preferences correctly, so we want to make sure that we’re fully up to date going in³,

% sudo apt update
% sudo apt upgrade

Now, we want to update our /etc/apt/sources.list to add the repositories for Lunar Lobster. This is as simple as adding the following line to the file,

deb http://us.archive.ubuntu.com/ubuntu lunar main restricted universe multiverse

We now have added the Lunar repositories to apt. To verify this, run another update to update the repositories, and then use apt-cache to verify that lunar is available,

% sudo apt update
% sudo apt-cache policy

You should see a long list of all of the repositories that are set up, which will include Lunar’s,

...
 500 http://us.archive.ubuntu.com/ubuntu lunar/multiverse amd64 Packages
     release v=23.04,o=Ubuntu,a=lunar,n=lunar,l=Ubuntu,c=multiverse,b=amd64
     origin us.archive.ubuntu.com
 500 http://us.archive.ubuntu.com/ubuntu lunar/universe amd64 Packages
     release v=23.04,o=Ubuntu,a=lunar,n=lunar,l=Ubuntu,c=universe,b=amd64
     origin us.archive.ubuntu.com
 500 http://us.archive.ubuntu.com/ubuntu lunar/restricted amd64 Packages
     release v=23.04,o=Ubuntu,a=lunar,n=lunar,l=Ubuntu,c=restricted,b=amd64
     origin us.archive.ubuntu.com
 500 http://us.archive.ubuntu.com/ubuntu lunar/main amd64 Packages
     release v=23.04,o=Ubuntu,a=lunar,n=lunar,l=Ubuntu,c=main,b=amd64
     origin us.archive.ubuntu.com
...

The number in front of the repositories is the priority, which apt uses to determine where to get packages when installing or updating. By default, you should see 500 used both here, and for Jammy’s repositories. Generally speaking, higher numbers mean higher priority, though the details are a bit more complicated than that (we’ll get there later).

The upshot of this default configuration is that, because lunar and jammy both have the same priority, apt will prefer to install the most up-to-date package available from either of them. In practice, this means that lunar will be the default repository for your system. You can verify this by starting an apt upgrade; it will attempt to upgrade nearly every package on your system, even though we ensured it was fully up-to-date before getting here.

Setting Repository Priorities

This behavior is great if it is your intent to upgrade to Lunar Lobster, but probably not so much if you only want to install a couple of packages from there. So, we need to adjust the priorities so that Jammy’s repositories are preferred.

The way that you set the priority for repositories is by either creating an /etc/apt/preferences file, or adding files to /etc/apt/preferences.d, and specifying priorities in there. We’ll go the single-file route. Create (if it doesn’t already exist) /etc/apt/preferences and place the following inside it,

Package: *
Pin: release n=jammy
Pin-Priority: 700

Package: *
Pin: release n=lunar
Pin-Priority: -10

and then run apt update again to load the new preferences. Now, if you attempt an apt upgrade, it shouldn’t try to install any updates from the lunar repositories.

The first part of each section of this file specifies the packages that the configuration applies to. You can configure this on a package by package basis, or use *, as we do here, to apply the settings to any and all packages. If you did want to do this for a specific package, simply replace the * with the package name.

Then, we specify the repository using Pin: release n=x. If you look back at the output from apt-cache policy, you’ll see that each repository listed there has a couple of parameters listed, an n, a, o, etc. You can use any of these to specify which repository or repositories you are referring to with this setting. a is the most specific, so a=jammy will refer specifically to the jammy repository, but not jammy-updates or jammy-security, whereas n=jammy will collectively reference all three.

Finally, the Pin-Priority sets the priority of the specified repositories for the specific packages. The general rule is that the highest priority wins when trying to install or update, but it’s actually a bit more complicated than that. The details are in the manual (man apt_preferences), but I’ll repeat them here. Note that P refers to the priority number.

   P >= 1000
       causes a version to be installed even if this constitutes a
       downgrade of the package

   990 <= P < 1000
       causes a version to be installed even if it does not come from the
       target release, unless the installed version is more recent

   500 <= P < 990
       causes a version to be installed unless there is a version
       available belonging to the target release or the installed version
       is more recent

   100 <= P < 500
       causes a version to be installed unless there is a version
       available belonging to some other distribution or the installed
       version is more recent

   0 < P < 100
       causes a version to be installed only if there is no installed
       version of the package

   P < 0
       prevents the version from being installed

   P = 0
       has undefined behaviour, do not use it.

We’ll discuss what a “target release” is next, but first some general comments. Priorities between 500 and 989 are the ones that behave as you probably expect: if all your repositories are in this range, the package will be installed and/or updated from the highest priority repository that has it. If there’s a tie, the repository with the newest version wins. You will never downgrade a package–so if one is installed from a lower priority repository it won’t get replaced by an older version in a higher priority repository.

The other priority ranges have special properties. If you set a priority over 1000, this will cause packages from that repository to be used, even if it results in a downgrade. It isn’t often that this is what you want, so you want to generally stay away from this. Priorities below 0 will cause packages to never be installed, unless you manually set the repository as a target.

In this example, we’ve set the package priority for lunar to -10, which means that packages will never get installed from there. If we try to install a package that is in lunar, but not in jammy, the install will fail. If you were to instead set lunar to a priority of 600, then jammy will be the default source for packages, but if a package is not in jammy, lunar will be checked for it. I find this useful in Debian, where stable is sometimes missing things that you may want to install, but haven’t needed it in Ubuntu as of yet.

Installing a Package from a Specific Repository

We have blocked installing packages from lunar by default. If we want to install something from there, we can force it by setting lunar as the target repository when we call apt. As an example, at the time of this writing the yt-dlp package is on version 2022.04.08 in the jammy repositories. So if we install it, we’ll see,

% sudo apt install yt-dlp
% yt-dlp --version
2022.04.08

This version is actually broken now, because YouTube has updated its API and so trying to use it will result in an error,

% yt-dlp https://www.youtube.com/watch?v=NQ5uD5x8vzg
[youtube] NQ5uD5x8vzg: Downloading webpage
[youtube] NQ5uD5x8vzg: Downloading android player API JSON
ERROR: [youtube] NQ5uD5x8vzg: Unable to extract uploader id; please report
this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out
the appropriate issue template. Confirm you are on the latest version
using  yt-dlp -U 

However, we can forcibly install a newer version from lunar by using the -t flag in apt to specify an install target repository,

% sudo apt install -t lunar yt-dlp
% yt-dlp --version
2023.02.17
% yt-dlp https://www.youtube.com/watch?v=NQ5uD5x8vzg
<actually works now!>

By specifying the target, we override the block on using packages from lunar. Further, this package will now be updated based on the repository it was installed from, so when a new version is pushed to lunar, apt will download it without us needing to do anything special.

There is one caveat to this. If the package you install has any dependencies, these may be satisfied from lunar if necessary. This means that any other packages that share this dependency will need to be updated to their lunar versions too. If you later attempt to install a package that depends on an older version of one of these dependencies, apt will fail. This is because the installed package will come from the jammy repositories, and the dependencies installed on your system will be the newer ones from lunar. Unless jammy is set with a priority greater than 1000, apt won’t downgrade these dependencies to match the package you’re installing, and so it won’t be able to install the package. Should this happen, you need to install the version of this package from lunar instead, using the -t lunar flag in the your install command.

Rolling Everything Back

It’s possible because of all of these dependency shenanigans that you will accidentally upgrade more stuff to the newer repositories than you want, or that you accidentally break something. Should this happen, fear not! apt is a powerful tool, and is more than capable of fixing things.

If you decide that you regret everything and want to roll back to the default jammy versions of all of your packages, apt can do that, and it probably won’t break your system. All you need to do is bump the jammy entry in your /etc/apt/preferences file to have a priority of over 1000, and apt will force all the packages it can into compliance. This may result in some packages being uninstalled, if apt can’t make it work out.

More specifically, update /etc/apt/preferences so that it contains,

Package: *
Pin: release n=jammy
Pin-Priority: 1100

Package: *
Pin: release n=lunar
Pin-Priority: -10

and then run,

% sudo apt update
% sudo apt upgrade

This will cause apt to downgrade packages as necessary to ensure that they all are coming from the jammy repositories.

Conclusion

Pinning packages with apt is an incredibly useful feature, and has changed the way that I look at “stable” distributions like Ubuntu LTS and Debian. Using this feature, you can have a stable base system, without losing access to new versions of specific software that must be kept up to date, or without missing out on the latest features in some userland applications. In a sense, it gets you the best of both worlds: rolling release for selective userspace applications on top of a stable base system.

I used Arch Linux for over a decade before I learned about pinning, and now I’m happily running Debian as my main Linux distribution. Selectively installed packages from the unstable branch let me me have the latest toys, but I don’t have to deal with the instability of running the core system components under a rolling release model.

apt converted me away from Arch Linux–I just had to learn how to actually use it first.