Revision Hell

Anyone who reads this will be familiar with the Python issue of using a different version of the language. That is just one glaring example where even users are affected. This is due to old brilliant projects that have stopped maintaining the software. In many other situations, you also need great control over what is running and what libraries are available. Programming in C and C++ uses libraries that often need to be the exact version when you compile. Otherwise, you will be re-writing parts of the software you never intended to touch. Many developers use a container with all the libraries, and all other works happen on the host computer.

The Nix Fix

How does nix take care of this problem? Well, they have all the files in a store with hashes to identify the exact version. The environment you are going to use, then link to the library or execute, is something you would want to use for your current situation. For a running system, you can then use many versions of an application and even libraries. When you want to develop, you create a configuration file that covers the needs of your current project.

Configuration Files

When you have NixOS installed, the configuration.nix will control your environment for the whole computer. With that said, you can control it in every instance of your shell. Irrespective if you have NixOS or run any other distribution, you can use another nix file. The file is called default.nix by default. You can use this to make a directory structure that has a particular environment. The workflow is to create the default nix file to reflect what you want your environment to support. Then change the directory and run nix-build, followed by running the nix-shell. You can also use any name for the file if you specify it on the command line.

The parameter for the nix-shell will be implied, but if you want to have several in one directory, then you can use the switch. With the correct values set, you now have your environment the same every time you start nix-shell. If you move the nix file, you will be able to get the same anywhere! The big issue becomes; what do I put in the nix files?

The files use the Nix expression language, it is almost a programming language.

A few examples

Below, there are a few examples that can help you. There are many more things you can do to tweak your environment. This is a long exciting journey, but it will probably slow you down from the beginning. Before you get there, use other people’s code. This list is short, so look for ideas across the web.

Python

When you want to create a Python project, you would normally use virtual environments. With Nix, this is not necessary. Instead, you can create a shell.nix file that declares which version you want. The simplest way to do this is to use python38Full.

This compiles an environment with all parts of the Python 3.8 that comes with NixOS. If you want to minimize your environment, you can choose its particular parts. You can also add your source code remotely with fetch functions.

Python Flask

An example of web development is a flask. It is a very powerful package for making web pages, even really complex ones, without much effort. Since Flask is a popular framework, there is a ready NixOS package for it. The file to control the build for this is called default.nix.

As you can see, there are packages from nixpkgs that cover flask. If you want to use something else, you add them inside the square brackets. This goes for all types of packages that are included in the NixPkgs repository. If the package does not exist, use a fetcher.

Python Development

If you want to start a Python development environment, you add packages you need according to revision and others.

In the shellHook, between the double apostrophes (”), you can put any scripts you like. Again, think about what might already exist, as there are many smart people out there that are already developing using NixOS.

JavaScript

The standard version to use JavaScript, or more precisely, nodejs, is the nix script below. Name it shell.nix and place it in your project directory, then start with the nix-shell command.

This is the simplest, possible trick, although there are much more available. You can see how to add a script that you would otherwise run manually. Use this carefully and look for full alternatives before you do this.

Jupyter

The script below initializes a directory to host a batch of functions where you can run Jupyter. The other packages are for statistics and machine learning. You can also remove and add according to your needs.

Configurations

For your IDE, editor, or anything, really, you can also bake in your settings. For developers, vim and Emacs will be the first candidates for this specialization. Vim has its own set of plugins available as nixpkgs.

Fetchers

The basis of the packages in NixOS are files that point to sources and what is needed for compiling the packages. You can use this if you are missing a package. As long as you can find the source package, you can use a fetcher to install it. The standard fetcher fetches tarballs but is named fetchurl.

You can use it the way it is in the above code. You also have fetchgit and other version control systems. On top of this, the major git services are covered with fetchFromGitHub, fetchFromGitLab, and more. With all these fetchers, you should be able to find any package you want for NixOS.

Conclusion

Using NixOS requires a bit more effort than other distributions. Having said that, if you want to develop software, the effort is worth it. You will keep your main system cleaner and can hop between projects without creating troublesome conflicts between environments.

In particular, it is a non-free software package, so you must enable this option. You will also need to handle the ‘glXChooseVisual failed’ problem. The process will work one way in NixOS and another way on other distributions. It is more complex with just the Nix package manager.

What is Steam?

Most people who come already know this but let’s cover it here anyway. Steam is a platform and market for games and gamers. It started as a way to update your game from Valve. It was the only one for the first time in life. As the company added more games, they also added them to the platform. With so many games available, they rebuilt it to be a market place and community platform. You can now both play and buy games and stay in touch with fellow gamers on the platform. Given all this, of course, you want to have it installed on your NixOS system.

Installing the Main Executable

There have been some problems with Steam on NixOS in the past. The problems were solved but still require some extra actions compared to other packages.

One issue is that this is not free software. Second, the packages use Glx of the 32-bit variant, something that is not clearly reflected in the packages. These two issues need to be addressed in the setup of the package manager: Nix or NixOS configuration (.nix) file. The actual solution was to set the dri support 32bit value to true. There were a few others, but thanks to a new module from Maciej Krüger, you can now just add the module with the code below.

This is a module that has solved several problems with some quirks of the Steam software. Once you have this set correctly, you can run the install. If you are interested, the below code is from the commit that adds the module to make it happen.

As you can see in the code, it activates the support for 32-bit direct rendering and audio. It also adds the package ‘pkgs.steam’, which is the main Steam package. With the earlier configuration, you should get the whole system up and running at the next rebuild switch. Once you have saved your configuration, run:

For most of you, this will allow the install to go forward. You now need to make sure you have enough disk space for the install. Also the games you will install need disk space too.

If things go wrong, use:

There are many other optional packages to install if you have any special needs or desires.

nixpkgs.steam-run (steam-run)

Why do you need steam-run? Steam-run makes it possible to run using NixOS libraries instead of the ones Steam provides. This may work better when you want to run games that expect a regular Linux system beneath. Some games need patching to run, using the Steam environment. The reason is that only the Steam provided games are built for a closed environment. To use these, add the steam-run or steam-run-native to your configuration file.

You can also use steam-run directly as a command, like this:

This will run the game in a Steam environment directly.

Missing Dependencies

Some games may need dependencies that NixOS does not automatically provide. To fix this, you can add them to the configuration file under systemPackages.

The above code adds dependencies for many cases. You will pick the ones you need yourself, of course. You can also look for other dependencies that may be missing. However, you will be on your own if you do, so be prepared to use the terminal to start and trace it when you ask for help on the different support forums.

Other Useful Packages

You also have some special packages that may help you with some issues.

nixpkgs.steamcmd (steamcmd)

This package adds Steam command-line tools. You can use this for installing software and running your own servers; some tasks can be automated.

You also have many other packages available. To use them, you add them to your packages and rebuild. The currently available ones are below:

nixpkgs.kodiPlugins.steam-launcher (kodi-plugin-steam-launcher)

Launch Steam in Big Picture Mode from Kodi

nixpkgs.pidgin-opensteamworks (pidgin-opensteamworks)

Plugin for Pidgin 2.x, which implements Steam Friends/Steam IM compatibility

nixpkgs.bitlbee-steam (bitlbee-steam)

Steam protocol plugin for BitlBee

nixpkgs.eidolon (eidolon-1.4.6)

A single TUI-based registry for drm-free, wine, and steam games on Linux, accessed through a rofi launch menu

nixpkgs.kodiPlugins.steam-controller (kodi-plugin-peripheral.steamcontroller)

Binary addon for the steam controller

nixpkgs.matterbridge (matterbridge-1.18.0)

The simple bridge among Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via XMPP), Matrix, and Steam

nixpkgs.steamcontroller (steamcontroller)

A standalone Steam controller driver

nixpkgs.sc-controller (sc-controller-0.4.7)

User-mode driver and GUI for Steam controller and other controllers

Conclusion

Steam presents a small problem because a large part of the platform and some games still require 32-bit libraries, and you need to enable that. Hopefully, you have gotten your answer here. If not, you may ask on the forums! NixOS is extremely versatile, but getting to grips with the Nix language is a chore. When you switch, make sure you have some fundamental understanding of the language to avoid long winding searches for solutions. You should be able to come up with many yourself with enough grasp of the Nix language.

What does NixOS Package Manager do?

Most package managers use a file that contains the executable or source code. They then calculate what it needs on the system and then make sure that it exists. In Nix, things work very similarly. The big difference is that Nix creates all the files, and compiles them if necessary, then put them in one place; the nix-store. The first question you have may be, “Will the files not have the same name?” The system avoids this by having one directory for each version AND naming all files with a hash. To make the application “feel at home”, all dependencies are then linked to their correct directories using ordinary symlinks. A profile keeps track of which version each user runs.

NixOS User Installs

With this system, you can have different versions installed in each user’s directory. If they are the same in several users, the administrator can let Nix re-link binaries, so only one exists at a time. This is useful in saving disk space. You can also create specific environments for each version of the package. This is especially useful when you want to test a new version or develop software.

Installing for common distribution

For most common platforms, you can install Nix, the package manager with a simple script. This script is available on the Nix website. The script will need root user access, but if you are very security conscious, you should read the script before you use it. If you want to avoid using root in the script, just create the /nix directory on your system.

If you have no root access or just super cautious, you can have Nix as a user only package manager.

This binary works well for most, if not all, distributions. Platforms are x8664, i?86, aarch64, and x8664-darwin, which cover almost all platforms available. If you use any other platforms, you can probably use the source code and build your own. When the installation is done, you will then have a bunch of new commands.

Adding your first program to NixOS

To install software and set when it can be used, you have nix-env. The install option (-i) is the most common one since you use it always and put a package as an argument.

This looks the same as in other distributions, so does the query argument. The install will take some time, though. The reason is that it must compile the software unless there is a pre-compiled version in the Nix Cache. Reaching the cache is not always very fast either. There is a difference that you should take note of; you can pick a version! If you want a special version, you must find which are available using regular expressions.

You will receive a list of all the available packages. You can install it the same way but using the value in the list.

This can fail if you already have an installed version. Option ‘–preserve-installed’ will not erase the installed version. You may end up with two versions of the same priority, which you can fix by setting the priority.

Now, you will run the old version the next time you start Firefox. To switch which one you run, you can set the priority accordingly. You can also start a shell to choose a binary. This is a developer’s option, and the command is nix-shell.

Updating NixOS

Once you have a collection of software, you want to stay updated. Same as always, you use the same command with an argument. But you must also keep the channel updated. The command is nix-channel.

This reads down the current versions of all packages available. After that, you can start upgrading your software with nix-env.

An upgrade like this will upgrade your old version of the software. In this case, the old Firefox will be replaced with the newest version. You may not want this for whatever reason, usually development.

Removing applications from NixOS

Removing applications is equally simple, with a small caveat. No applications are removed by a remove command.

This command will remove the links to the current build of this version of Firefox. All the files will always stay on disk. You have these versions available to do a rollback. A rollback means that you go back to using the old version. This can be useful if you have tried the newest and it has unforeseen problems.

You rollback an entire generation, which means all the programs that were upgraded since the last generation. The option runs two commands; that list and then switches to that old generations. All installed packages exist in a generation on disk.

NixOS Roll-back and Cleaning up

The rollback function will lead to a lot of disk space being used by old versions. You can clean this up (you need to clean this up!). When you have had a long enough period, at your own choice, you can also clean up these old generations to save disk space.

With this command, you delete all generations except the two last ones. You can go back and forth in the list with more complex parameters to leave the specific generation that worked best for you. Unless you have many testing or development projects that need many versions for testing, you should use a scheduled removal of all old generations.

A simple script to keep your generations clean comes with a Nix package manager install.

You should also set up the collector to run automatically using systemd or other systems.

Conclusion

Nix package manager is a powerful system that can get you running complex development environments on your machine. You can also use it to keep your software tidy and have a simple way to recover on a new machine, should the catastrophe of a disk crash occur.

With that said, if you can follow the NixOS manual, you will be fine. You can choose a default desktop environment if you want, but make sure you are comfortable with the command line and can edit a text file for configuration tasks.

A powerful configuration

The ability to configure NixOS is both an advantage and a challenge. Traditional package managers bring the package into the established LSB structure of the files. In NixOS, the installer puts the files in the store with a hash before it. This convention may sound complicated, but it enables many features.

When you install a program, the package manager prepares a directory with all files and adds links to the positions where they should be placed. It also copies the dependencies in the same directory and links them in the structure. To track which programs need which dependencies, a profile is used. With the store and the profiles, you can have many different combinations of packages.

You can also switch over with a few commands, and rolling back is super easy: just pick the old generation at the next reboot. If you are playing around with configurations, you will end up with many generations. However, you can use nix-collect-garbage -d to clear the boot partition (although you must them run the nixos-rebuild command!).

Handling revisions

In the Nix Store, where all your software is stored, you have one file for every executable. At first glance, this convention appears no different from those adopted by other systems; however, there is a big difference: Every time you upgrade, a new binary is added and then linked to your profile, which can very quickly lead to wasted disk space.

To address this issue, there is another garbage collection option, which is the same program that is used with the entire system. If you need old revisions for only a short test period, then you can set systemctl to run at a regular interval. Furthermore, you can save disk space by using the ‘nix-store –optimise’ command, which finds identical files in the store and links the files to that one file.

Setting up development environments

At first, it seems hard to develop software with this system. In fact, you can start a shell with a specific development environment each time. When you pick an environment, nix-shell will install the environment you need so that you can start a specific environment for some odd language you never use or create a file that collects everything you normally need.

Docker and other clouds

NixOS is an operating system, and Nix is a package manager. The two work together to provide a straightforward and reproducible configuration process. In other words, if you create a full configuration file that covers all your needs, then you can use that for your next machine.

The installation procedure starts by detecting hardware. In the second step, you define your environment and system packages using the configuration.nix file. Once you have the correct content in the file, the installer will recreate the same system when you use it on a second machine.

This functionality is useful because, for regular systems, a new disk needs only the file to rebuild your system (in addition to your user file backup, of course). Furthermore, for cloud computing, you have an even bigger advantage: While the files you need to write for a docker image are really long, the corresponding file for NixOS is short and easy to move between systems. In addition, you can use the import function to create special nix files for your odd configurations and import them into your config.

Appimage, snap and flatpak

While NixOS has many brilliant ways to run your applications and separate them from each other, a lot of software is delivered in other ways. Appimages and Flatpak are easy to use to distribute packages. Fortunately, NixOS has packages for handling these formats, and you can install these packages to run your favourite AppImages and Flatpaks. You can define the packages in your configuration.nix file and have them available when you need them.

Conclusion

NixOs seems intimidating because it has no graphical installer and you need to create a configuration file. However, only in NixOS do you set the same values in both cases. To back up a NixOS system, not including the user files, only a single file is needed. With this file, the system recreates the packages and settings. Furthermore, NixOS provides a built-in method for running a shell in a specific environment: Just use the same type of file! In the file default.nix, you can define all your libraries and dependencies and then run nix-shell in that directory.

This system has a lot of potential. Try it out: You can start with your own distribution and the nix package manager.

While you can run applications under NixOS because they follow the above standard, the files are not where they would be in another system. The developers of NixOS and GNU Guix have strong opinions about this system, and they have come up with clever ways to comply with it.

A different system

Your software storage system affects functionality in a way that is much deeper than it seems at first glance. For the software to find the files it needs, NixOS uses symlinks. Each application has its own directory that contains the executable and links to the libraries that run it.

With this organisation system, you can have different files and versions installed at the same time. By default, all packages and their dependencies should compile during installation. However, it requires a lot of time and processing power to do so at every install, there are caches.

Downloading

With NixOS, there is always more than one way to do something. Like other distributions, with NixOS, you have an ISO on a USB stick. You have choices regarding how you want to install NixOS on your distribution. However, before we discuss this topic in more detail, it is important to understand that there are two slightly confusing parts of this process.

First, Nix is different from NixOS, and you must understand the difference between Nix, the package manager, and NixOS, which configures your system. You can download the Nix package manager and use it on your current system. With it, you can keep many versions of applications on your system without them interfering with each other.

Second, with NixOS, while you cannot not declare the partitioning scheme, everything else can be left in one file. Most users leave the automatically created hardware configuration file alone. When you first start out, you can keep your packages declared in the file, but over time, you will probably make separate files that you import into your configuration file.

Partitioning

Before installation, you must partition your drives. In other distributions, there are defaults you can accept; however, with NixOS, you must do your own partitioning. Partitioning is not very complex, but you can run into trouble when you have to set your configuration for the partitioning scheme you choose. It is important to understand that the instructions and scripts prefer if your file systems are labelled correctly.

The standard manual shows the partitioning commands. Note that the commands differ for a UEFI and an MBR disk, and setting the wrong values will cause many problems. The manual suggests using the values provided below for the initial installation, but it is really easy to test new values.

Standard partitions:
MBR:

UEFI:

Mounting the partitions in MBR:

Mounting the partitions in UEFI:

The next section will show you how to create your configuration file.

The Config File

Once you have your disks set up, you can start the configuration process. With NixOS, you configure first and then install. The following instructions assume that you have booted using the ISO, but you could boot with chroot.

With nixos-generate-config, the system generates a standard configuration file.

This command creates two files: /mnt/etc/nixos/hardware-configuration.nix (you do not change this file) and /mnt/etc/nixos/configuration.nix. You can edit the second file in your favourite editor.

Usually, the options do not change depending on the method used to boot. You can use grub or another boot configuration. There are many options, but here are some standards.

Add this line for MBR only:

Add these lines for UEFI only:

Change the config files as little as possible to start. These values are all in the original file: just un-comment them and make changes to fit your needs.

Add the packages you want to use as standard packages. All standard packages go in the square brackets with wget and vim. You can install more packages once the system is running.

Building

Once your configuration file is correct, you can run the install.

Next, the installer will ask for a root password that will be used on the real system. All programs will be compiled or downloaded from cache.nixpkgs.org and then installed in the nix store on your computer. Then, you can reboot, and you should get a login prompt.

Now, provide a password for your user using root. The user you defined in the configuration file will also have a home directory.

New Config

Once you have completed the above steps, you can play around with the configuration file. When you change something, try it out without installing it as follows:

Once you have new values that work well, run the rebuild command:

Now, you will see if you have set the boot values correctly. It is important to note that the changes to the configuration are reversible. You can simply reboot and choose an older version, which is called a generation, and try again. However, each configuration does require disk space, so make sure you are familiar with the garbage collection function.

Conclusion

NixOS requires a few manual steps to set up, but you can return to a running system much quicker than with other systems. Furthermore, version control is easier if you need many versions of the same application or if you need many versions of the same libraries. At first glance, there may seem to be many limitations, but these limitations can be overcome with the more advanced parts of the system.