geschätzte Lesezeit: 1 min

In my workflow with Git, I prefer using SSH authentication for platforms like GitHub and GitLab. It allows me to avoid saving multiple credentials on my machine; instead, I simply register my already existing public SSH key in my account settings. It seems like a clear victory, right?

Unfortunately, it's not a one-size-fits-all solution:

  • Recently, I encountered an issue that could be easily resolved using Git submodules in GitLab CI. GitLab has the ability to automatically check out submodules, albeit only if they are referenced using their HTTPS URLs.
  • The convenience of copying and pasting HTTPS URLs of GitLab/GitHub repositories directly into your git clone command is undeniable. I've been guilty of lazily using these HTTPS URLs when I assumed I'd only need read-only access, only to later realize that I had to switch them over to SSH URLs...

However, to my surprise, I discovered the following configuration options in Git. You can actually substitute the visible URLs of remote repositories based on their prefixes! This means that you can use SSH for local development and HTTPS for CI environments without having to manually switch back and forth.

Here's how to do it:

[url "ssh://git@github.com/"]
    insteadOf = https://github.com/
[url "ssh://git@gitlab.com/"]
    insteadOf = https://gitlab.com/

This config ensures that every time you try to interact with a remote URL that starts with https://github.com/ or https://gitlab.com/, Git will substitute it with the corresponding SSH URL. This means you can continue using HTTPS URLs in your .gitmodules file, but when you clone or pull locally, it will use SSH. It also works for git clone.

I have no idea how I missed this feature before, but it's certainly a quality of life improvement.

geschätzte Lesezeit: 1 min

I have a problem with mixing unstable, stable & home-manager that I cannot figure out. Any help would really be appreciated.

I am trying to use helix in its newest version (23.10) instead of the previous (23.05). This works for root, but not for non-root:

$ hx --version
helix 23.05
$ sudo -i
# hx --version
helix 23.10 (f6021dd0)

which is what is expected with the follwing configuration (3 files following):

I bring in home-manager and unstable like this:

sudo nix-channel --add https://github.com/nix-community/home-manager/archive/release-23.05.tar.gz home-manager
sudo nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixos-unstable
sudo nix-channel --update

My /etc/nixos/configuration.nix is essentially this:

{ pkgs, ...}:
let
  unstable = import <nixos-unstable> {};
in {
  imports = [
    ./hardware-configuration.nix
    <home-manager/nixos>
  ];
  nix.settings.experimental-features = [ "nix-command" "flakes" ];
  environment.systemPackages = [
    unstable.helix
    pkgs.zsh
  ];
  users.users.jps = {
    # ...
  };
  home-manager = {
    extraSpecialArgs = { inherit unstable; };
    useGlobalPkgs = false;
    useUserPackages = true;
    users.jps = import ./home.nix;
  };
  system.stateVersion = "23.05";

The home.nix is essentially

{ config, pkgs, unstable, ... }:
{
  imports = [
    ./helix.nix
  ];
  home = {
    stateVersion = "23.05";
    sessionVariables = {
      EDITOR = "hx";
    };
    packages = [
      pkgs.helix
      pkgs.zsh
    ];
    # ...
  };
}

and helix.nix is just including a configuration file:

{
  programs.helix.enable = true;
  xdg.configFile."helix/config.toml".source = ../common/helix/config.toml;
}

Now, if I try to use the helix package from unstable (like root does) by changing home.nix to

    packages = [
      unstable.helix
      pkgs.zsh
    ];

then I get a collision:

error: collision between `/nix/store/nayfcdjkl4d1n5w5c0n72np99zvy9x3m-helix-23.10/bin/.hx-wrapped' and `/nix/store/1g9pz0sc47y6v3q7ya6mik0gj37vjiay-helix-23.05/bin/.hx-wrapped'

Why is that?

geschätzte Lesezeit: < 1 minWarum Matrix? :: Matrix Dokumentation

Matrix an der TU Dresden > Warum Matrix?

Es gibt sehr viele Dienste und Programme zum instantanen elektronischen Kommunizieren. Matrix wurde ausgewählt, da es sich durch folgende Eigenschaften auszeichnet:

  • Offener Standard (keine technische Sackgasse)
  • Kommunikationsprotokoll für föderierte Echtzeitkommunikation (keine isolierte Lösung sondern wie E-Mail mit weltweiter Konnektivität)
  • Differenzierte Kommunikation möglich (1:1, multiple Teams, themenspezifisch)
  • Dezentralisierte, persistente und interoperable Kommunikation (keine zentralen Kontrollinstanzen, verbindbar mit anderen Protokollen und Tools)
  • Datenschutz: Ende-zu-Ende-Verschlüsselung ist Standard in 1:1-Gesprächen und (Gruppen-)Räumen, in letzteren bei Bedarf ausschaltbar
  • Web-Anwendung + Desktop-Client + Mobile Apps vorhanden
    • Die Kontakte auf dem eigenen Handy werden nicht automatisch an Dritte weitergegeben (kein Adressbuch-Upload)
  • Anbindung an bestehende Authentifizierungssysteme (bspw. der TU Dresden)
  • Bestehende erfolgreiche Sicherheitsauditierung
  • Autonomie und Kontrolle: lokal installierbar (Serverstandort TU Dresden)
  • Erkennen des Anwesenheitsstatus
  • Aktive Weiterentwicklung
  • Aufstrebende Nutzung durch weitere Wissenschaftseinrichtungen (bspw. viele deutsche Hochschulen)

Ausführliche Informationen: https://matrix.org/faq/#intro

Ausführliches Fachbegriffs-Glossar: https://matrix.org/faq/#definitions

Hochschulen und Universitäten mit einem Matrix-Dienst:

Text and images are licensed under CC BY-SA 4.0.

geschätzte Lesezeit: 8 min

This post is part of a series about NixOS. The series is done in support of my upcoming Opus Magnum, "Practical NixOS: the Book", and you can read the detailed post here.

Should you find this article lacking some information, be sure to tell me in the comments or elsewhere, since it is a living document and I am very keen on expanding it.

This article's full code alongside the rest of configuration is available on GitHub.

Table of Contents (Click to Hide/Show)

Home Manager is a tool for managing user environments using the Nix package manager.

That's it, pretty succint.

But why? Isn't nix already doing that for you? Well, there are several reasons why.

First of all, Home Manager provides an abstraction to manage user-specific configuration. By that I mean dotfiles and packages, which are unique to a single user on your nix-enabled machine. If you happen to have a necessity for such a thing, you're better off using Home Manager, else you would end up implementing parts of Home Manager by yourself.

Next, Home Manager would be the only sensible choice for a non-NixOS nix-enabled system to be declaratively-managed. For MacOS though, Nix-darwin exists, but we'll take look at that one in another article.

Last but not least, Home Manager provides a lot of configurable options, which simplify your nix life a lot, e.g. it has a one-liner not present in NixOS Options to enable Wayland. These nice things could also have been a part of Nixpkgs or NixOS Options, and that also leads us to another question:

The best answer you may get comes from a NixOS Discourse thread, with replies from Nix creator Eelco Dolstra and Home Manager creator Robert Helgesson (rycee).

Long story short: Robert Helgesson took inspiration from an already existing tool called nixuser which served a similar purpose, and, in his own words,

As it turned out, the basic functionality of installing packages, generating configurations, linking files into $HOME, and so on is actually very simple and it was pretty easy to just implement everything from scratch.

So, for the historical reasons Home Manager exists as a separate project, and it was also later added to the Nix Community GitHub Organization which is the umbrella org for the officially endorsed Nix tools.

Eelco Dolstra says, that many things in nixpkgs find themselves as they are because Nix/NixOS was (back then) lacking proper modularity (and it is now solved with flakes), that led to tightly-coupled systems. Having Home Manager as a separate project makes complete sense in that regard, and allows for "both quality and speed of development". Also think Conway's Law.

There are talks however that it would be beneficial to merge some parts of Home Manager into nixpkgs, e.g. some general utility functions.

As per Home Manager docs, there are 3 possible ways to install it:

  1. Using the standalone Home Manager tool. For platforms other than NixOS and Darwin (MacOS), this is the only available choice.
  2. As a module within a NixOS system configuration. This allows the user profiles to be built together with the system when running nixos-rebuild.
  3. As a module within a nix-darwin (MacOS) system configuration. This allows the user profiles to be built together with the system when running darwin-rebuild.

3rd option is out of scope for us, since we assume for now that you're on NixOS. 1st one does not apply to us either, since for the sake of simplicity and convenience on a single-user machine that our setup is, we would like to manage both the system and the home configurations with the same and single application of the nixos-rebuild command. Thus, we are left with the 2nd option - using Home Manager as a NixOS module.

Note, that we'll be using a flaked-out NixOS system which we have configured previously. You may want to refer to the previous post on how to enable flakes if you haven't yet, and take the initial configuration from there. The final result for the NixOS system with flakes and Home Manager enabled is always available on the GitHub.

Using Home Manager with flake gives us the possibility to not use a separate tool to perform upgrades.

First, let's add Home Manager urls to the inputs. In your /etc/nixos/flake.nix:

inputs = {
  nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  home-manager.url = "github:nix-community/home-manager";
  home-manager.inputs.nixpkgs.follows = "nixpkgs";
};

The line home-manager.url = "github:nix-community/home-manager"; should be pretty obvious, as it fetches the flake of home-manager from GitHub, evaluates it and makes whatever outputs are available within that remote flake for our local flake.

The home-manager.inputs.nixpkgs.follows = "nixpkgs"; is less straightforward, but this is actually a flake built-in functionality to reduce the number of inputs a flake depends on. In our case, both local flake and the remote home-manager flake depend on nixpkgs. Should you omit this line, the result would be evaluated using 2 possibly distinct nixpkgs with one of them being referenced in the remote flake.lock on Home Manager's GitHub repo, and the other one in the local flake.lock for your own flake. This won't cause any trouble, but would affect the size of the data stored in the Nix store, as well as increase the required bandwidth and evaluation time, so if you wan't to save these, you would use home-manager.inputs.nixpkgs.follows = "nixpkgs";. It may cause some problems though, if, say, Home Manager depends on something that is not present in the nixpkgs from your (possibly outdated) flake.lock, but such scenario is unlikely.

Next, let's make home-manager availabe within the scope of our flake:

outputs = inputs@{
  self,
  nixpkgs,
  home-manager,
  ...
}: {
  # omitted
}

What does it do? Per flake docs,

outputs: A function that, given an attribute set containing the outputs of each of the input flakes keyed by their identifier, yields the Nix values provided by this flake.

So, outputs = {}: {} is a function assigned to a key within the attribute set (which is the flake itself).

The first pair of curly brackets {}: contains the function arguments, with the outputs provided by the other flakes (nixpkgs and home-manager found under respective urls listed in inputs).

The second pair of curly brackets {} is an attribute set, which is also the outputs function body. It contains all the key-value pairs which are to be made available for the reference as a flake output, either for standalone consumption, or as an input within some other flake.

So, if you check the Home Manager flake source, you'll see a lot of possible outputs, like nixosModules, darwinModules, defaultTemplate, etc.

Now, we want to reference one of these outputs within our local flake, and we do it exactly like this:

modules = [
  ./configuration.nix
  home-manager.nixosModules.home-manager
  {
    home-manager.useGlobalPkgs = true;
    home-manager.useUserPackages = true;
    home-manager.users.theNameOfTheUser = import ./home.nix;
  }
];

In this code, the interesting part is home-manager.nixosModules.home-manager - it is a utility function which is usable as a NixOS module, and it accepts some attribute set as function arguments. nixosModules.home-manager is defined in this file.

The rest are the options provided by the said function, as per their source definition:

  • home-manager.useGlobalPkgs = true; - use the system configuration's pkgs argument in Home Manager. This disables the Home Manager options nixpkgs
  • home-manager.useUserPackages = true; - install the user packages through the users.users.theNameOfTheUser.packages option
  • home-manager.users.theNameOfTheUser = import ./home.nix; load home.nix to be used as the configuration for the user theNameOfTheUser

Instead of importing a file (which may be) called home.nix (or some other name), we may directly inline it into the flake.nix for the sake of a smaller config, simplicity, and visibility:

home-manager.users.theNameOfTheUser = { pkgs, ... }: {
  home.username = "theNameOfTheUser";
  home.homeDirectory = "/home/theNameOfTheUser";
  programs.home-manager.enable = true;
  home.packages = with pkgs; [
    # your desired nixpkgs here
  ];
  home.stateVersion = "23.11";
};

The functionalities of these options are self-evident from the names, except for the home.stateVersion = "23.11";. From this option's definition:

It is occasionally necessary for Home Manager to change configuration defaults in a way that is incompatible with stateful data. This could, for example, include switching the default data format or location of a file. The state version indicates which default settings are in effect and will therefore help avoid breaking program configurations. Switching to a higher state version typically requires performing some manual steps, such as data conversion or moving files.

Let's add a couple of packages via Home Manager's provided options, so these packages would be made available as user-specific and not system-wide:

home.packages = with pkgs; [
    thunderbird
    keepassxc
  ];

Both Thunderbird and KeePassXC are available in nixpkgs!

The whole flake now looks like this:

{
  description = "flake for yourHostNameGoesHere with Home Manager enabled";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    home-manager.url = "github:nix-community/home-manager";
    home-manager.inputs.nixpkgs.follows = "nixpkgs";
  };
  outputs = inputs@{
    self,
    nixpkgs,
    home-manager,
    ...
  }: {
    nixosConfigurations = {
      yourHostNameGoesHere = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          ./configuration.nix
          home-manager.nixosModules.home-manager
          {
            home-manager.useGlobalPkgs = true;
            home-manager.useUserPackages = true;
            home-manager.users.theNameOfTheUser = { pkgs, ... }: {
              home.username = "theNameOfTheUser";
              home.homeDirectory = "/home/theNameOfTheUser";
              programs.home-manager.enable = true;
              home.packages = with pkgs; [
                thunderbird
                keepassxc
              ];
              home.stateVersion = "23.11";
            };
          }
        ];
      };
    };
  };
}

Finally, just do the usual nixos-rebuild --flake .#yourHostNameGoesHere switch unser sudo from within the /etc/nixos directory, wait till Nix is finished rebuilding and applying the configuration, and we're done!

You now have a flaked NixOS with user environment managed by Home Manager!

This article's full code alongside the rest of configuration is available on GitHub.

geschätzte Lesezeit: 25 min

In the previous post I took a look back on my dotfiles journey through the last decade or so. That post started as a preface to this one, but grew to the point where it deserved a post of its own.

As I’ve journeyed deeper into the world of NixOS I have realized that there were some major issues with the way I managed machine configurations. What I’m presenting here is the outcome of that cleanup work. Now I have something I’m quite happy with and I rounded off the last post by saying:

I feel it’s a very exciting setup and would like to dive into the details, but those are saved for another post… Stay tuned!

The triggering moment for when I finally decided to refresh my system configurations turned out to be this tweet from Mitchell Hashimoto, which again led me to his YouTube video demonstrating his macOS virtual machine setup. Since I’ve recently been investing more time and effort into flakes and combined with an increasing annoyance of the state of my system configuration it was the perfect timing to start basing configurations fully on nix flakes. To begin with I started copying pretty much the same things Mitchell was doing, but I quickly started digging more into how others were building NixOS from flakes (see “Note:” section below). I have since borrowed whichever tricks and methods I found neat or useful, and so I’ve gathered inspiration from quite a few sources in the end.

The title of the post is obviously a word play on “configuration” and “federation”. I’ve gone through the process of uniting together the various bits and pieces of a previously unstructured configurations repository into something which feels much more consistent, not just as a source tree, but also the machine states it generates. Components can be shared across various deployments and architectures, yet configurable to accommodate for special needs. With higher guarantees for reproducibility the configurations are now also much more robust.

I only started experimenting with this setup not too long ago and so I consider it highly immature still.

Neither am I a pioneer in this space by any means and there are other configurations much more mature than this personal experiment of mine. Here are a notable few of the ones I’ve come across so far:

The devos README.md also lists the “Shoulders” upon which it’s built. Go check it out for more inspiration!

I should also note that if you, the reader, feels as if I’m reinventing the great stuff the projects above already solve, you’re absolutely right! One of my goals with this migration has been to further deepen my knowledge of NixOS machine configuration, usage of the nix language and flakes. And I’m bound to make a bunch of silly mistakes people have made before me of course, but therein lies some of the fun!

The flake

Let’s gets started by jumping straight in to the action by looking at the flake.nix.

This post will assume some familiarity with nix flakes. I suggest reading up some of the following resources to get your bearings on the basics:

The flake.nix is included as follows in its entirety, it’s not too long:

{
  description = "myme's NixOS configuration with flakes";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    nixos-wsl.url = "github:nix-community/NixOS-WSL";
    flake-utils.url = "github:numtide/flake-utils";
    agenix = {
      url = "github:ryantm/agenix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    home-manager = {
      url = "github:nix-community/home-manager/master";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    doomemacs = {
      url = "github:doomemacs/doomemacs";
      flake = false;
    };
    i3ws.url = "github:myme/i3ws";
    annodate.url = "github:myme/annodate";
    nixon.url = "github:myme/nixon";
    wallpapers = {
      url = "gitlab:myme/wallpapers";
      flake = false;
    };
  };
  outputs = { self, nixpkgs, flake-utils, home-manager, ... }@inputs:
    let
      system = "x86_64-linux";
      overlays = [
        inputs.agenix.overlay
        inputs.i3ws.overlay
        inputs.annodate.overlay
        inputs.nixon.overlay
        self.overlay
      ];
      lib = nixpkgs.lib.extend (final: prev:
        import ./lib {
          inherit home-manager;
          lib = final;
        });
    in {
      overlay = import ./overlay.nix {
        inherit lib home-manager;
        inherit (inputs) doomemacs wallpapers;
      };
      # NixOS machines
      nixosConfigurations = lib.myme.allProfiles ./machines (name: file:
        lib.myme.makeNixOS name file { inherit inputs system overlays; });
      # Non-NixOS machines (Fedora, WSL, ++)
      homeConfigurations = lib.myme.nixos2hm {
        inherit (self) nixosConfigurations;
        inherit overlays system;
      };
    } // flake-utils.lib.eachDefaultSystem (system:
      let pkgs = import nixpkgs { inherit system overlays; };
      in {
        # All packages under pkgs.myme.apps from the overlay
        packages = pkgs.myme.apps;
        devShells = {
          # Default dev shell (used by direnv)
          default = pkgs.mkShell { buildInputs = with pkgs; [ agenix ]; };
          # For hacking on XMonad
          xmonad = pkgs.mkShell {
            buildInputs = with pkgs;
              [ (ghc.withPackages (ps: with ps; [ xmonad xmonad-contrib ])) ];
          };
        };
      });
}

And as seen by nix:

❯ nix flake show .
git+file:///home/myme/code/dotfiles?ref=refs%2fheads%2fmain&rev=37e5dccd614bfb4b6e369697e7c285327ef59668
├───devShell
│   └───x86_64-linux: development environment 'nix-shell'
├───homeConfigurations: unknown
├───nixosConfigurations
│   ├───Tuple: NixOS configuration
│   ├───map: NixOS configuration
│   ├───nuckie: NixOS configuration
│   ├───qemu-server: NixOS configuration
│   ├───qemu-vm: NixOS configuration
│   └───vmware: NixOS configuration
├───overlay: Nixpkgs overlay
└───packages
    └───x86_64-linux
        └───git-sync: package 'git-sync-0d0s33l2..hhjz'

I try to keep the flake.nix “high level” and easy to navigate. So I factor out unnecessary details into helper functions and nix expressions that typically gets placed into ./lib.

Inputs

Everything under inputs are the inputs (aka. dependencies) of the configuration flake. To ensure reproducibility all inputs are locked in the flake.lock file which is managed by nix commands, but is also added as a git file to track its history of changes.

The inputs typically list other nix flakes that have a similar structure to the one from my dotfiles repo. The <input>.url field is resolved using nix registry symbolic identifiers and we can also tell nix that an input is not a flake by using <input>.flake = false. This lets a flake track and pin for instance any git repository, which is very convenient. In my case I use that for tracking Doom Emacs and my wallpaper repo.

Outputs

The outputs (more interestingly that the inputs) list the various derivations, configurations and environments that the flake can generate. In this case it’s the following:

  • A nix overlay
  • A set of x86-64 packages
  • NixOS configurations
  • Home Manager user profiles
  • A development environment

The outputs section starts with the parameter list, or perhaps more correctly the destructuring of the input attribute set. Flake outputs should define a function which is applied to the attribute set of flake inputs. Only a few of the inputs are bound in the function parameter list since @inputs is used to pass on all the inputs to helper functions later on.

outputs = { self, nixpkgs, flake-utils, home-manager, ... }@inputs:

Nixpkgs and overlays

After the parameters come a few local variables: system, overlays and pkgs.

let
  system = "x86_64-linux";
  overlays = [
    inputs.agenix.overlay
    inputs.i3ws.overlay
    inputs.annodate.overlay
    inputs.nixon.overlay
    self.overlay
  ];
  pkgs = import nixpkgs {
    inherit system overlays;
  };

The system is hard-coded to x86-641 and used to instantiate pkgs from the nixpkgs input. In the “pure” world of flakes the system argument to nixpkgs cannot be deferred from the running system and must always be provided. The last one is overlays which is just a list of all the nix overlays in use. Most overlays come from the other input flakes, but it also includes its own overlay from self.overlay:

overlay = import ./overlay.nix {
  inherit home-manager;
  inherit (inputs) doom-emacs wallpapers;
};

The overlay of the flake just adds library functions, apps and packages that are bundled along with the dotfiles. This makes them available to the NixOS configurations and Home Manager profiles as well as possible to export using:

# All packages under pkgs.myme.apps from the overlay
packages = pkgs.myme.apps;

NixOS & Home Manager

I find that the next part of the flake.nix is quite interesting. This is the NixOS configurations and Home Manager user profiles generated from my dotfiles:

# NixOS machines
nixosConfigurations = pkgs.myme.lib.allProfiles ./machines (name: file:
  pkgs.myme.lib.makeNixOS name file {
    inherit inputs system overlays;
  });
# Non-NixOS machines (Fedora, WSL, ++)
homeConfigurations = pkgs.myme.lib.nixos2hm {
  inherit overlays system nixosConfigurations;
};

This makes use of a couple of (home grown) library functions that automatically generate host/machine configurations and associated user profiles from files on disk. Instead of enumerating all machines in the flake.nix there is an allNixFiles function that finds files and directories under ./machines/ and treats them as individual host configurations:

{ lib }:
dir:
let
  isNixFile = { name, type }: type == "directory" || lib.strings.hasSuffix ".nix" name;
in builtins.map (x: x.name) (builtins.filter isNixFile
  (lib.mapAttrsToList (name: type: { inherit name type; })
    (builtins.readDir dir)))

Host names are inferred from the basename of each directory entry under ./machines/:

❯ tree machines/
machines/
├── map.nix
├── nuckie.nix
├── qemu-server.nix
├── qemu-vm.nix
├── Tuple.nix
└── vmware.nix

This lists only the public part of my dotfiles and does not show an example of a “directory” based host configuration. Basically that’s similar to the top-level ones except defined as a directory with a default.nix and possibly auxiliary files.

The nixos2hm function extracts user profiles from all the NixOS machine configurations, exposing the Home Manager profiles for all users on all machines. This is particularly useful for non-NixOS environments so I can reuse “global” NixOS configurations even though I don’t build a complete NixOS system profile. As a more concrete example, I use the machine role configuration to determine whether or not the machine should have graphical tools installed or not. By defining a “mock” NixOS machine for these machines the Home Manager part of the configuration can still depend on the NixOS configurations and make decisions based on the values:

{ home-manager, lib }:
{ overlays, system, nixosConfigurations }:
let
  removeHostname = str: builtins.head (builtins.split "@" str);
  userAtHostConfig = { host, config }: (
    lib.mapAttrsToList
      (username: hmConfig: {
        name = "${username}@${host}";
        value = hmConfig.home;
      })
      config.home-manager.users
  );
in with builtins; (listToAttrs (concatMap userAtHostConfig
  (lib.mapAttrsToList (host: config: {
    inherit host;
    inherit (config) config;
  }) nixosConfigurations)))

Since nix flake show only lists homeConfiguration as “unknown” we can use the nix repl to list all user profiles pulled from the machine configurations:

❯ nix repl
Welcome to Nix 2.9.0pre20220530_af23d38. Type :? for help.
nix-repl> :lf .
Added 13 variables.
nix-repl> builtins.attrNames homeConfigurations
[
  "myme@Tuple"
  "myme@map"
  "myme@nuckie"
  "myme@vmware"
  "nixos@qemu-server"
  "nixos@qemu-vm"
  "nixos@vmware"
  "user@qemu-server"
]

Development shell

Finally there is the devShells. Flakes can define multiple shell development environments. The Haskell toolchain involved with hacking on xmonad is quite heavy, so I keep that in a separate shell environment called xmonad that’s not used by default. In the default shell I currently only expose agenix which I use to manage the small set of secrets (aka. semi-sensitive data).

devShells = {
  # Default dev shell (used by direnv)
  default = pkgs.mkShell { buildInputs = with pkgs; [ agenix ]; };
  # For hacking on XMonad
  xmonad = pkgs.mkShell {
    buildInputs = with pkgs;
      [ (ghc.withPackages (ps: with ps; [ xmonad xmonad-contrib ])) ];
  };
};

Machines

The configurations I push to github.com/myme/dotfiles is of course not the whole truth of the configurations I have for machines. Traditionally I’ve been keeping various machine configurations off in separate host branches. Each branch with a set of tweaks specific for that machines only. These tweaks are mostly mutually exclusive since that’s the whole point of not propagating it back to the shared main branch:

"dotfiles git branches"

My workflow has been to make changes to a machine and try it out for anything from a couple of seconds to weeks or even months. Changes that are useful for other machines are rebased to the beginning of the “host branch” and the main branch is updated by a simple fast-forward to the rebased commit. All other host branches are then rebased in turn on top of the main branch. Ideally the set of tweaks should be few to minimize general configuration differences between machines as well as reducing the hassle of conflicts while rebasing.

The boilerplate of setting up a configuration for each machine is done with the makeNixOS function:

name: machineFile: { inputs, overlays, system }:
let
  inherit (inputs) self agenix home-manager nixpkgs nixos-wsl;
in nixpkgs.lib.nixosSystem {
  inherit system;
  modules = [
    ../system
    ../users/root.nix
    agenix.nixosModule
    nixos-wsl.nixosModules.wsl
    home-manager.nixosModules.home-manager
    machineFile
    {
      # Hostname
      networking.hostName = name;
      # Let 'nixos-version --json' know about the Git revision
      # of this flake.
      system.configurationRevision = nixpkgs.lib.mkIf (self ? rev) self.rev;
      # Nix + nixpkgs
      nix.registry.nixpkgs.flake = nixpkgs;  # Pin flake nixpkgs
      nixpkgs.overlays = overlays;
    }
  ];
}

It includes several upstream NixOS modules that are pulled in as flake inputs: nixos-wsl, home-manger and agenix. The ./system directory contains most of the defaults for a machine as well as custom NixOS configurations that mostly serve as high-level feature management for each machine:

options.myme.machine = {
  name = lib.mkOption {
    type = lib.types.str;
    default = "nixos";
    description = "Machine name";
  };
  role = lib.mkOption {
    type = lib.types.enum [ "desktop" "laptop" "server" ];
    default = "desktop";
    description = "Machine type";
  };
  flavor = lib.mkOption {
    type = lib.types.enum [ "nixos" "generic" "wsl" ];
    default = "nixos";
    description = "Linux flavor";
  };
};

The name is obviously used to give the machine a hostname, and so on.

The role determines if it’s a headless server machine, a laptop that requires e.g. battery and power management, or a desktop computer.

The flavor is used to specify the Linux flavor of the installation. Is this a NixOS machine, a “generic” Linux like Ubuntu or Arch, or WSL.

Generic config

Each machine configuration follow a pretty similar setup. The following is the configuration for map, a WSL installation running on a Microsoft Surface tablet:

#
# `map` is a Windows 11 machine and this configuration is for WSL on that host.
#
# Graphical apps are supported, but unfortunately not GL, see:
#
#   https://github.com/guibou/nixGL/issues/69
#
{ config, pkgs, ... }: {
  myme.machine = {
    role = "desktop";
    flavor = "wsl";
    highDPI = false;
    user = {
      name = "myme";
      # This maps to the `users.users.myme` NixOS config
      config = {
        isNormalUser = true;
        initialPassword = "nixos";
        extraGroups = [ "wheel" "networkmanager" ];
        openssh.authorizedKeys.keys = [];
      };
      # This maps to the `home-manager.users.myme` NixOS (HM module) config
      profile = {
        imports = [
          ../home-manager
        ];
        config = {
          home.packages = with pkgs; [
            mosh
          ];
          programs = {
            # SSH agent
            keychain = {
              enable = true;
              keys = [ "id_ed25519" ];
            };
            ssh = {
              enable = true;
              includes = [
                config.age.secrets.ssh.path
              ];
            };
          };
          myme.dev.haskell = {
            enable = true;
            lsp = false;
          };
        };
      };
    };
  };
  age.secrets.ssh = {
    file = ./../secrets/ssh.age;
    owner = config.myme.machine.user.name;
  };
}

Most of the configuration goes into the myme.machine configuration. This is because it also contains the home-manager configuration for each machine under myme.machine.user.profile which is mapped directly to the home-manager configuration options.

On hardware

I have a couple of machines running regular NixOS on hardware. My main work computer is a Lenovo P1 laptop and the configuration for it is not currently public. However, one useful part of its configuration is the use of nixos-hardware which is added as a flake input and provides useful configurations for the Lenovo P1’s quirks:

imports = [
  inputs.nixos-hardware.nixosModules.common-gpu-nvidia
  inputs.nixos-hardware.nixosModules.lenovo-thinkpad-p1
  inputs.nixos-hardware.nixosModules.lenovo-thinkpad-p1-gen3
  ./hardware.nix
];

…and don’t get me started on the nvidia graphics of that machine:

# NVidia - 😭
hardware.nvidia = {
  package = config.boot.kernelPackages.nvidiaPackages.beta;
  modesetting.enable = true;
  powerManagement = {
    enable = true;
    finegrained = true;
  };
};
services.xserver.displayManager.sessionCommands = ''
${lib.getBin pkgs.xorg.xrandr}/bin/xrandr --setprovideroutputsource NVIDIA-G0 modesetting
'';

The X server still craps itself when adding/removing external displays, so it’s not a very desirable setup for hot-desking at the moment. Besides the regrets of picking out a nvidia-based laptop, I’m quite happy with the power and screen of the P1 in general though.

I also have an Intel NUC that I use for home automation which runs NixOS directly on hardware. It’s not a graphical installation and there’s not really much more interesting to say about it.

VMWare

My daily driver in the days of home office is an AMD Ryzen desktop computer running Windows on the metal. It’s the same computer I use for home studio music production and most of the relevant music editing software I use is mostly macos and Windows only. I expect there would be quite a bit of suffering jumping onto Linux-based music production, and so I haven’t justified spending time on it.

Since the machine is powerful enough for my needs I don’t have any trouble doing most of my work in VMWare virtual machine and have been doing so for several years. There’s not much interesting to say regarding the VMWare-specifics in my NixOS setup, besides perhaps the guest tools:

# VM
virtualisation.vmware.guest.enable = true;

WSL

I’ve been making more and more use of Windows Subsystem for Linux v2 over the past months. Thanks to the excellent NixOS-WSL project I’m now able to run a pretty much complete NixOS installation on my Windows machines.

To learn more about how you can run NixOS on WSL check out Xe’s post Nix Flakes on WSL.

My configuration enables the NixOS-WSL configurations if the machine has a Linux flavor of wsl:

(lib.mkIf (config.myme.machine.flavor == "wsl") {
  wsl = {
    enable = true;
    automountPath = "/mnt";
    defaultUser = config.myme.machine.user.name;
  };
})

For WSL I also don’t include the boot and some networking parts of the configuration:

# Disable boot + networking for WSL
(lib.mkIf (config.myme.machine.flavor != "wsl") {
  # Boot
  boot.loader.systemd-boot.enable = true;
  boot.loader.systemd-boot.configurationLimit = 30;
  boot.loader.efi.canTouchEfiVariables = true;
  boot.kernelPackages = pkgs.linuxPackages_latest;
  # Network
  networking.networkmanager.enable = true;
  networking.firewall.enable = true;
})

QEmu

Even with the atomic rollbacks that NixOS provides it can be convenient to occasionally try out experimental configurations in a controlled environment such as a virtual machine. I’m already relying on VMWare or VirtualBox for some hosts and could of course use those virtualizers to test out configuration.

However, NixOS provides a very convenient sub-command to build QEmu virtual machines through nixos-rebuild build-vm. Through clever mounts of the host’s nix store the guest gets access to a read-only version of it. Also with the performance KVM provides it’s a very quick and lightweight way to spin up a configuration:

# QEmu
#
# Full graphical NixOS setup on QEmu.
#
{ config, lib, pkgs, ... }: {
  myme.machine = {
    role = "desktop";
    flavor = "nixos";
    user = {
      name = "nixos";
      # This maps to the `users.users.nixos` NixOS config
      config = {
        isNormalUser = true;
        initialPassword = "nixos";
        extraGroups = [ "wheel" ];
      };
      # This maps to the `home-manager.users.nixos` NixOS (HM module) config
      profile = {
        imports = [
          ../home-manager
        ];
        config = {
          myme.wm = {
            enable = true;
            variant = "xmonad";
            conky = false;
            polybar.monitor = "Virtual-1";
          };
        };
      };
    };
  };
  # Security
  security.sudo.wheelNeedsPassword = false;
}

From my dotfiles repo such a configuration can be built and run using:

nixos-rebuild build-vm --flake .#qemu-vm
./result/bin/run-qemu-vm-vm

Bringing NixOS to life inside QEmu running in NixOS in WSL (yo dawg, I heard you like virtualization):

image

Raspberry PI

I have a Raspberry PI that I’ve momentarily relieved from service as its purpose being a home automation driver has been replaced by a NUC. I intend to bootstrap the PI with NixOS at some point, but it’s not really on the top of my personal backlog. It would be a fun exercise though as it would allow me to test some other architectures for NixOS.

Should I end up doing this I’ll try to make sure I’ll write about it and add a link to it here.

The NixOS: On Raspberry Pi 3B post is now available!

2022-12-06

Configuration

The Home Manager configuration entry point is home-manager/default.nix and should be familiar to those who’ve already used Home Manager:

{ lib, pkgs, ... }: {
  imports = [
    ./barrier.nix
    ./dev.nix
    ./emacs
    ./git.nix
    ./irc.nix
    ./nixon.nix
    ./spotify.nix
    ./tmux.nix
    ./vim.nix
    ./wm
  ];
  config = {
    # ...
  };
}

I’m slowly but surely trying to move to a structure where the ./home-manager directory and configurations are the same for all machines, but controlled through custom high-level configurations. This in an attempt to try to minimize the differences between machines. The alternative would be to include Home Manager configuration modules into each machine configuration, which does lead to more boilerplate and repetition.

I don’t really want to go into deeper details regarding how I structure my Home Manager stuff, because I don’t really think it’s unique in any way. Going in-depth sounds like a topic for a dedicated future post.

Secrets

I honestly don’t have many things I consider secrets in my configurations at the moment. However, since I’m trying to move more and more common configurations into my main dotfiles branch I do want to hide some configurations, like ~/.ssh/config hosts and whatnot.

I’ve found agenix to be simple enough for my needs so far. For my use-case I use ssh key infrastructure to encrypt secrets to each machine’s host key for sshd.

agenix needs a (by default) ./secrets.nix file containing the keys associated with each encrypted secrets file:

let
  hostKeys = {
    map =
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILaNxtQ37YaiXRXx+Ff3sPEbzsjA2i934r0Bl+eXVh3P root@map";
    tuple =
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMhSCm/KiFfhkTcLaza/GFrpPVEzIFhALxM6gBmNK3Gi root@Tuple";
  };
  userKeys = {
    map =
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII1Qsv8MA+cyu7n+4H1kpbVrAmOosJJxjPWAdl08YDvL myme@map";
    tuple =
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH+9tnNlMesGrK/lDvycgzyS4pPrsGqcGQP6yLCsr/LN myme@Tuple";
  };
in {
  # Files to manage - used by the agenix cli to encrypt/decrypt
  "./secrets/ssh.age".publicKeys =
    [ hostKeys.map hostKeys.tuple userKeys.map userKeys.tuple ];
}

The agenix command line tool can the be used to edit the secrets file (it’s available through the default devShell, remember?):

agenix -e ./secrets/ssh.age

For each machine the secrets file must have a corresponding entry in the NixOS configuration:

age.secrets.ssh = {
  file = ./../secrets/ssh.age;
  owner = config.myme.machine.user.name;
};

The secrets can then be included from the ~/.ssh/config by referencing e.g. the path attribute, like so:

myme.machine.user.profile.config.programs = {
  ssh = {
    enable = true;
    includes = [
      config.age.secrets.ssh.path
    ];
  };
};

The ssh.includes attribute then resolves to the following line in the ~/.ssh/config file:

Include /run/agenix/ssh

agenix will then ensure that files are decrypted using each machine’s sshd host key and the content made available through files under /run/agenix.

In the repl

One thing that I find quite useful (and also quite amazing) is using the nix repl to browse through nixpkgs and flakes. This is a great way to explore the properties of derivations and other nix expressions. I can’t get over how awesome it is to do exactly that with your NixOS configuration as well.

When loading the dotfiles flake into the nix repl I can browse the various nixosConfiguration and homeConfigurations as well as build individual apps and programs. This is great to explore a system configuration to learn or validate that it’s configured correctly. And it’s not restricted to the configuration for the current system, but all systems specified in the current flake.

As an example, I was debugging why the X server didn’t want to start properly in the qemu-vm example configuration. I realized I had to have a look at some of the generated files for the user’s home directory. Instead of tweaking configurations, rebuilding and booting the VM, I could simply load up the repl and build the user’s home-files, a part of the NixOS configuration provided by the Home Manager’s NixOS module. The built derivation contains all the dotfiles that will be symlinked into the user’s home directory, which can be easily inspected:

I should note that you don’t need the repl to build sub-parts of a NixOS like this. The tab completion and interactive navigation of the repl makes exploration a lot more seamless.

One of the most empowering benefits of automatic version locking/pinning is how easy it makes working with unstable software “channels”2. Personally the main reason it’s a hassle to base machine configurations on unstable upstreams is when things break exactly because of upstream updates. It can be immensely frustrating when anything from a simple configuration to the whole system stops working because of an uncontrolled software update.

Having controlled and atomic rollbacks fixes part of this problem as it gets you back to a working state and NixOS have been having this for a long time. Having complete control of when to upgrade fixes most of the remaining issues. An unstable upstream is only as unstable as the new features that come in, so by ensuring that upstream updates don’t automatically trickle down to your system the system state for a locked version should remain the same. For example if your system is working well under a specific version of nixos-unstable it is stable for you and should remain so until you decide to pull in new changes.

Whereas nix channels traditionally have been updated globally either on the system or user level, flake inputs are locked to specific versions for each individual project. Firstly, this means that no global command affect how individual nix flake projects manage their dependencies. Furthermore, a flake can easily track unstable upstreams as it simply locks down that upstream to a specific version ignoring new changes to the upstream and only on the user’s request will it update that lock information. This happens primarily due to two things:

  1. The user changes a flake input url.
  2. The user runs nix flake lock --update-input <input>.

Updating inputs

$ nix flake lock --update-input nixpkgs

And similarly for a non-flake input:

$ nix flake lock --update-input doomemacs

I mentioned I have both a NUC and a RaspberryPi, which aren’t the most powerful computers. I have experimented a bit with using my desktop computer to build the NixOS configuration for the NUC. It worked well enough once I started getting grips on the binary cache signature checks of nix. I did end up with some unusable derivations that were synced over and that I weren’t able to properly remove. One of the key parts (no pun intended) was to set the binaryCachePublicKeys to include that of the host that built the configuration:

nix.binaryCachePublicKeys = [
  "tuple:RLwVT0X7XUres7PkgkMLgsMfWhbHP0PYIfQmqJ2M6Ac="
];

I definitely see a lot of potential of doing remote builds using nix. Tools like NixOps have been around for a while, but I have yet to really give it a spin. Alternatively, there are other options like serokell/deploy-rs which seem to integrate even nicer with flakes.

Bootstrapping

The majority of time spent managing NixOS running on general purpose machines is done through changing configurations and then invoking nixos-rebuild to apply it. This, of course, depends on the system already running NixOS.

I had some big plans making sure that bootstrapping new machines with NixOS would be a breeze with proper automation. Due to a couple of reasons I haven’t been able to deliver on this promise to myself yet.

In the interim I’ve first of all not really (re-)installed machines all that much - and for the few times it’s happened I’ve used a plain NixOS installation medium and pretty much installed machines the way I’ve always done.

Then, once a machine is running NixOS I copy the flake template from whichever machine is the most similar and tweak it to my specific needs.

I have done some experimentation with scripting the installation process and managed to get it working quite well for any machine running a live-ISO version of NixOS. It requires a running system with an open SSH port. Then the bootstrap/copy.sh script uses rsync to copy over the configuration.

The scripts can be found under ./bootstrap in the dotfiles repo, but aren’t really very configurable making them neither very impressive nor useful.

There are obviously things to improve when it comes to the bootstrapping of new machines. It’s somewhat less rewarding work because there’s often quite a long time between each time I actually need to setup a new machine from scratch. However, with the structure of the dotfiles settling down for now I might find some motivation to eventually improve that too. Perhaps it could be fun to combine it with something like serokell/deploy-rs. I don’t know…

Rounding off

I’m very happy I started on this next chapter of the never-ending journey of configuration management. I had to make quite a few changes and updates to have things fit into this next-generation solution. Regardless of this, most of the actual configurations and setup have been preserved from the previous non-flakes NixOS setup and non-NixOS, standalone Home Manager approach. I believe it’s good to be making these kind of incremental changes, not worrying about getting everything “right” or “perfect” the first time around, but rather have most things working along the way.

As always, I don’t expect I’ll ever be done, but another configuration milestone reached. Fun was had, and so it was time to share my experience of migrating to a fully flakes based NixOS configuration across all machines.

Let me know if you also decide to take the plunge!

Thanks to @evenbrenden for proof-reading.

Footnotes

  1. For the time being I don’t have any interesting machines that run on anything other than x86-64. This is why I haven’t made the flake map over other system architectures, although that’s a very trivial fix to make should the need arise.↩︎

  2. Not talking about nix channels here.↩︎