Colmena

Overview

Colmena is a deployment tool which makes it easier to deploy multiple servers at once.
This can be the Skynet cluster or multiple home versions.

Examples

Here are some examples of colmena being used with teh folder stucture outlined below

https://forgejo.skynet.ie/Skynet/nixos
- The Skynet repo itself
https://gitlab.com/nix17/personal
- @silver 's repo'

Folder structure

For best results I recommend a folder structure as follows:

nixos/
├─ applications/
│  ├─ application1.nix
│  └─ application2.nix
├─ machines/
│  ├─ $MACHINE1/
│  │  ├─ configuration.nix
│  │  ├─ hardware-configuration.nix
│  └─ $MACHINE2/
│     ├─ configuration.nix
│     └─ hardware-configuration.nix
├─ .gitattributes
├─ .gitignore
└─ flake.nix

The rest of this page will go through how to set up the basic structure.

Move out from `/etc/nixos`

By default Nixos stores its configuration in /etc/nixos.
However to edit teh files you need root access (sudo)

Hostname

Colmena deploys based on hostname.
Normally this is fine since Nixos will set a default hostname of nixos.
However if you have two devices, each calling themselves nixos this can cause issues.

The solution is to gie each device its own hostname.
In the examples above Skynet uses the names of AI for hostnames.
In Brendan's one he uses greek gods.

Change default hostname

Edit networking.hostName = "nixos"; in /etc/nixos/configuration.nix to be whatever hostname you want
sudo nano /etc/nixos/configuration.nix
TODO: Add note on how to use nano

If the new hostname is glados then:

-networking.hostName = "nixos";
+networking.hostName = "glados";

Rebuild and apply
sudo nixos-rebuild switch
Reboot teh computer to get teh new hostname
sudo reboot now
You should now see the new hostname in teh terminal ($USER@$HOSTNAME)

Copy files out of `/etc`

Setup the base Repo

For the new foundation I would recommend using https://forgejo.skynet.ie/Skynet/project_base.

# create the new directory and move into it
mkdir -p ~/nixos
cd ~/nixos

# Clone the template into the ~/nixos folder we are currently in
# the dot is required and signifies the current folder
git clone https://forgejo.skynet.ie/Skynet/project_base .
# Dont want to be setting the origin to be the project_base so remove it here
# Set this to your own (remote) repo later.
git remote rm origin

Copy in the files

Now that we have the repo set up its time to import in our current configuration

# we have no need for the pre defined folders so get rid of them
rm -rf docs src

# will be useful for splitting out common configs later
mkdir -p applications

# where $HOSTNAME is the new hostname of your computer
# in the example above it would have been machines/glados
mkdir -p machines/$HOSTNAME

# copy in teh configuration.nix and the hardware-configuration.nix
cp /etc/nixos/* machines/$HOSTNAME

# we need to create a file to put the config in
## KDE
kate flake.nix
## gnome
gnome-text-editor flake.nix
## general (will use this for the rest of teh tutorial)
## OPens the file in the default editor
xdg-open flake.nix
# ctrl+s to save after pasting in teh file below

# create teh abse config file that is shared across all machienes
xdg-open machines/base.nix
# ctrl+s to save after pasting in teh file below

Now we need to fill out the flake.nix to make it work. You can copy the file below using teh copy button at the top right of the section as the image is below.
Change glados to whatever your $HOSTNAME is.

copy_icon

`flake.nix`

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-unstable";

    # utility stuff
    home-manager.url = "github:nix-community/home-manager/release-26.05";
    flake-utils.url = "github:numtide/flake-utils";
    colmena.url = "github:nix-community/colmena";
    alejandra = {
      url = "github:kamadorueda/alejandra";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    alejandra,
    colmena,
    ...
  } @ inputs: let
    pkgs = nixpkgs.legacyPackages.x86_64-linux.pkgs;
  in {
    formatter.x86_64-linux = alejandra.defaultPackage."x86_64-linux";

    devShells.x86_64-linux.default = pkgs.mkShell {
      name = "Build env";
      nativeBuildInputs = [
        pkgs.buildPackages.git
        colmena.packages."x86_64-linux".default
      ];
    };

    colmenaHive = colmena.lib.makeHive {
      meta = {
        nixpkgs = import nixpkgs {
          system = "x86_64-linux";
          specialArgs = {
            inherit inputs;
          };
        };
      };

      # installed for each machine
      defaults = import ./machines/base.nix;

      # colmena apply-local --sudo
      glados = import ./machines/glados/configuration.nix;
    };
  };
}

`machines/base.nix`

Copy this into machines/base.nix and adjust as required.
These need to be replaced with teh correct values:

REPLACEME_USER
REPLACEME_KEY
REPLACEME_FIRSTNAME
REPLACEME_SURNAME

{
  pkgs,
  inputs,
  lib,
  ...
}: {
  imports = [];

  config = {
    nix = {
      settings = {
        # flakes are essensial
        experimental-features = ["nix-command" "flakes"];
        trusted-users = [
          "root"
          # replace REPLACEME_USER with your username
          "REPLACEME_USER"
        ];
      };

      # to free up to 20GiB whenever there is less than 5GiB left
      extraOptions = ''
        min-free = ${toString (1024 * 1024 * 1024 * 5)}
        max-free = ${toString (1024 * 1024 * 1024 * 20)}
      '';
    };


    # Define a user account. Don't forget to set a password with ‘passwd’.
    users.users = {
      root = {
        initialHashedPassword = "";
        openssh.authorizedKeys.keys = [
            # this is part of the remote step, return to this later with the value of
            # cat ~/.ssh/id_ed25519.pub
            "REPLACEME_KEY"
        ];
      };

      # change the below to suit youself, change REPLACEME_USER to your own username
      # some of this might have been in the configuration.nix already
      REPLACEME_USER = {
        isNormalUser = true;
        description = "REPLACEME_FIRSTNAME REPLACEME_SURNAME";
        extraGroups = ["networkmanager" "wheel"];
      };
    };

    security.sudo.extraRules = [
      {
        groups = ["wheel"];
        commands = [
          {
            command = "ALL";
            options = ["NOPASSWD"];
          }
        ];
      }
    ];

    # Allow unfree packages
    nixpkgs.config.allowUnfree = true;

    # List packages installed for everyone
    # this is a useful set of tools to have installed
    environment.systemPackages = with pkgs; [
      # git is key for this system
      git git-lfs
      # terminal based process monitors
      htop bottom nvtopPackages.full
      # file tree explorer
      ncdu
      # this is how ye get teh shiny termianl showing off yer system
      hyfetch
    ];

    # allow ssh access
      services.openssh = {
        enable = true;
        settings.PermitRootLogin = "prohibit-password";
      };
  };
}

Add everything to git

Flakes need everything to be at least added to teh repo.
Although it is good practice to commit

# -A adds everything, includong hidden files
git add -A

# Commit the files, you can replace the message with whatever ye want
git commit -sm "initial commit"

Update

To drop into a shell with colmena run this command

nix develop

However if its teh first time you might need to do

nix develop --extra-experimental-features nix-command --extra-experimental-features flakes

Local

To be abe to use this command you need to add this to your machines/$HOSTNAME/configuration.nix under teh imports

xdg-open machines/$HOSTNAME/configuration.nix

Add this below the imports

deployment = {
  allowLocalDeployment = true;
};

Command

colmena apply-local --sudo

Remote

To deploy remotely we must add an ssh key to teh root user.

To do that we generate a key

# check if you have a key
# if this succeeds then use the value outputted
# if this fails then follow the steps after this
cat ~/.ssh/id_ed25519.pub


# generate the key
# skip if ye already havea key
ssh-keygen
# then press enter 3 times

# copy the output of this file
cat ~/.ssh/id_ed25519.pub

# Add the value to base nix,replace  REPLACEME_KEY with the value
# then save and exit

xdg-open machines/base.nix

# apply yer changes
colmena apply-local --sudo

Single computer

colmena apply --on $HOSTNAME

Group

(Add this shortly)