Skip to content

New Server (LXC)

This is the instruction guide for setting up a new LXC server.

Rough steps are as follows: 1. Plan the servers config 2. Login to Proxmox 3. Create Container using the base LXC image 4. Login to Server 5. Push new configuration

Plan server Configuration

To allocate he correct resources there are a few questions that need to be asked and answered.

  1. What will this server be For?
  2. What will its Name be?
  3. What will its IP be?
  4. Fill the details into the tracking sheet.

What is it For?

What a server is for dictates what hardware resources need to be allocated.
Is there one already existing that you can copy the configuration of?
Do you have prior experience with what will be hosted on it?
Is there documentation that you can use as a foundation?

When you know these, write it down and save for later.

What is its Name?

In our cluster we have a very definite naming scheme for the servers we have.
There are two rules:

  1. The login server (where folks have their home dirs and websites) is called Skynet.
  2. All other servers are also named after AI's

In the current cluster we tend to use groups of AI names for particular functions.
For example:

  • Vigil/Vendetta were AI's in the Mass effect series, pointing Shepherd forward, so they are our DNS servers
  • Glados/Wheatly from Portal, that game runs on the Source Engine, so they are our Source control servers
  • Optimus/Bumblebee from Transformers, their origin is a line of toys, so fittingly our games servers.
  • Neuromancer/Wintermute, from Neuromancer, each with multiple minds, thus became our backup (redundancy) servers.

If at all possible try to get the name to match its task.
Some past names and ideas for others can be found on the nixos wiki

What is its IP address?

We have a /26 allocation, so about 60 IP's we can make use of.
Like with the names above several servers are grouped together IP wise.
Check the Server Inventory to see what addresses are available.

Tracking sheet

Now that ye have all the details about the build it is time to add them to the Server Inventory.
Add a new one, incrementing the index, and fill in the rest of the information.

For the IP address if the last segment (ABC in this: 193.1.99.ABC) is less than 100 then add a leading 0 to it.
This is so that it can be easily sorted.

Login to Proxmox

Login to Proxmox

Create Container Using the Base LXC Image

The Proxmox documentation for LXC's is available here.

Top right there is a button [Create CT], that brings up a window.
Each section below is one of the tabs in the window.

General

Hostname

This is the Name of the server, lowercase.

Unprivileged

Ensure this is ticked.

Nesting

Ensure this is ticked.

SSH Public Keys

Enter the root pub ssh key.
This is used to login to teh container later.

Template

Select the container image, most likely nixos-system-x86_64-linux.tar.xz.

Disks

Storage

Most likely it is main_pool, it should have a significant amount of storage available. DO NOT use local-zfs, this is on Proxmox's own drive and not suited for container data.

Disk Size

Self-explanatory, how much space you want to give teh container.
A minimum of 30Gb is suggested.

CPU

One core minimum, larger servers will require up to 6 or so.

Memory

Nixos will happily run on 512Mb if its load is not too intensive

Network

Bridge

The main bridge we use is vmbr0 which is for most servers as it connects to the normal Skynet DMZ.

We also have vmbr1 which is for skynet.skynet.ie and connects to Skynet-EXT DMZ.
This is due to our users needing ssh access.

IPv4

IPv5/CIDR

This is the IP followed by /26.
For example 193.1.99.75/26

Gateway

The main gateway we use is 193.1.99.65.
There is a secondary one for skynet.skynet.ie which is 193.1.96.161.

DNS

You can either use use host settings or fill in 193.1.99.120 and 193.1.99.109.

Confirm

Use this as a chance to review all the options.
There are a few gotcha's outlined above that ye do have to look over.

Start after created

Tick this box if you want it to boot up immediately after being installed.

Login to Server

Now that the server is up and running it is time to login to it.
Assuming you have your SSH configured like (TODO: admin ssh config).
You just have to use ssh root@IP (for example ssh root@192.99.1.111)

Getting the server ssh key

We are logging in because we need to get the servers own ssh key.
You can find it in /etc/ssh.
You have a choice between ssh_host_ed25519_key.pub and ssh_host_rsa_key.pub.
ssh_host_ed25519_key.pub is the recommended one.

Using the server ssh key

This key is used to decrypt secrets stored in our Nixos repo and as such needs to be added there.
1. Add the key with the other system keys here.
2. Add it to the systems array underneath that. 3. In the secrets folder run cd secrets && agenix -r to rekey the secrets. * This is to give the new server access. 4. Commit all the changed files

Push new configuration

Create config

An example server config for nixos is available here.
Copy it to a new file and name it name.nix.
Fill in all relevant details (name/ip/name details/its purpose/...).

Finally add it to the bottom of the flake.nix.
Save and commit the files.

Push Config

There are two ways to test out the config, manual and pipeline.
Each has pros and cons.

Manually

This requires your key to be added the the keys for the root account and your ~/.ssh/config to be set up properly (TODO: SSH tutorial)

  1. Build it with colmena build --on name
  2. If it builds ye can test deployment
  3. Deploy it using colmena apply --on name

Pipeline

Just push it to the repo and the pipeline will handle building and deployment.
The disadvantage of this is you cannot fix any mistakes before they are pushed.