Last year I spent more hours playing Minecraft than I care to admit. Being an introvert that I am, I play on a small private server with my partner and a few friends, and Minecraft Realms has been a great hosting for us, fast, cheap, secure, with backups. The only issue — it can’t have mods, and eventually we wanted to play with mods. After shopping around for hosting services and trying a couple of them we came away mildly disappointed: they were either very slow, or very expensive. I decided that I will simply move the server to my former gaming laptop, which has been sitting idle for the last year or so. It had plenty of power and, most importantly, RAM, while being fairly energy-efficient. The cost of electricity running it 24/7 was a couple of times smaller than a comparable hosting.

We wanted a few very specific things that may or may not be relevant to everyone:

• Minecraft Java Edition.
• Decent performance with 2-5 players online.
• Ability to run any mods we like.
• Frequent backups (they saved our world a few times in the past).
• Our friends should be able to connect without too much hassle.

This post will serve me as a reminder how on Earth our setup works.

🤚 Pause for a second

Self-hosting is great and may give you more control and flexibility in exchange for additional responsibility and risks:

• Your home network is less reliable than in a datacenter, which may be a bad time for your friends.
• Your hardware is likely less reliable as well.
• You need to manage the whole stack, from the OS to the game server itself, backups gateway.
• Letting people from the internet into your home network is generally a risky business, if you make a mistake and open up too much, bad actors will invite themselves into your home.

💽 Choosing OS

There wasn’t much deliberation here: Ubuntu Server 22.04 LTS. Mainly because I know how to cook it and it can be run pretty much hands-free. It also has been historically very good with hardware support, so that’s a plus too.

🎮 Running a Minecraft Server

The hosting we’ve previously used offered AMP, which was neither great nor terrible. It is a commercial software though, and I didn’t need most of its features. What I needed was:

• Easy installation.
• Open source, preferably written in a language I can understand and debug if the need be.
• Uses as little resources as possible. Those are better used for running the game :)
• Has a simple Web UI, which allows to create and monitor a Minecraft server.

Turns out there are plenty of options to choose from, whether you use Windows or Linux, or prefer CLI or Web UI. I went with PufferPanel, which ticked all my boxes and seemed to have minimal bloat. Installing it was as straightforward as it gets, starting a Minecraft instance also took no effort.

💾 Backups

That feature was conspicuously missing from PufferPanel feature list, as well as most of its competitors. I think AMP is the only one that officially had it, but on a rather primitive level: no retention policies, no incremental backups. And those features were rather important if you want to have more than 5-10 most recent snapshots.

We have also tried using some Minecraft mods like Textile Backup, but it was also missing retention policies and incremental backups, and it impacted game performance considerably while it was running. So I prepared to set something up on my own.

If I mentioned earlier that there are many Minecraft server management solutions, that’s peanuts compared to backups. There are a lot of options with various pros, cons, levels of maintenance and features. Restic, rdiff-backup, Duplicati and Kopia to name a few.

In the end it was a close call between Restic and Kopia, and I chose the latter mainly because I already had my eye on it for my DIY NAS project (which I one day will write about). I liked a few things about Kopia:

• It supports incremental backups and deduplication, which worked quite efficiently for Minecraft files.
• Support for a variety of remote storage types, including an append-only mode, which could be a good defence against a ransomware attack.
• General ease of setup and management.
• Support for Linux, Windows and MacOS. I could learn it once and then use across all my computers.

For now, I set it up with a local filesystem as a backup destination (though I should setup a remote backup too):

1
2
3
4
5
6
7
8

# Backups will be run by the same user that runs the game server to avoid permission issues.
$ sudo -s -u pufferpanel
# Create backup repository
$ kopia repository create filesystem --path /home/backups/
# Authenticate to the repository and set up the client.
$ kopia repository connect filesystem --path /home/backups/
# Take first snapshot.
$ kopia snapshot create /var/lib/pufferpanel/servers/1234dead/

One interesting feature of this system is that once you’ve set up connection to your repository, you can backup and restore any path on your system. You don’t have to tell how to store each particular directory, it will automatically remember where it came from and will allow restoration to the original location (or somewhere else, if you want it).

All that remains is to build a bridge between PufferPanel and the backup system.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

#!/usr/bin/env python3
from requests import Response
from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session
import os
import subprocess

# This is where your PufferPanel runs.
HOST = '127.0.0.1:8080'

# Create these in PufferPanel → Account → OAuth2 Clients
CLIENT_ID = 'very-long-and-random-thing'
CLIENT_SECRET = 'another-random-thing'


class Client:
    def __init__(self, host: str, client_id: str, client_secret: str) -> None:
        self.prefix = f'http://{host}'
        self.client = OAuth2Session(
            client=BackendApplicationClient(client_id=client_id))
        self.client.fetch_token(token_url=self.prefix + '/oauth2/token', client_id=client_id,
                                client_secret=client_secret, include_client_id=True)

    def _get(self, url, *args, **kwargs) -> Response:
        return self.client.get(self.prefix + url, *args, **kwargs)

    def _post(self, url, *args, **kwargs) -> Response:
        return self.client.post(self.prefix + url, *args, **kwargs)

    def servers(self):
        return self._get('/api/servers').json()['servers']

    def command(self, server_id, command):
        self._post(f'/daemon/server/{server_id}/console', command)

    def broadcast(self, server_id, message):
        self.command(server_id, f'/tell @a {message}')

    def backup(self, server_id):
        try:
            self.broadcast(server_id, 'Starting backup...')
            self.command(server_id, '/save-off')
            self.command(server_id, '/save-all')
            subprocess.run(['kopia', 'snapshot', 'create',
                           f'/var/lib/pufferpanel/servers/{server_id}'])
            self.broadcast(server_id, 'Backup finished!')
        except:
            self.broadcast(server_id, 'Backup failed!')
        finally:
            self.command(server_id, '/save-on')


def main() -> int:
    # Allow unencrypted connection, since we are over localhost anyway.
    os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
    client = Client(HOST, CLIENT_ID, CLIENT_SECRET)
    for server in client.servers():
        client.backup(server['id'])

    subprocess.run(['kopia', 'snapshot', 'expire'])
    return 0

if __name__ == "__main__":
    main()

It connects to PufferPanel, enumerates all existing game servers and then triggers kopia as a subprocess to create a snapshot. To make sure the snapshot is consistent, it also uses PufferPanel API to send some commands to the Minecraft server console.

Save that script to /usr/local/bin/, make it executable and add a Crontab entry to make sure it runs regularly:

1
2
3
4

$ sudo -s -u pufferpanel
$ crontab -e
# Add this line to run the backup every hour:
# 0 * * * * mc_backup.py

🌍 Remote access

One last challenge is making the machine hosted at our home accessible to friends outside of our home network. If you have a static IP, setting up port forwarding on your router is by far the easiest option. Unfortunately, this is not something our IPS provides, so I decided to tunnel the connection instead from my VPS.

Update: Apparently, there’s also https://playit.gg/, which supports TCP and UDP forwarding and doesn’t require you having a server with a public IP. I haven’t tried it, but it may be a good option for some people.

The easiest way of doing it is with SSH port forwarding (on your Minecraft machine): ssh -N -R 25565:localhost:25565 minecraft@example.com. Then everybody connecting to example.com:25565 would connect to localhost:25565, which is where the Minecraft server is.

Update #2: Make sure that in server.properties you set prevent-proxy-connections=false. If you don’t do this, everything will be working fine for you, but any one of your friends outside of your home network will not be able to log in with a $username tried to join with an invalid session error. I think this setting is supposed to make evading IP bans harder, but because the tunnel we are about to set up is a kind of a proxy, minecraft gets upset and refuses the connection.

Unfortunately, it wasn’t sufficient for us, because we have the Simple Voice Chat mod, which (like many VoIP applications) also requires a UDP port to work. The solution came in a form of https://github.com/snsinfu/reverse-tunnel. This project allows forwarding both TCP and UDP traffic to a destination behind NAT. It consists of a gateway that runs on a public-facing VPS and an agent that runs on a machine behind NAT, which establish a channel between them. The agent is able to handle reconnection automatically in case network temporarily drops out, which is nice. One last bit is making sure both the gateway and the agent start automatically with the system, which systemd can do for us.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

$ cat /etc/rtun-server.yml # Gateway config.
$ Gateway server binds to this address to communicate with agents.
control_address: 0.0.0.0:9000

# List of authorized agents follows.
agents:
  # Generate your own key: `openssl rand -hex 32`
  - auth_key: hunter2
    ports: [25565/tcp, 24454/udp]

$ cat /etc/systemd/system/rtun-server.service # Systemd config to run the gateway process.
[Unit]
Description=Reverse tunneling server
After=default.target

[Service]
ExecStart=rtun-server -f /etc/rtun-server.yml
Type=simple
Restart=always


[Install]
WantedBy=default.target

# Systemd commands necessary to enable and start the gateway service.
$ sudo systemctl daemon-reload
$ sudo systemctl enable rtun-server.service
Created symlink /etc/systemd/system/default.target.wants/rtun-server.service → /etc/systemd/system/rtun-server.service.
$ sudo systemctl start rtun-server.service
$ sudo systemctl status rtun-server.service
● rtun-server.service - Reverse tunneling server
     Loaded: loaded (/etc/systemd/system/rtun-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-01-06 23:42:20 UTC; 35s ago
   Main PID: 2774361 (rtun-server)
      Tasks: 4 (limit: 1066)
     Memory: 1.3M
     CGroup: /system.slice/rtun-server.service
             └─2774361 /usr/local/bin/rtun-server -f /etc/rtun-server.yml

Jan 06 23:42:20 gateway systemd[1]: Started Reverse tunneling server.
Jan 06 23:42:20 gateway rtun-server[2774361]: ⇨ http server started on [::]:9000

# Open the ports up in the firewall:
$ ufw allow 25565
$ ufw allow 24454
$ systemctl restart ufw

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

$ cat /etc/rtun-agent.yml # Forwarding agent config.
# Specify the gateway server.
gateway_url: ws://example.com:9000

# A key registered in the gateway server configuration file.
auth_key: hunter2

forwards:
  - port: 25565/tcp # Minecraft port.
    destination: 127.0.0.1:25565

  - port: 24454/udp # Simple Voice Chat port.
    destination: 127.0.0.1:24454

$ cat /etc/systemd/system/rtun-agent.service # Systemd config to run the agent process.
[Unit]
Description=Reverse tunneling agent
After=default.target

[Service]
ExecStart=rtun-agent -f /etc/rtun-agent.yml
Type=simple
Restart=always


[Install]
WantedBy=default.target
# Systemd commands necessary to enable and start the agent.
$ sudo systemctl daemon-reload
$ sudo systemctl enable rtun-agent.service
Created symlink /etc/systemd/system/default.target.wants/rtun-agent.service → /etc/systemd/system/rtun-agent.service.
$ sudo systemctl start rtun-agent.service
$ sudo systemctl status rtun-agent.service
● rtun-agent.service - Reverse tunneling agent
     Loaded: loaded (/etc/systemd/system/rtun-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-01-06 23:45:36 UTC; 6s ago
   Main PID: 8991 (rtun-agent)
      Tasks: 8 (limit: 18991)
     Memory: 1.8M
        CPU: 12ms
     CGroup: /system.slice/rtun-agent.service
             └─8991 rtun-agent -f /etc/rtun-agent.yml

Jan 06 23:45:36 strix systemd[1]: Started Reverse tunneling agent.
Jan 06 23:45:36 strix rtun-agent[8991]: 2023/01/06 23:45:36 Listening on remote port: 25565/tcp
Jan 06 23:45:36 strix rtun-agent[8991]: 2023/01/06 23:45:36 Listening on remote port: 24454/udp

It’s important to note that the gateway and the agent must share a secret (make sure to generate a strong one!), so unauthorized actors won’t be able to piggyback on your gateway.

🏁 That’s all!

After all that I repointed my minecraft server domain to the new IP address and the whole migration went pretty much unnoticed by anyone except me and my partner who can now connect to the server via local network for better latency 😎