Close Call for my Server - Always Keep Backups!

I had a close call this week with my Yunohost server which hosts all my essential services including WriteFreely, and Nextcloud. I had a few days of downtime before ‘resolving’ the problem.

Being the summer break, I attempted to try self-hosting a few more services on my Yunohost server.

First, I tried Own Photos, a self hosted alternative to Google Photos. It looked really neat but once I installed it I saw that it was running on 6 interconnected Docker containers. I like the idea of Docker, but don’t know much about troubleshooting Docker stuff. I quickly deleted Own Photos because it didn’t work immediately after installing. I am very careful to avoid time-sinks. Google Photos works great, but I would rather self-host my photos. I shall keep looking for alternatives.

Then I tried Mastodon. I have been thinking about running my own instance for some time now as I could try inviting some of my family or other Taiwanese expats to join my instance. Since the app was rated 7/8 on the integration scale I thought I’d give it a try. After it installed, I was getting 401 errors. I loathe troubleshooting something right away, so I uninstalled it. While uninstalling Mastodon I got a few weird errors but ignored.

Next, I tried Pleroma. There was also a level 7 integrated Yunohost app so I thought there was no harm in trying it. It installed no problem, but then also had errors right away. I couldn’t login to the admin account, or nor with any other users. I gave up. I don’t really need to have my own Mastodon instance, but I wanted one.

Finally, after installing and uninstalling these apps I started to notice strange dbus errors which prevented me from updating apps, and doing other things on my server. More information can be found here.

It seemed that nobody knew how to solve this mysterious problem so I was just about ready to reinstall my whole Yunohost system today when somebody on the forum suggested I reboot my server. That seemed ridiculous because I already rebooted before posting my problem on the forum but I was ready to try anything at this point.

So, I rebooted the server. I restarted nginx and ssh services. Suddenly, everything was working again although I’m not sure why. I’m still thankful that everything is working again. I’m not scared to reinstall my server, I do have backups, but it certainly is time consuming.

I have a pretty good automatic backup system in place using cron tab, and Borg Backup with borgmatic.

  1. Run sudo yunohost backup create once a week to local hard drive.
  2. Borg Backup of yunohost backup + Nextcloud Data folder once a week to external RAID 1.
  3. After Borg Backup, remove that yunohost backup on local hard drive.
  4. Wait one week, then repeat step 1!

Now that I’ve had this scare I’m going to reevaluate how I can make my backups more complete and better optimized to save me time if, God forbid, I every have to actually use it.

  • Save borgmatic config file - /etc/borgmatic/config.yaml
  • Save my contacts & calendar CalDAV data that is in Nextcloud
  • Save WriteFreely blog posts in separate file
  • Save Pihole blocklist files

So remember everybody, plan for the worst, hope for the best! Don’t trust your computer but check out your backups. Backups are even more critical for self-hosters, than people who outsource their data to Big Data.

Built with Hugo
Theme Stack designed by Jimmy