Self-hosting - vindicated (again)

Today, there was an AWS outage at US-East-1. I'm not sure what the cause was - some say DNS, others say an issue at the DC.

Lots of folks reporting that their Ring cameras and video doorbells don't work, banking apps offline, Reddit having trouble. Chaos.

I was unaffected (mostly)

My cameras? Still recording and responding.

Photos? Yep, still syncing and accessible.

Calendar? Yeah, I can still create events.

Reddit and Github? OK, yeah, I had trouble with those, but I can't self-host either of them (OK, I'm sure self-host alternatives exist, but that doesn't bring in the content of those platforms).

Todays events once again vindicated my choice to self-host as much as possible over the past years.

What am I self-hosting?

I touched on this briefly in the previous blog post, but todays outage felt like a great opportunity to dive deeper. Let's start with core infra.

Firstly, Unraid. This has the core of much of my self-host infrastructure, though it doesn't host everything - more on that later.

Secondly, and perhaps most importanly, opnsense. OK, I know you literally can't 'remote host' your router or firewall, but understanding that I'm using opnsense rather than some ISP-provided option might help explain other parts later. I was originally running pfSense (pre-installed on the system I bought), but in recent months decided to switch to opnsense.

Thirdly, Adguard-Home. I originally setup Pi-Hole on an R-Pi, but I eventually decided to move it out once I'd setup Unraid. As I quickly learned, hosting your DNS server on a NAS is a bad idea, especially if stopping the array also takes that DNS server offline. I then setup a secondary/fallback DNS server on a separate box, but it always annoyed me because syncing them and same versioning them was doubling the work (and meant running 2 systems). Not that I had to do it often, but it was just something extra to keep check on. Finally, I decided to install Adguard-Home on opnsense - because if my router is offline, then I've got bigger problems to worry about.

I'm running a ProxMox box (and some other systems) too, I'll dive into this more later.

So, that covers the core infra. I know technically only OpnSense / router/firewall and a DNS server are core infra for a network. Now, let's dive into what I'm self-hosting.

I'm running both Netbird and Tailscale. The reasons for this are redundancy and counter-shittification. Netbird is my primary choice because it's self-hosted, but Tailscale exists as a backup. Netbird relay does run externally, primarily because I started having trouble connecting to it when it was running internally. I run Netbird clients on my systems (including phones), and have a dedicated 'masquerade' node on ProxMox (LXC) to enable access to other internal systems and also to act as an exit node (ensuring I can still access IP-restricted resources). If for some reason Netbird becomes unavailable, I've got Tailscale to fallback on, and have a dedicated Ubuntu VM running on ProxMox with Tailscale configured.

What's on Unraid?

I run the majority of things from my Unraid system. The below are all Docker containers unless specified. I'm not going to list absolutely everything, but the main things are written below.

Frigate. Ideally, I should move that out to another system, have a cache drive for footage which then rsyncs across to Unraid (for redundancy reasons), but the Unraid system is the only one with an Intel CPU in, so video transcoding is super-cheap from a power perspective when compared with the AMD systems. The AWS outage means my cameras still work and record. The fact they're firewalled means that I've already had them in a 'no outside connectivity' scenario.

MQTT. This is generally only used for Home Assistant, but since I've move HA off of Unraid (and into a HA OS in a VM on ProxMox), this could probably be deprecated in favour of running MQTT within HA itself. I also run 'MQTTExplorer' which lets me view MQTT data easily - the only reason I run it here is because I got tired of running the client version on both my PC and Laptop (which also mean I can even access it on my phone).

VaultWarden. I've been using a password manager since about 2018, starting with 1Password. In recent years, we've seen breaches at some of these cloud-based password managers, and even though I trust the encryption they employ, I would have no way of knowing if a breach had occurred until they announce it. Sometime in the past year, I decided that the risk was too great, and looked for alternatives I could self-host. Initially, I considered using something like Hashicorp Vault, but reading deeper I found that it wasn't really a suitable option for password managers (especially if you still value the convenience of cross-platform apps to actually access it). Then, I found VaultWarden, and set about securing it in every feasible way imaginable. It already does most of the heavy lifting, but limiting access to it is still incredibly important.

Searxng. This is a self-hostable search engine which uses multiple other commercial search engines, but avoiding cookies by acting as something of a proxy. Admittedly, I don't use this as much as I perhaps should - I'm still very much stuck in a 'Google first' mindset for search. What I did find is that SearXNG results aren't always as contextual as I'd like. For example, if I search for "Expo network" on Google, it'll usually bring up the documents page for the 'expo-network' package for react native apps. If I search for that same term in SearXNG, it'll bring up things related to "Expo's", as in 'exposition' (large scale public exibitions) about networking (as in 'socialising'). To resolve this, I have to explicitly search 'react native Expo network' or something similar. It's not a lot of extra typing, but it's also not as convenient as Google.

Immich. This is probably my favourite self-host project, and one of the most used for sure. Originally, I was a Google photos person because I'd been using Android since ~2011. I did have an iPhone 4 before that, but I generally didn't bother with iCloud sync. I did have an iPhone XS in 2018, but decided that the Huawei P30 Pro was a better option all-round. In December 2021, I was looking around for a new phone. I'd been using the Samsung Z Fold 2 at that time, and I was starting to get annoyed by it. The battery life wasn't ever as good as I'd liked, and I found myself hating having to unfold it (because the 'tiny' front screen was a pain to type on). So, looking around, I decided to switch over to iPhone once more, with the iPhone 13 Pro Max. Surprisingly, the battery was great, the cameras were phenomenal, and the overall experience was nice to use. So, I found myself also paying for iCloud too.

In early 2024, I decided enough was enough, and decided to setup Open Media Vault and install Photoprism on a mini PC. It seemed OK at first, but OMV is horrible to navigate, and I found that Photoprism didn't actually protect or proxy assets - if you knew the URL of an asset, you could access it directly without authentication. It's not that I have photo that are risky, but that doesn't mean I want people randomly stumbling upon any photos by checking random URL's. I also found that it was really unreliable and slow with transcoding, though this was likely due to it running on an Intel N95 system. After 3 months, I decided this wasn't long-term supportable, and sought alternatives. Immich was heavily recommended, and felt much nicer to use - very close to Google Photos. I decided it was time to setup a NAS, which is where the Unraid installation was born from, and where Immich ultimately ended up being installed. I setup encrypted remote backups, verified a restoration procedure, and then cleared out Google Photos and iCloud, then cancelled the subscriptions.

Baikal. I wasn't ever a calendar person. I respected their uses, but to me, they were a corporate tool I had no use for in the home. That was until I started socialising again in late 2024. I found that keeping track of events was getting problematic, and if I ever wanted to plan something which didn't have a MeetUp or Facebook event page, I had no way of reliably reminding myself about it or checking for time conflicts. So, I set about searching, and found Baikal. I had some trouble getting different calendar apps linked up with it, but it generally worked. After a few months, I discovered that there's a WebDAV authentication setting in the admin panel, which by default was set to 'Digest'. Changing this to 'Basic' resolved the auth issues I'd started having after reinstalling client calendar apps.

Timetagger. As a software engineer working in a commercial setting for various clients, I have to track my time for most of them. I was using Toggl for the longest time, and it worked, but this was another tool I decided to self-host. I did this primarily for privacy reasons more than anything, but also because I wanted reliability in the event of outages. It doesn't have client apps, so using the PWA or website is the only solution, but that works just fine across Windows, Mac, and iOS (and likely Android too).

Neos-Memos. I don't use notes apps very often, but I had a bunch from the past that I didn't want to lose. Again, installing a self-host option was more for privacy than anything. I migrated my notes from Google Keep to Neos-Memos (by manually copy-pasting each note across), and then deleted them from Google Keep. One less thing for Google to snoop in on.

Kavita. I have a bunch of ebooks but I don't always want to use the Kindle app. Self-hosting makes sense. I don't read books or ebooks that often these days, but it's handy to have something around because I never really know what my brain will feel like doing.

Dawarich (a recent addition to self-hosting). I wasn't ever big on Google Timeline or companies knowing where I'd been at every moment. Recently, I realised there's value to myself in knowing where I've been at a particular time or on a particular day. I recently used it to find out when was the last time I'd seen certain people. Rather than saying "I think it's been 2 weeks", I could say with certainty "It was [specific date] when I last saw you" (yeah, I'm a stickler for being precise and explicit about what I say as it reduces the chance of miscommunication).

Ollama + Open Web UI. I'm not big on how AI is being used to spew out oceans of slop, but I respect that it can be used as a tool. I typically use it for coding, but not for 'vibe coding' or writing entire apps. Rather, I use it for individual pieces within functionality, or for reminding myself the syntax of things. For example, with SQL, I usually forget how to correctly nest into JSON and JSONB data (usually using the wrong -> or ->>). I recently used it to figure out some CSS for this blog - specifically for code blocks, and fixing a weird bug with the padding or offset of the first line inside the blocks. I also used it to help me figure out how to use various Next-related functionality when creating this blog (specifically how to get data from files when using SSG). AI is often wrong, either hallucinating functions which don't exist, or being correct but it's for older versions of libraries and no longer works. In terms of models, I tend to stick with Deepseek R1 (14b) or GPT-OSS (21b), but I've got a range of different models ready for different scenarios.

Paperless NGX + AI. I use multiple computer systems, but I absolutely do not want my documents in the cloud. I don't tend to store many of them, but I still don't want them being sent to random cloud services. So, I run Paperless NGX. The paperless-AI tool does automatic tagging for me to make it easier to organise and search.

Jellyfin. Folks seem to like Plex, but I don't, because they still rely on cloud connection and an account. Jellyfin doesn't care about any of that, instead focusing on what it's meant to - transcoding media and serving a UI to play it. The community seems to be obsessed with writing their own front-ends, which doesn't really interest me, but I'm glad that it's open enough that people can do that.

What's on ProxMox?

ProxMox hosts less than Unraid. Ironically, it's got more a more powerful CPU and almost as much RAM (compared to the Unraid system which is basically a desktop PC), but it's also a mini PC running AMD, so transcoding on it would lead to louder fans. It doesn't run all that much right now, but I set it up as a means to expand in future.

N8N. I'd known about N8N for a few years, but I'd never paid much attention to it. See, for the longest time, I stuck by trying to do everything with code. If I wanted to automate something, I'd run a cron script on a system somewhere in my network, and 'tada, it worked'. This is fine for simple stuff, but complex stuff could take hours to build out and ensure it worked. Recently, decided to take a look at N8N in more detail, and it turns out I had more use cases than I could have imagined. Not loads, but as time goes on, I'm finding more and more. I'll probably dive into what I'm using N8N for in a future blog post.

Remote VSCode server / host. OK, so I'm not talking about 'VSCode Workspaces'. Rather, it's literally an Ubuntu server VM with projects on which I can SSH into. VSCode can work with this, and that means I can work on projects from both my desktop PC and my laptop. And since I have Netbird, I can access them even if I work remotely (very rarely). The downside of this is that for React Native projects, I can't do certain things like build a new binary for iOS, and anything that tries to work with web browsers generally won't work (e.g. I can't launch the React Native Devtools because there's no browser installation or even a desktop environment on this VM). This VM is given a lot of resources to work with to avoid performance issues.

Home Assistant OS. The first thing I ever self-hosted (if you exclude pi-hole). Borne from frustration with a wifi lightbulb sometimes just not working (and never working during outages) and having rate limits (screaming). I was running the Docker version of this for the longest time because it was the easiest for my use-case (always mini PC's until I got to the Unraid system). It eventually ended up running on Unraid so I could retire some of the older N95 and N100 systems I was using. When I setup ProxMox, I decided to migrate HA here - mainly because having it on Unraid subjects it to outages if I stop the array (= no lightbulb control), and decided that running HA-OS was a better option.

Anything else being hosted anywhere?