~/blog/sre_101/

SRE 101

Edward Thomas
· 2 min read
SREcloud

So, the power went out on my servers at the house whilst I was away for a few weeks. The servers only needed to be switched on again, but this clearly showed that I need some backup or failover mechanism.

Goals

  • Implement an failover using AWS (Need that keyword on my CV)
  • Keep costs 0/minimal
  • Have it be automatic

Why Not Just Use a Bucket For All Traffic?

Mainly, it’s because I like the idea that my hardware is doing something, I like looking at the physical server and know it’s something I’ve built and configured serving my website.

Also, it lets me run a web server, which is sometimes more useful e.g. being able to serve .md more easily. While I could do this with the cloud, using VMs or containers makes it harder to have this cost nothing.

Architecture

The core piece of tech is AWS CloudFront, this acts as my cloud router in a sense.

I configured an Origin Group to point to:

  • Primary Origin: My home server - home.ed-thomas.dev
  • Secondary Origin: A S3 bucket with the static files of my website.

CloudFront first tries my home server, and if it times out or returns a 50x error then traffic is routed to get the site from S3 instead.

Challenges

1 - SNI (Server Name Indication) issues

So my homelab is configured to serve ed-thomas.dev with a SSL certificate specfically for that domain. The issue is, in order to have CloudFront sit between my server and a browser, it needs the root domain for my site. This is fine and easy to change in DNS records.

However, in order to find my home server, it needs a new subdomain like home.ed-thomas.dev.

This creates an issue that would be so easy to resolve if I actually setup tailscale. Since cert manager is configured to only create a certificate for the root domain, when a request for the website uses the home. subdomain, it causes an SSL mismatch, throwing an error.

Solution: Host Header Trickery So since I can edit