Card dynamic cap

HomeLab

Last Updated: Sept. 12, 2024

HomeLab



During the summer of 2023 I watched a few videos about the concept of a homelab after looking for easy ways to run an at-home analytics stack for the purpose of getting more familiar with a few pieces of software and furthering a few projects. Well, today I have fully jumped into the concept of running docker containers at home in a “Homelab” environment and I wanted to share some of the ones I am using or have used before and liked.


Self-hosted software is by no means easier than using a web-based tool or installing something new directly on your computer. It has been extremely fun (and frustrating at times) to tinker and have an environment I can work in that is all running on my home network.


My initial use-case was having a full analytics environment at home that could mimic a work environment in a way that was easy to manage and maintain. Docker is far from just a tool for this use-case but I can highly recommend at least looking into Docker if you’re not familiar with it. Below are a few of the videos that made it “click” for me.


If you have anything you would recommend, please reach out!



Analytics & Data:

  • Postgres/PGAdmin - link - I previously had a postgres database running but running it in a container that is accessible to any computer on my home network has allowed me to use it far more often.
  • Airflow - link - Surprisingly easy to get setup. It has been amazing having an airflow instance at home to schedule tasks I was running manually before.
  • Superset - link - My favorite self-hosted viz tool I have used so far - but slightly more involved setup vs others here. It may be familiarity from previously using Superset at work but it is my favorite free viz software and the SQL Lab makes it that much better.
  • JupyterLab - link - I haven’t fully given up on Collab notebooks but I rarely use Anaconda any more now that I have my own Jupyer server running locally.
  • minio - link - S3-like object storage but open source and free. It has come in handy more than I expected but it was a great way to get more familiar with packages like boto3.

Others:

  • Planka - link - this was a more recent addition but it is a Trello clone. I really like managing projects and tasks in Trello so this was a fun addition to have everything stored locally and self-hosted.
  • Docmost - link - I really like putting notes for projects in Google Docs. Keeping things organized and having a clear visual representation of pages and sub-pages in Docmost has been awesome.
  • Gitea - link - Won’t fully replace github for me. For projects I am working on at home and all these new docker-compose files, it’s nice to have a github-like UI to manage new projects
  • ntfy - link - a messaging service that runs locally, has a mobile app, and was very easy to setup. Works super easy by initiating messages with a POST request. With a single line added to a few DAGs I get a notification on my phone when data freshes (or doesn’t) and how many records were added.
  • Wireguard/WireguardUI - link - With all this amazing stuff running at home, I wanted to be able to access it while away. Self-hosted VPN fixed that and the wireguard UI made it easy to manage/add devices.
  • Adguard - link - Ad blocking for all devices connected to my home network … amazing. I know very little about networking and have very little interest in it but I did not know what I was missing until I tried this software out.
  • Nginx Proxy Manager - link - 192.168.86.150:5432 vs pg.website.com- adding a domain and SSL to everything was not a necessity but it sure looks a lot better.
  • syncthing - link - keeps files in sync and backed up between different machines. Easy initial setup it is still working well.

Tried but no longer actively use:

Good stuff but maybe didn’t have much of a use-case or found an alternative above that I liked more.

  • Ollama + Ollama Web UI - link - Needed to run on my desktop but I underestimated how easy it would be to have an LLM with a web UI to manage/interact with models. Do not have much of a use-case but it was fun to try out.
  • pingvin share - link - really easy to use file share. Can drag a drop files and get a link to access/share them, set file expiration, deletion, and access controls. I used this initially to store a csv file with zipcodes for a redfin scrape.
  • prefect - link - looked at airflow alternatives but I liked the workflow of creating DAG’s more than prefect jobs. I can see how an enterprise environment would really like using this software.
  • metabase - link - Really solid viz tool. After a quick trial run, I stuck with Superset but I may give metabase another try.