Daniel Aguirre

After the increases in Europe’s electricity prices, I have decided to run some basic numbers to understand how much my homelab installation costs me, as well as to have some reference numbers to have an informed opinion when purchasing new devices. For this analysis, I will start by calculating the cost per KWh as well as the yearly cost per W. I will proceed to estimate the rough power consumption of different devices to finally calculate the total cost of running them....

As the amount of personal webscraping projects increased, I realized the importance of having an structured project template. The reason being that it allows to systematically create new projects without having to worry about refactoring later on if it doesn’t match with my webscraping service. Because of this, I have decided to explain in this post how I structure most of my webscraping projects: 1. Considerations I like to keep each project as independent from the rest as possible....

As a data engineer is quite common to be on the lookup for useful (or at least interesting) websites to scrape data from. However, it is quite common to have different projects to start the same (e.g. downloading the HTML from the internet) but to end up processing them (ETL) in different ways. Because of this, I have setup a simple Raspberry Pi as my centralized webscraping service where: Websites, files, … can be downloaded into a raw format....

With the homelab hardware set up, it is time to start preparing the machines so they can work together. As I intend to use my homelab as a safe and easy to use “IT playground”, I have decided that the following elements would need to be setup: Static IP address on each machine. This facilitates troubleshooting as well as easing the communication between the machines. Remote control software and SSH. This facilitates troubleshooting my “remote” machines without having to hook a keyboard+mouse+monitor to them....

As a data engineer, it is quite obvious that I like tinkering with technology, specially with software related stuff. Because of this, some time ago I decided to setup my own simple homelab where I could experiment and run scripts around without worrying about setting up cloud services. As such, I have decided to share how I set up my homelab, so others can use it as an inspiration or just have a look at it....