Comparing Running Umbraco on the Raspberry Pi against other mini PCs

Introduction

As discussed in my last post, over the summer I recently started looking at running Umbraco on a variety of mini PCs eventually putting together a longer talk, however this blog post serves as a more permanent record of the findings. Running your own inhouse server is never going to be a recommended solution for a larger production site, but for a smaller personal site it’s a great way of getting up and running while at the same time allowing you to learn and play about with a full server at low cost. It also means you have a machine you can repurpose for other things internally, such as a home automation server, an internal SatIP TV streaming server, a PiHole adblocker or many more things, all of which would be difficult to do on a regular remote hosting package.

Of course if you’re going to be running this 24/7, keeping that ‘low cost’ part is going to be very dependent upon having a machine running at a low power consumption. This is important to keep energy costs down but also to lessen the wider impact of your own personal carbon footprint on the planet. People will often here look immediately towards the Raspberry Pi, which is a very well known and energy efficient ARM-based mini PC. However it has its limitations. Supply of these machines new can be difficult at times (between 2020 and mid 2023 the RPi4 was nigh on impossible to get hold of new, and unless you were there within the first few hours of its announcement the new RPi5 looks likely to have the same issue until early in 2024), or alternatively expensive if stuck with a second hand market full of sellers who had scalped the limited supplies. Even if you do manage to get one new though, there’s an important carbon cost to be considered. Any new device will always come with a relatively high environmental cost from its initial manufacture and this is something which could actually take several years of full time running from an older reused machine even at a higher power consumption before being able to recoup. However older machines also come with their own differences in performance and it’s also worth remembering that a machine which runs at a much higher power when at load but is faster will achieve goals in a shorter space of time, so may actually spend more time running idle than a lower power device. So to try and balance all of these up, I set about trying out some of the Raspberry Pi models available to me against a variety of other repurposed mini PCs which are commonly available now on the secondhand market. The aim being to balance off the cost, performance and sustainability factors, and place some sort of metrics on these.

In this initial roundup I have 5 machines available. Including breadboard PCs, thin client machines, ex-corporate mini desktops and Android TV boxes, leading to a nice mix of both ARM-based and more traditional x86/x64 processor architectures.

	Specification	Released	Architecture	RRP
Raspberry Pi 3B+	Broadcom BCM2837B0, Cortex-A53 (ARMv8) 64-bit SoC @ 1.4GHz. 1GB LPDDR2 SDRAM	2018	ARM	£35
Raspberry Pi 4B	Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.8GHz. 1, 2,4 or 8GB RAM – 2GB tested	2019	ARM	£45 (new)
Lenovo M93P Tiny	4th Generation Intel Core i3 4130T, 4GB/8GB RAM	2014	x86/x64	£40-50
Mecool M1 Pro	Amlogic S905D 64-bit octa-core ARM Cortex-A53 CPU, up to 2 GHz. 2GB DDR4 RAM. Android TV Box with built in Digital terrestrial/Satellite tuners	2017	ARM	£25-35
Dell Wyse 3040	Intel Cherry Trail Atom x5 Z-8350 (1.44 GHz Quad Core). 2GB RAM. 8/16GB of soldered eMMC SSD.	2017	x86/x64	£20-30

Methodology

Each of these were set up to run Linux, either Ubuntu or Debian-based depending upon what was possible or available on the given hardware, with .net 7.0 installed. The framework was provided either from the OS repo (if Ubuntu-based) or from the main Microsoft repo (if Debian-based). All of these were then ran and accessed directly via the dotnet command (in a true production use you’d normally proxy this via Apache or NGINX instead). To give a consistent test Umbraco base for this, I created a very simple Umbraco 12 website using SQLite as the underlying database storage, and added in the very-well-known uSync package. A standard webapi controller was then added in with methods to perform some of the following tasks where a back office interface didn’t exist:-

Backend-heavy Tests
1) Create 1500 nodes of content using ContentService. Each of these has some randomly generated Lorum Ipsum text in a body and title field. After creating all the content via contentservice in a loop, the complete set is published at the end.
2) Perform a full export of the 1500 nodes using uSync’s full export option in the back office.
3) Reset the database to an ’empty’ state and then reimport the 1500 nodes using uSync’s full import option.

Frontend-output tests
4) Return a basic json list of the titles of all 1500 nodes, as pulled from Umbraco’s regular cached output layer.
5) Return a more complicated json list of filtered titles. This still uses Umbraco’s cached output layer, but attempts to put more complexity into the process with a bunch of ugly arbitrary code, such as searching for text within a field within a node, converting IEnumerables to Lists and using manual iteration rather than clever LINQ queries. Rest assured it would fail any code review, but the aim here is not efficiency but to provide something a bit meatier on the output side of Umbraco.

The backend-heavy tests were slow enough to be timed from the start of the webapi call to a ‘success’ message using a regular stopwatch. However the frontend-output tests were usually so quick this had to be timed in code with .net’s stopwatch class instead. Both of these approaches do have some inaccuracies and fluctuations, however all machines were given the same chance and every test was run multiple times to average out results. In most cases this would be 3 times, but if there were unusual outliers this was usually run again. Throughout all the testing, a mains plugthrough device was also used to provide a measure of the wattage power draw at the wall both when the machines involved were at their more idle state or when under load running the tests.

Results

The more detailed results initially for all machines came out as follows.

Raspberry Pi 3B+
Power Consumption: Idle 2/3w, Load 3/4w
Test #1: 15:15, 15:09, 14:45, Average – 15:03
Test #2: 2:10, 1:36, 1:35, 1:36 – Average – 1:36
Test #3: 5:27, 5:30, 5:29 – Average – 5:29
Test #4: 6ms, 7ms, 6ms – Average – 6ms
Test #5: 955, 1106, 831, 872, 912, 885, 934, 937, 881 – Average – 923ms

Raspberry Pi 4B
Power Consumption: Idle 2/3w, Load 4/5w
Test #1: 9:47, 9:50, 9:43, Average – 9:46
Test #2: 0:54, 0:54, 0:56- Average – 0:55
Test #3: 13:20, 3:26, 3:11, 11:56, 3:28 – Average (with outliers removed)- 3:22
Test #4: 4ms, 3ms, 2ms – Average – 3ms
Test #5: 634, 641, 506, 579, 580, 534, 526, 540, 559 – Average – 567ms

Lenovo ThinkCentre M93P Tiny
Power Consumption: Idle 10/11w, Load 18/21w
Test #1: 2:27, 2:28, 2:36, Average – 2:30
Test #2: 0:15, 0:15, 0:14 – Average – 0:15
Test #3: 0:50, 0:46, 0:45 – Average – 0:47
Test #4: 1ms, 1ms, 1ms – Average – 1ms
Test #5: 141, 141, 97, 95, 97, 96, 104, 112, 133 – Average – 112ms

Mecool M1 Pro
Power Consumption: Idle 3/4w, Load 4/5w
Test #1: 15:02, 15:03, 15:08, Average – 15:04
Test #2: 1:36, 1:36, 1:35 – Average – 1:36
Test #3: 5:47, 5:40, 5:35 – Average – 5:41
Test #4: 6ms, 5ms, 4ms – Average – 5ms
Test #5: 835, 798, 969, 1130, 795, 794, 845, 807, 816 – Average – 865ms

Dell Wyse 3040
Power Consumption: Idle 3/4w, Load 4/5w
Test #1: 12:27, 12:45, 12:42, Average – 12:38
Test #2: 1:01, 1:05, 1:07 – Average – 1:04
Test #3: 3:39, 3:28, 3:18 – Average – 3:28
Test #4: 3ms, 3ms, 4ms – Average – 3ms
Test #5: 463, 405, 406, 483, 462, 427, 458, 462, 442 – Average – 445ms

The Medal Table

As this creates a lot of results to process, I decided it best to find some way to put these into a simpler and more comparative form, although my way of scoring for this is also one which may cause some differences in opinion. With access to power consumption figures, I was also able to add in calculations to give some ideas of cost and the sustainability aspect all using the following assumptions:-

• Scores are all awarded using a simple approach of ‘Green, Good, +1’, ‘Amber, Okay, +0.5’, ‘Red, Bad, -1’. (Alternative title tags are also available on each table cell to indicate these scores, so as to not depend solely on the visual colour)
• Daily power usage is based off an assumption of running at 22 hours idle, 2 hours load. Given the number of active business hours in a typical day, and then what fraction of those you’re likely to be hitting Umbraco heavily this seemed a good time average. This is only to provide balanced testing though – in practical use a faster machine should finish its heavier loads faster before dropping back to idle state.
• Carbon calculations are based off a UK average of 0.26kg/kWh. Carbon intensity will of course vary across the course of every day based upon how the mix of energy within the grid is being generated at a given point in time so this can only ever be a guide. This figure was however a UK average for 2022. Sources: https://ourworldindata.org/energy, https://github.com/owid/energy-data
• Energy pricing is based off 30p/kWh, which was around the standard UK price cap as of October 2023.
• Manufacturing carbon cost for a new device has been calculated at around 50kg. It’s very difficult to get exact figures on mini PCs, however a lot of research has been done around smartphones, with 80kg coming out as a common average. As a mini PC will share many of the same components as a smartphone, 50kg seems a sensible calculation after removing the battery and screen. Sources: Deloitte, 8BillionTrees – https://8billiontrees.com/carbon-offsets-credits/carbon-footprint-of-iphone

	Price	kWh/year / Cost / CO2	Mfr CO2	Test #1-3 (BE)	Test #4-5 (FE)	Total
Pi 3B+	£35	27 / £8 / 7.02kg	0	4th	5th	2
Pi 4B	£45	27.7 / £8.31 / 7.2kg	50kg	2nd	3rd	0
M93P	£40	94.9 / £28.47 / 24.7kg	0	1st	1st	2.5
K1 Pro	£30	27.7 / £8.31 / 7.2kg	0	5th	4th	2
Wyse 3040	£25	35.8 / £10.74 / 9.3kg	0	3rd	2nd	3.5

Conclusions

Based off this, my main recommendation would be the Dell Wyse 3040. It wins out in cost terms on the second hand market, has a mid-range performance between a Raspberry Pi 3 and 4, with only a little more power consumption. Unfortunately the Raspberry Pi 4 lost out quite a bit on this one. Being one of the most expensive, and being hampered by the ‘new’ device carbon cost. However it will be interesting to see how this changes over the next few months. As the Raspberry Pi 5 grows out into the market and the 4th generation model presumably falls out of production and onto the second hand market, I would expect the RPi4 to lose both that new device cost and for the pricing to fall, with the Raspberry Pi 5 then inheriting those same issues instead for all its performance may be better.

I fully intend to keep revisiting this whole again in the future as the newer versions of the .net 8 framework (with Microsoft frequently touting its own performance increases) and Umbraco 13 come out. But also if I get my hands on any other interesting old mini PCs with a sensible balance of power and cost I’ll look to add these into the mix. Of course it would also be nice to try out the RPi5 too, but I’m loathed to acquire a brand new device and add needlessly to a carbon footprint for the sole purpose of a test so that one may require some cleverer thinking.

Addendum: I’ve only really focused on the results from the machines with this post and not how you’d go about setting this up for use. However if this is something that you want to look doing yourself, Sebastiaan Janssen already has an excellent blog post on how he’s done this for his own setup on a Raspberry Pi. Except for the initial OS setup, most of these steps can be applied much the same on any of the devices as they’re all Linux-based. Ubuntu-based distributions will usually have .net 6/7 available in their own apt package repo, whereas Debian-based distrobutions may need you to add the Microsoft package repo as a package source before otherwise working the same.

https://cultiv.nl/blog/how-i-run-this-very-website-on-a-raspberry-pi-in-my-closet-at-home/