My home server has a long story. It all started with a Pentium 4, an old 20GB HDD and FreeBSD 6.2 … hell, I don’t even remember the exact year.
Anyway, after a few years, the hardware was finally put to rest, since it died and got resurrected thrice, I got an Intel Atom D410-based miniATX board, switched to linux, first debian, then agilia, then arch… Anyway, it used to be a nice server for personal needs, that crashed only on occasional HAD-effect, so it was… sufficient.
Now, the time has come to move on, to arm. The benefits were simple and straight:
- 10W peak power consumption
- Fully passive cooling
- eMMC for the root partition
- 4 cores!
- Always a serial terminal, starting from uboot phase, so that I don’t have to carry a monitor to the closet where it is stationed.
I picked ODROID-X2 based around Exynos4212 Prime. ODROID-U2 looked worse, since had NAND soldered onboard. eMMC looked easier to replace. And the benchmarks said eMMC was faster.
So, here go my adventures with this hardware.
First things first. Power. People from hardkernel did it the strange way. If you _instantly_ load all 4 cores, all usb/hsic stuff will reset. The very easy test in bash:
#!/bin/bash cat /dev/urandom > /dev/null& cat /dev/urandom > /dev/null& cat /dev/urandom > /dev/null& cat /dev/urandom > /dev/null&
Poking for a few minutes with an oscilloscope, a few minutes of thinking and voila – a workaround: I put a 4700uF cap on the power input. Yeah, yeah, an overkill, but it works. Even if you hotplug some nasty stuff that has a big inrush current. Luckily there was a test pad and ground pad nearby.
With that hack done – all these problems were instantly gone.
Second bug – power jack. The contact was loose, the cable I had was shitty, so I ditched that in favor of a JST connector.
So long for the hardware, now the software. I started by benching the storage subsystem with a simple test:
SATA HDD over usb:
[email protected]:~$ sudo dd if=/dev/sda of=/dev/null
1198177+0 records in
1198176+0 records out
613466112 bytes (613 MB) copied, 31.2201 s, 19.6 MB/s
Oops, forgot to disable usb mass storage verbose debug.
[email protected]:~$ sudo dd if=/dev/sda of=/dev/null 1975674+0 records in 1975673+0 records out 1011544576 bytes (1.0 GB) copied, 46.847 s, 21.6 MB/s
[email protected]:~$ sudo dd if=/dev/mmcblk0 of=/dev/null 258017+0 records in 258016+0 records out 132104192 bytes (132 MB) copied, 12.2988 s, 10.7 MB/s
[email protected]:~$ sudo dd if=/dev/mmcblk1 of=/dev/null 718049+0 records in 718048+0 records out 367640576 bytes (368 MB) copied, 11.4995 s, 32.0 MB/s
That says it all: eMMC – rootfs, SD – webpages, since we have fast random access there, HDD for data, logs and the rest of the junk. Before doing anything, I carefully tested SD and eMMC with flashbench, and formatted according to this tutorial to improve performance.
It looks like on eMMC bl1/bl2/tz/uboot are locked into a secure piece, which is read-only via /dev/mmcblk1boot0, and /dev/mmcblk0 has only uboot env.
As a result, my eMMC partitioning looks like this:
Device Boot Start End Blocks Id System /dev/mmcblk1p1 2423 4095 836+ 83 Linux /dev/mmcblk1p2 4096 32767 14336 b W95 FAT32 /dev/mmcblk1p3 32768 15269887 7618560 83 Linux
Well, uboot from samsung/hardkernel is a pile of junk dating back to 2010. ext2load is broken, ext4load is not there uImage doesn’t boot, only zImage and fatload. My idea to boot fresh kernels via tftp from an openwrt router also didn’t work out: Hardkernel people put ethernet on HSIC via SMSC LAN95x, not built-in exynos MII, (saved a few cents on the phy?). And uboot doesn’t have the code to run LAN95x. Shit.
Well, could be worse. My personal code shit-o-meter showed a total shittiness of something like 0.32 realteks for the samsung code.
Anyway, first partition on eMMC is for uboot env, so that I can easily access it via cmdline, next go vfat boot partition with zImage and rootfs. Simple s that. The rest is straightforward.
Ethernet is only 100mbit, and works out all of the throughput. Since I have a 100mbit network this is sufficient.
Finally, Apache Bench of Intel Atom D410 vs Exynos from outside of my network. Configs of the webserver, php and mysql are totally identical:
ab -n 1000 -c 100 http://ncrmnt.org/ Server Software: awesome Server Hostname: ncrmnt.org Server Port: 80 Document Path: / Document Length: 37804 bytes Concurrency Level: 100 Time taken for tests: 33.119 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 38100000 bytes HTML transferred: 37804000 bytes Requests per second: 30.19 [#/sec] (mean) Time per request: 3311.930 [ms] (mean) Time per request: 33.119 [ms] (mean, across all concurrent requests) Transfer rate: 1123.42 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 31 46 90.6 36 2016 Processing: 202 3110 1081.8 3225 9784 Waiting: 163 2956 885.8 3167 5907 Total: 237 3157 1065.2 3261 9819 Percentage of the requests served within a certain time (ms) 50% 3261 66% 3283 75% 3295 80% 3307 90% 3423 95% 5098 98% 6112 99% 6466 100% 9819 (longest request)
ab -n1000 -c100 http://ncrmnt.org/ Server Software: awesome Server Hostname: ncrmnt.org Server Port: 80 Document Path: / Document Length: 37804 bytes Concurrency Level: 100 Time taken for tests: 34.914 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 38102552 bytes HTML transferred: 37804000 bytes Requests per second: 28.64 [#/sec] (mean) Time per request: 3491.388 [ms] (mean) Time per request: 34.914 [ms] (mean, across all concurrent requests) Transfer rate: 1065.75 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 32 63 192.1 35 3758 Processing: 150 2806 3082.1 1945 30606 Waiting: 78 699 1052.0 292 9206 Total: 184 2869 3098.2 2012 30655 Percentage of the requests served within a certain time (ms) 50% 2012 66% 3161 75% 4218 80% 4394 90% 5738 95% 7662 98% 10636 99% 17378 100% 30655 (longest request)
Although, taking the time to optimize things would definitely improve things.
EDIT: optimized a few bits, tested with siege with -c 150 getting the wordpress frontpage.
Transactions: 2691 hits Availability: 100.00 % Elapsed time: 37.74 secs Data transferred: 97.58 MB Response time: 1.53 secs Transaction rate: 71.30 trans/sec Throughput: 2.59 MB/sec Concurrency: 109.01 Successful transactions: 2691 Failed transactions: 0 Longest transaction: 1.79 Shortest transaction: 0.07
So far it looks like an epic win. Yes, now I’m ready for the HAD-effect!