No matter what server you put into place… do a long memtest on it.
I don’t really worry so much about performance per watt here (our power prices are very low), and as such my clients have a mix of SuperMicro, Lenovo, HP and Dell. My vendor preference is by far Supermicro, then HP, Lenovo, Dell. for me, eBay is the best seller, though sometimes Amazon and Walmart resell server gear - in particular 6/12G SAS shelves are available in some of the strangest places!
Test/Verify your gear when you buy it, and then just before you deploy it. Also, make as accurate a simulation of their environment as you can.
To be honest, I’m still buying E5 v3 and E5 v4 systems for my clients. Outfits like construction companies, architechs, dentists, etc. These guys don’t care about the razoo stuff on the latest greatest, they just want something that will perform well. And the latest proc - for the intended client will not provide a file or query results any faster than a E5 v1. The #1 thing I strive for is balance. And I’m not an SSD nut, I like SSD, and if the budget allows it great. but in enterprise gear, they run SAS, and SAS HDDs are really really cheap!
Latest installation (approximate prices in USD)
Dell T340 Xeon E-2124 3.3Ghz, 32G ram ($600)
8 HGST UltraStar He10 10Tb 7200rpm SAS. 10 purchased to have 2 spare on hand ($1000)
Supermicro NVMe PCIE card ($40)
Intel Optane 56G NVMe drive ($40)
Dual Intel 520 180G Sata SLC SSD. ($80 for both)
this machine has TrueNAS on the Intel 520 for it’s OS mirror, then the data drive is a RaidZ2 with the nvme as the log drive. compression is on, atime is off, relatime is off.
Machine had a bonnie++ test of 96G files and is able to read and write at 2G+ rewrite is 800mb+. IOPs hovers around 1200 up to 2k depending on the operation. each drive is low-level formatted to 4k to verify it’s integrity, the hardware is fully disassembled, cleaned, new heat sink paste on ALL the heatsinks - North/south bridges and the CPU, along with any other misc heat sinks.
Anyway, this machine isn’t going to win the performance races, but it does AMAZING for them on a 10G switch with their 2.5G enabled iMacs (4 of those) and wifi iMac laptops things. They have a pile of those.
this is a tattoo shop, they video all their stuff. have over 15TiB of data stored from the last few years, were using those portable USB drives prior, this is a HUGE postive change for them.
it didn’t break the bank, I’m not chasing the most amazing hardware, but they aren’t getting garbage either. The system performs ZFS snapshots and sends said snapshots to another site. so they have local rollback capacity and remote restore capacity.
This setup came in well under $3k. You could easily do similar just usuing ns8 instead of TrueNAS.
There’s a couple ways to look at eBay hardware.
- it’s used therefore it’s either junk or untrustworthy.
- it’s pre-tested and already burned in.
I like the second option because electronics, if they don’t fail in the first couple of months will last until the capacitors dry out - some 12 years later on most computing gear.