FreeNAS, Plex, and Plex Automation – Part 2 – The Hardware

January 6, 2016 / Greg / 4 Comments

WARNING: The hardware specs you are about to read are NOT needed and are complete overkill for a normal FreeNAS build. It is simply me living by the adage of “anything worth doing is worth overdoing.”  You can find the FreeNAS hardware recommendations in this thread on the FreeNAS forums. I suggest you spend some time doing your own research into what will be best for you and your situation. I’ve also gotten a lot of heat from folks on the forum for some of my choices. I’ll admit that some choices aren’t ideal, but I’m also trying to reuse the hardware I already own as much as possible to lower the cost.

This will be the 3rd (actually the 4th) hardware iteration of my FreeNAS server and it’s taken me quite some time to decide on what I wanted to be in this build. When I first decided to build a NAS for my home, I wanted to use some of the hardware still laying around from my bitcoin/litecoin/altcoin mining days. I had sold off many of the GPUs, but still had a few different CPU/Motherboard combos that were collecting dust. This video gives you a very small idea of what things were like back then. After taking a quick inventory of what was available, I decided to go with this:

  • Intel G3220 CPU
  • ASUS Z87-A Motherboard
  • 8GB DDR3-1600
  • Thermaltake Commander G42 mid-tower case
  • 6x WD Red 3TB HDD in RAIDZ2 (~12TB useable storage space)
  • EVGA SuperNOVA 1000G2 80+ Gold PSU

The only thing I needed to buy were four of the hard drives since I already had everything else. Two of the WD Reds were sitting in my ESXi host, unused. After I put everything together and got it running, I realized I needed more RAM due to ZFS’s use of RAM for ARC. 32 gigs went in. I then realized that the G3220 wasn’t powerful enough to handle multiple Plex streams, so I wanted to upgrade it. When I was swapping it for a Core i7-4790K, I bent some pins on the motherboard, so while waiting for a new motherboard to arrive, I put in an AMD FX-4130 CPU and Gigabyte GA-990FXA-UD3 mobo in order to keep things running. That was technically iteration #2, but it was only that way for about 10 days. At the same time I ordered the new motherboard, I also ordered a rackmount chassis for it.

Iteration #3 is what is running currently. Here’s those specs:

  • Intel Core i7-4790K CPU
  • ASRock Z97 Extreme6 Motherboard (bought because it had 10 SATA ports)
  • 32GB DDR3-1600
  • Rosewill RSV-L4412 Rackmount Chassis
  • 2x A-Data Premier Pro SP900 64GB 2.5″ SSD
  • 6x WD Red 3TB HDD in RAIDZ2 (~12TB useable storage space)
  • EVGA SuperNOVA 1000G2 80+ Gold PSU

I moved to SSDs for the boot device because I was having issues with the USB drives constantly getting errors. I had two 64GB SSDs that were purchased for a previous project and ended up not being used, so I threw those in there and I haven’t had any errors on my boot devices since. You DO NOT need SSDs for your boot devices. A couple high quality USB drives will be fine. Even though I have those drives in the computer and mirrored, I can’t use that space for anything other than the FreeNAS operating system, so it’s wasted. As you can see, I’m currently only using 1GB of space.

Screen Shot 2016-01-06 at 11.23.11 AM

I have multiple reasons for creating iteration #4 and for picking the parts I ultimately chose.

  • I want to be able to consolidate my ESXi host and my FreeNAS server into one unit
    • The ESXi host is also running an i7-4790K maxed out with 32GB of RAM.
    • Haswell can’t handle more than 32GB of RAM and I need more than that to run the VMs currently installed.
    • FreeNAS can act as a VirtualBox host. I don’t know how well it works, but we’ll soon find out.
  • I want to be able to handle anything I can throw at Plex.
  • I want to be able to use this server for a long time.
    • I should build something that has enough horsepower that I don’t need to build another one in 3 months. That stuff is starting to get old pretty quick.
    • Making it last means making sure I can add more CPU and RAM in the future. The i7-4790K’s are the most powerful processors I can use with the Z87/Z97 chipset and 32GB is the most Haswell can handle. I can’t upgrade further without changing out motherboards and RAM.
  • I don’t want to have to worry about running out of storage space anytime soon.

With all these things in mind, I spent some time looking into what would not only fit my need, but also be able to use as much of the gear I already have as possible. I knew I was going to have to buy something that can handle ECC RAM and I wanted dual CPUs for the 4K transcoding. So, without further ado, here’s the hardware that will be going into Greg’s FreeNAS v 4.0:

  • Dual Intel Xeon E5-2660 CPUs (used hardware)
  • Dual Supermicro 4U Active CPU Heatsink Cooling for X9 UP/DP Systems SNK-P0050AP4
  • SuperMicro X9DR3-F Motherboard (open box)
  • 128GB (2x 64GB kits of 4x16GB) Kingston KVR16R11D4K4/64 DDR3-1600 Registered ECC RAM
  • 2x A-Data Premier Pro SP900 64GB 2.5″ SSD*
  • 6x WD Red 3TB HDD*
  • 6x WD Red 6TB HDD**
  • LSI SAS9921-8i HBA**
  • 24 port expansion card for the 9921-8i (don’t eBay while drinking, kids)**
  • Rosewill RSV-L4412 4u chassis*
  • EVGA SuperNOVA 1000G2 PSU

*Reused Hardware
** Purchased before I decided on the CPU/Mobo upgrade

Yes, I’m going with dual Xeons and 128GB of RAM. Complete overkill and I love it. The only “new” hardware is the CPU, motherboard, RAM, and coolers. Everything else was already purchased with the idea to upgrade the old server. The parts have already started arriving and should finish getting here next week, which means I’ll probably build it out on a test bench on the weekend of January 16th. The plan is to build the new box with the six 6TB and a spare PSU on the test bench, do some testing and burn-in, then move everything into the rackmount chassis. I’ll use my current FreeNAS config on the new server, add a new zpool with the 6TB drives and go from there.

The guys on the FreeNAS forums are giving me a hard time about two things, the chassis and the power supply. They really think I should have redundant power supplies in the server, and while I’ll probably look into it, I doubt I’ll do it. First off, I have redundant power going to that PSU in the form of dual APC SMX1500RM2UNC UPS systems and a Tripp-Lite PDUMH15ATNET Auto Transfer Switch. Secondly, a redundant 600w power supply isn’t cheap. Even if you have dual PSUs, you only have 1 motherboard and 1 set of wires from the PSU chassis, not to mention the backplane of that chassis. You still have multiple single points of failure. As far as the chassis is concerned, they think it’s garbage. Here’s a couple quotes from the forums:

“I do have to say that building out that much of a server and not going with a better case and redundant power seems like dropping the ball.”

“Considering the amount of money you’re sinking into this, why not just return or resell the Rosewill clunker and find a nice Supermicro 846 or 847 chassis on eBay? It would be a shame to build a Ferrari powertrain and drop it in a Pinto chassis.”

Well, they’re not paying for all this hardware, and adding redundant power and one of those Supermicro chassis would add another $1,000 to the cost. If I need to add more hard drives in the future, there are ways of doing that. I could use an HP SAS expander and put another chassis with nothing but hard drives in it, or I can get one of those Supermicro cases at that point and transfer all this hardware into it. I just don’t foresee needing more than 12 bays. Also, I’m starting to think that Supermicro must secretly pay the people on the FreeNAS forums. Those guys absolutely LOVE Supermicro hardware. It’s the only thing they ever want to talk about. The reason I picked the Supermicro X9 motherboard was because I realized it wouldn’t be hard to get support for it from the forums. That’s something you might want to keep in mind too. If something doesn’t work in your build, you’ll wish that you had picked hardware common on the forums, otherwise you’ll spend a ton of time trying to figure out the problem.

Well, that’s where we stand as of today. I’m thinking about documenting the build on YouTube as well as here, so keep your eyes peeled for links, should I decide to do that. In the meantime, head over to the FreeNAS forums and start reading so you can be informed enough to pick out your own hardware. The decisions you make on hardware will be the most important decisions you make with the whole thing. It can be the difference between a relatively quick and painless setup or an absolute nightmare. Whatever you decide, make damn sure it’ll support ECC RAM!!!