My new FreeNAS box

bp2008

bp2008

Staff member
IPCT+ Member
Mar 10, 2014
12,841
14,397
USA
Ironically my earlier thread about replacing my RAID 5 storage server with a FreeNAS box was lost when ipcamtalk suffered a disk failure. :rolleyes:

My new box is a Xeon E3 system with 24 GB of ECC RAM, based on this Lenovo server: http://www.amazon.com/dp/B00FE29IWK
Lenovo did a pretty good job with this server, considering it is actually quite low-end. It is one of the quietest desktop machines I've ever run and there were zero problems getting it running smoothly. Memory testing over two nights indicated no errors so I think I'm all set.

I installed 4 Western Digital Red 6 TB drives inside it this morning and configured them as RAID-Z2 in a fresh FreeNAS installation. This type of RAID sacrifices the capacity of two disks, and provides protection from any two disk failures. Total usable space is just 10.3 TB, a bit less than I was expecting for a "12 TB" pool considering my old "9 TB" pool has 8.18 TB usable. The resaon, I suppose, is the file system. My 9 TB pool uses the NTFS filesystem, while my 12 TB pool uses ZFS which evidently has higher overhead.

Read and write speed are both 100+ MB/s with this new setup, limited by the 1 Gbps network interface. This pleases me greatly. Previously I always used RAID 5 with my storage servers, and suffered abysmal write speeds around 20 MB/s. I've tried the included disk benchmarking tool in FreeNAS, but it is a command line tool and I wasn't able to make much sense of its output... or its input for that matter. What matters though is that this maxes out the network interface, and any additional disk speed on top of that is just gravy.

So far, the CPU usage graph doesn't show anything reaching 20% so I know I chose a fast enough system. It is even using lz4 compression on everything that goes onto the disks. That is pretty cool. I'm sure this machine will handle more CPU-intensive tasks like media transcoding with ease.
 
As an Amazon Associate IPCamTalk earns from qualifying purchases.
Another thing worth noting. As I was copying data off my Blue Iris system today (it hosted my old RAID 5 that degraded), I noticed that my transfer rates were being cut in half as long as Blue Iris was running. I don't know whether to blame the ~50% constant CPU usage of Blue Iris or the ~80 Mbps of transfer from 20 cameras, or both. All I can say is that once I shut down Blue Iris, my transfer rates shot up to 100+ MB/s and CPU usage fell to insignificant levels. I use an Intel gigabit network adapter in this system so I can't blame a cruddy NIC. Perhaps my unmanaged switches weren't able to sustain full speed when dealing with that many camera streams?
 
Thanks for sharing. Great to hear you are back up and running.

Any notes on power consumption to keep that humming 24/7 with 4 hard drives? Peak power? Standby power? (drives asleep)

I've grown tired of slow throughput on consumer nasses as well, and would love to hit 100MB/S; at the same time my infrequent shared access needs would make leaving a server running 24/7 a bit expensive electricity wise. I've considered going with another nuc dedicated for network storage though that opens up a few os network share troubles.
 
I would like 10 tb of solid state. One can dream I guess. Didn't realize they were up to 6tb. Here I was thinking I was high on the chain with a 4 tb external
 
10TB of SSD?

We're getting there, slowly. You have to wait just a little bit longer. Or maybe a fair while. Seems drive makers are stuck around 1TB for the size of the drives. Heard about 4TB, but haven't seen it yet. Plus at what cost. But the 100-1000GB SSD have been coming down nicely in price.

The thing I'm having a harder time with is the cost of NAS boxes; little hardware, BYOD, old cpus, non upgradeable RAM, ... I keep looking at newer Synos & Qnaps and what little you get for the money. And, the real-life read-write speeds with consumer drives in a mixed environment. Aka not a lab with direct ethernet connection and raid 0.
 
I haven't measured power consumption yet, but I bet it is 30-40 watts. As I said CPU usage never gets very high so the hard drives are probably the biggest variable here. A FreeNAS appliance box from Amazon would have been less power. They run a little rarely-seen 8 core Atom CPU with 20W TDP that gets surprisingly good benchmark scores. But their box is also $1000 for the diskless model.

I put in 24GB of RAM because ZFS is extremely RAM hungry. In fact the FreeNAS devs say less than 8GB can lead to instability so they recommend 8 GB as the minimum and they actually recommend 16GB for typical home use. In my case, the box came with 4GB. I got another 4GB stick to match for dual channel performance and then two more 8GB sticks to fill the remaining two slots. Also, apparently memory errors can cause all your data to be corrupted and/or lost, which is why I went with server-class hardware and ECC ram, and then tested it all overnight with memtest86+.
 
Thanks. Good to know. Never used ZFS myself, but yes, I can imagine that any error or corruption causes catastrophic data issues. Almost did a FreeNAS at some point but had a couple unanswered questions and decided to get another syno. Now I'm kind of back on the fence again. Tired of slow NAS performance. But there's not rush. Plenty of work, to dos and honeydos to entertain myself ;)
 
I just saw the $1k freenas mini by ixsystems. Diskless. Ouch!

Last time I looked at freenas must have been when it was still in the rut after the lead developer leaving and before things got up to speed again with ixsystems.
 
excellent work bp2008, I am glad you are thrilled with the results.. thats a pretty impressive DIY NAS!

since you put all this money and effort into keeping the data integrity; just have to ask.. its got a UPS right?
 
Yes, of course.

Encountered one unrecoverable read error so far moving data off the old RAID 5. Just a DVD image from a TV show so no big deal. Been moving the data in chunks and am nearly done.
 
You must log in or register to reply here.
Top Bottom