Hardware verification for 15 to 20 cameras

[Lil'John]

Title states it but for lots of info:
Hardware in hand:
Dual AMD Opteron 2374 HE 2.2GHz - 8 cores total
64Gb ram
Two SSDs in RAID0 for OS & apps
Unknown number of hard drives in RAID6 using LSI RAID card
Five Ubiquiti UVC-G3 -known 'questionable' picture resolution/quality

Software:
VMware ESXi at bare metal
VM for BlueIris, unknown OS
VM for media/file server

Intended use:
Record on movement and off-site viewing of a house... I love snow. Downside is there is wildlife so probably much movement on a couple cams.

Questions:

• What is best OS for Blue Iris? I'd rather not go Win 10.
• How many cameras will AMD CPUs support? I am going to slowly add in cameras as I go.
• How much recording hard drive space should I budget for? Current plan is four 4TB hard drives for about 8TB in RAID6. This is all for cams

Right now, a new computer system isn't on the menu right now. In the future, I'd like to go with an E3 v6 Xeon.

 

[fenderman]

use windows 10..its great.
the number of cameras is irrelevant...what is relevant is the total MP and total fps...the dual amd will be a powerhog -an efficient system will pay for itself...you can buy an efficient intel based system that supports hardware acceleration for 150-500....you dont want to run any vms on a VM..also note that intel hd hardware acceleration may not work on a vm if you go with an intel system...

 

[Lil'John]

use windows 10..its great.
the number of cameras is irrelevant...what is relevant is the total MP and total fps...the dual amd will be a powerhog -an efficient system will pay for itself...you can buy an efficient intel based system that supports hardware acceleration for 150-500....you dont want to run any vms on a VM..also note that intel hd hardware acceleration may not work on a vm if you go with an intel system...

The AMDs are only 55W each. Not typical current 100+W per chip

VMs are run on VM host which I believe ESXi is. If you are saying I shouldn't VM this, please explain. Most of the reading I've done says VMs will pass through hardware acceleration.

If I recall right the cameras are 3MP. For recording, I'd like 30 frames per second. Logic for recording is security... so more frames on good camera gives better chance for id.

I've got enough Intel NUCs to choke a horse including i7-6770 based SkullCanyon one But I don't believe they offer me enough storage space. Is network storage out of question for recording?

I've got an i7-6700k hanging around doing file server duties. But for something this important I'd rather go for robust server equipment including ECC memory.

FWIW, I get a very good discount on some Intel items

 

[fenderman]

The AMDs are only 55W each. Not typical current 100+W per chip

VMs are run on VM host which I believe ESXi is. If you are saying I shouldn't VM this, please explain. Most of the reading I've done says VMs will pass through hardware acceleration.

If I recall right the cameras are 3MP. For recording, I'd like 30 frames per second. Logic for recording is security... so more frames on good camera gives better chance for id.

I've got enough Intel NUCs to choke a horse including i7-6770 based SkullCanyon one But I don't believe they offer me enough storage space. Is network storage out of question for recording?

I've got an i7-6700k hanging around doing file server duties. But for something this important I'd rather go for robust server equipment including ECC memory.

FWIW, I get a very good discount on some Intel items

The TDP does not reflect efficiency....amd, particularly those old models are very inefficient..you will also require a discrete card...put that machine on a killawatt meter under load and you will see how much money you are flushing down the toilet...you will never ever be able to record 20 3mp cameras at 30fps with that weak setup...
you dont need 30fps....15 fps is way more than enough but your system is still too weak...You have the same change of ID at 15fps as with 30...not sure where you are getting your info from...
you dont want to use a nuc...over priced...sell the nuc...buy a 400-500 dollar i7-6700/7700 system and dont look back...
you dont need "robust server equipment" or ecc memory...most folks here are using optiplex or prodesk/elitedesk systems...
I have 20 blue iris systems running on those machines...no issues...
Blue iris should be run on a dedicated system...no reason for vm...and things may have changed but in the past it would not work with HA...regardless, its pointless...

 

[Lil'John]

Thank you for the feedback and discussion fenderman I'm here to learn so my questions are being presented to gather information and not to be overly hostile or argumentative.

I definitely understand TDP doesn't equal efficiency. But I would suspect that running two machines will use more power than one machine with two VMs.

At the end of the day, I'm going to have three 'servers' running in my network closet at any given time:

1. pfSense for security on fiberoptic internet. It is on a i5-4570S 4/4 65W machine.
2. Blue Iris on unknown equipment. Strongly leaning toward VM solution.
3. File/Media server also on unknown equipment. Also strongly leaning toward VM.

I have the AMD equipment because of the rack chassis that will be perfect for the file/media server (24 hot swap bays) The Super Micro motherboard already has integrated graphics which is minimally used... ISMP port plus remoting to machine

On the NUCs, for a reason or two, I cannot sell them for other equipment at this time. They are in hand and available for Blue Iris but lack of storage capacity concerns me.

On not needing "server" grade equipment, my desire isn't entirely based upon need. It is a want. I have a LOT of time invested in creating my media library. I'd rather not have it take a dump over 'cheaper' equipment or overclocked equipment. There is also the security aspect of the cameras. Again, I don't want to lose a valuable video clip over equipment. CPU price is the smaller part of the equation for me. I don't overly care if I spend an extra $200 on a server motherboard and another $100 on ECC memory over standard 7700k system.

I'm not set on doing virtual machines. But unless I'm given a good reason not to, I'm inclined to go VM with ESXi as the baremetal host for them.

For the cameras, I'm not sure what the official final count will be. But the initial will be five. I'm hoping the AMD equipment will handle five for the time being while other things shake out.

For framerate of 30fps, I haven't done research on it at all. It is a guestimate based upon 'gut' feel and watching some content at a low frame rate.

 

[tangent]

In the future, I'd like to go with an E3 v6 Xeon.

A Xeon would be as bad a choice as the opteron. Many Xeon processors lack the graphics hardware acceleration found in the i7's.

On not needing "server" grade equipment, my desire isn't entirely based upon need. It is a want. I have a LOT of time invested in creating my media library.

You really will be better off running Blue Iris on a dedicated machine especially with the number of cameras you're talking about.

There are a lot of things in your list of specs that are just a waste or pointless. Lots of wasted money and electricity here.

 

[auscam]

The 2374 opteron you have is ~2800 passmark single CPU with only ~4800 dual CPU passmark.

The 7700k has ~12000 passmark score and quick sync video. The only xeon with quick sync are some of the E3 models.

I'm taking fendermans previous advice and building a dedicated 7700k system in 2u-3u rack chassis for my setup.

 

[Lil'John]

A Xeon would be as bad a choice as the opteron. Many Xeon processors lack the graphics hardware acceleration found in the i7's.
You really will be better off running Blue Iris on a dedicated machine especially with the number of cameras you're talking about.
There are a lot of things in your list of specs that are just a waste or pointless. Lots of wasted money and electricity here.

The 2374 opteron you have is ~2800 passmark single CPU with only ~4800 dual CPU passmark.

The 7700k has ~12000 passmark score and quick sync video. The only xeon with quick sync are some of the E3 models.

I'm taking fendermans previous advice and building a dedicated 7700k system in 2u-3u rack chassis for my setup.

For the E3 CPU, I was looking at the E3-1275 v6 which did have the graphic attached. I get it for a bit less than I get the 7700k CPU. In doing some reading, it sounds like the consumer side Kaby Lake CPUs are having a bit of an issue with hyperthreading.

On the electricity, I'm based in the US so the 'money' would be negligible for the various servers. Firing up my water cooled overcocked gaming machine probably makes a bigger dent in my electric bill.

On parts wasting money, computers are one of my hobbies. It is a given that most money spent on hobbies is "a waste".

Auscam, thank you for the passmark numbers. My google-ability failed me for finding a reasonable comparison. Very enlightening.

Back to the questions:

1. Is there a reason not to use a NUC for Blue Iris? Especially something like a Skull Canyon(i7 with Iris Pro 580 graphics) Especially if longer term storage is moved off machine.
2. At what point does an i3 or i5 machine make sense?
3. How large of an impact does the 'QuickSync' have on performance of Blue Iris? In the 'old days' with Sandy Bridge/Ivy Bridge, I recall reading that while QuickSync was often quicker, the quality suffered. Is this still the case?
4. What is minimum and ideal system RAM?
5. What is a good number for hard drive space?

If the long term solution isn't the AMD or Skull Canyon that I have in hand, I will be waiting for Coffee Lake to come out or the mentioned E3-1275v6.

 

[jsr182]

For the E3 CPU, I was looking at the E3-1275 v6 which did have the graphic attached. I get it for a bit less than I get the 7700k CPU. In doing some reading, it sounds like the consumer side Kaby Lake CPUs are having a bit of an issue with hyperthreading.

On the electricity, I'm based in the US so the 'money' would be negligible for the various servers. Firing up my water cooled overcocked gaming machine probably makes a bigger dent in my electric bill.

On parts wasting money, computers are one of my hobbies. It is a given that most money spent on hobbies is "a waste".

Auscam, thank you for the passmark numbers. My google-ability failed me for finding a reasonable comparison. Very enlightening.

Back to the questions:

1. Is there a reason not to use a NUC for Blue Iris? Especially something like a Skull Canyon(i7 with Iris Pro 580 graphics) Especially if longer term storage is moved off machine.
2. At what point does an i3 or i5 machine make sense?
3. How large of an impact does the 'QuickSync' have on performance of Blue Iris? In the 'old days' with Sandy Bridge/Ivy Bridge, I recall reading that while QuickSync was often quicker, the quality suffered. Is this still the case?
4. What is minimum and ideal system RAM?
5. What is a good number for hard drive space?

If the long term solution isn't the AMD or Skull Canyon that I have in hand, I will be waiting for Coffee Lake to come out or the mentioned E3-1275v6.

So I'm currently working with an e3 system (1275v2) that supports intel quicksync, but it has a matrix g200ew card onboard with no way to use the built in Intel graphics.

I can't determine if hardware acceleration is working, but I don't think it is (virtually no CPU change when enabling or disabling)

I wish there was a status or way to know for sure if the quick sync engine was being used!

 

[fenderman]

On the electricity, I'm based in the US so the 'money' would be negligible for the various servers. Firing up my water cooled overcocked gaming machine probably makes a bigger dent in my electric bill.

You are wrong...gaming machines are not run 24/7...the difference between an efficient machine and a powerhog can easily git 300-500 per year if you live on the east or west coast...

 

[DLONG2]

Here's a Crosstalk YouTube video on creating an NVR for 10-25 Ubiquiti cameras:

 

[bp2008]

So I'm currently working with an e3 system (1275v2) that supports intel quicksync, but it has a matrix g200ew card onboard with no way to use the built in Intel graphics.

I can't determine if hardware acceleration is working, but I don't think it is (virtually no CPU change when enabling or disabling)

I wish there was a status or way to know for sure if the quick sync engine was being used!

Download GPU-Z. It can tell you if there is usage on your Intel GPU. Mine shows 20% usage when quick sync is working in BI, and 0% otherwise.

 

[bp2008]

Here's a Crosstalk YouTube video on creating an NVR for 10-25 Ubiquiti cameras:

I will admit I am not going to watch that video, but if they are using ubiquiti's proprietary NVR software, that will run on very little hardware and it would be a terrible reference for Blue Iris.

 

[bp2008]

Back to the questions:

1. Is there a reason not to use a NUC for Blue Iris? Especially something like a Skull Canyon(i7 with Iris Pro 580 graphics) Especially if longer term storage is moved off machine.
2. At what point does an i3 or i5 machine make sense?
3. How large of an impact does the 'QuickSync' have on performance of Blue Iris? In the 'old days' with Sandy Bridge/Ivy Bridge, I recall reading that while QuickSync was often quicker, the quality suffered. Is this still the case?
4. What is minimum and ideal system RAM?
5. What is a good number for hard drive space?

If the long term solution isn't the AMD or Skull Canyon that I have in hand, I will be waiting for Coffee Lake to come out or the mentioned E3-1275v6.

1. The more powerful NUCs can handle the load, but they may run pretty hot. I was running Blue Iris on a gigabyte brix pro (similar to a NUC) and it got loud as a small vacuum cleaner and hot enough to thermal throttle, even on a fairly light constant load.
2. When you have a lighter load and a smaller budget. An i3 is basically half of an i7. i5 is somewhere in between. i3 can probably handle 300 megapixels per second without too much trouble. i5, maybe 450-550. i7, 600+ usually.
3. Quick Sync makes a very significant impact as long as your cameras are streaming h.264. Quick Sync doesn't affect video quality when decoding -- only when encoding, which Blue Iris does not use quick sync for.
4. 4 GB works for a few (6?) cameras, but 8 GB is better because Blue Iris isn't the only thing sucking up RAM on the system. Heavily loaded systems should have 12-16GB.
5. Depends entirely on how much recording you are going to do. Lets say you are going to continuously record 20 cameras at 4 Mbps each, that is 80 Mbps. Type into google "80 Mbps * 7 days" and you get 6.04800 terabytes. So a 6 TB drive would hold less than a week of video. But that same drive could hold months or years if you only record on motion detection.

Here is a page I wrote up a while ago discussing hardware needs. Choosing Hardware for Blue Iris | IP Cam Talk

 

[tangent]

You are wrong...gaming machines are not run 24/7...the difference between an efficient machine and a powerhog can easily git 300-500 per year if you live on the east or west coast...

Even at a best case scenario of around $0.10/kwh it adds up to $88 per year for each additional 100W of power consumption with 24/7 operation...

So even best case in a couple of years a more efficient computer would pay for itself.

 

[bp2008]

Many power hog machines idle at 150w and go up from there, and in many places power is over $0.20/kwh.

 

[aristobrat]

Is there a reason not to use a NUC for Blue Iris?

If you do a forum search for 'NUC', seems like there are way more problems reported than success stories, and these are from folks running significantly less cameras than your planned 15 to 20.

Intel NUC NUC6i5SYH
i5-6260U
Samsung 950 M.2 and 16gb ram
DS1515+ over network with a Unifi 16-150

Setup BI on a NUC, i5-6260U, samsung 950 M.2 and 16gb ram. Targets for storage are a local NAS piping through a Unifi 16 port switch.
The NUC regularly stays at +80% CPU and I cannot figure out why.

I have four NUCs myself, two as desktops and two as media servers. For 3-4 cameras, an i5 NUC, 8gb memory and a 1tb SSD would work but 7 cameras, you need more horseypower like suggested above.

Hey guys; fresh install of windows 10 on an Intel nuc d54250wyk. No tweaks to OS done, only Blue Iris and Blue Iris Tools running. Added this cam, with the wizard, all default settings. BI is at 100% cpu. Is my nuc too slow or do i have to configure the cam better?

So i have a nuc, the Intel NUC D54250WYK with the following specs;
CPU: Intel Core i5-4250U
Memory: 16GB DDR3L (SODIMM)
250GB SSD

According to the Intel page it supports Intel® Quick Sync Video; which i think means it supports hardware decoding of the streams.
did a fresh install of windows 10; no tweaks to OS done, only Blue Iris and Blue Iris Tools running. Added this cam, with the wizard, all default settings. When i add just one camera, the CPU usage jumps to about 60% of blue iris.

I had just gone through a setup with a NUC also, a 5i5RYH. Loaded up both WIN7 and WIN10 and it struggled with my 7 cameras at 30fps. It was slow even though CPU was at 50%. Couldn't use the timeline playback on BI without it jumping, freezing or smearing. Had everything setup with all updates and D2D. Had to lower fps to 15 and it still was at 50% but the playback was a little better. Still not very good. So I had an old i7 3770 MB sitting around and tried it out and it was light years better than the NUC. Same setup and CPU was below 20% at 30fps and timeline playback was stutter free. I use the NUC's for HTPC's and they do well for that but it can't handle the stress of BI it seems.

I'm running an Intel NUC5i7RYH with 8 Hikvision DS-2CD2142FWD-I 4mm Lens 4MP. I have all the recommended performance settings running on my BI setup for the exception of my FPS. When I run 5 cameras, everything is smooth but when I added the last 3 my processor jumped to 100%. I've gone as far as dropping everything to 10FPS and it still didn't help reduce processor load. My question is as follows:

1.) Is this processor underpowered for what I'm attempting to do? If so, what's a solid recommended hardware build out for up to 16 cameras? Would it be a custom build or is there a specific server I should get. The budget for horsepower is around $1000 and I already have a separate NAS so I don't need something with mega storage.

Here's more detailed NUC specs:
- 5th Generation Intel Core i7-5557U processor (3.1 GHz Intel Core i7)
- Intel Iris Graphics 6100
- 8GB DDR3 SDRAM

Thanks in advance for any advice here!

-L

On the hardware front, the BI site suggested an i7 powered PC, so I took a bit of a leap of faith and bought a new Intel NUC5i7RYH (NUC based on a 3.1GHz 2 physical core Broadwell-U CPU with integrated Iris 6100 graphics), and installed 16GB memory plus an SSD for the OS (Win 10 Home x64). To this I’ve added a Synology DS415+ NAS with 4x WD 6TB Purple HDDs (a very nice piece of kit). My network switch is a Netgear JGS524PE which has 12 PoE ports and – so far – has been great. Camera-wise I have a Lilin LR7022 (bullet, 2MP/15fps, h264 @ 8Mbit/s) and a Hikvision DS-2CD2632F-IS (bullet, 3MP/15fps, h264 @ 8Mbit/s).

I’ve been experimenting with all this for a couple of days. I hit performance problems early on: CPU usage was around 35% viewing the 2 cameras, without recording. For comparison I tried a couple of instances of VLC media player viewing the h264 streams from each camera. Here I saw 20%. Turning on DXVA 2.0 reduced this to 3%. Looking at some other threads here I see that GPU offload (hardware assist for encode/decode) is in the works for BI so no doubt this aspect will improve.

When I turned on record, the CPU was pretty much saturated. A bit more reading here and I realised that BI is decoding, overlaying with time/date, then re-encoding; direct-to-disk recording for the cameras almost eliminated the recording overhead, I was back to about 40%. Minimising the UI dropped it to under 30%, presumably this is a bit like running-as-service because there is some expense to rendering the live camera video UI. There may be deeper configuration options that further reduce CPU utilisation; can anyone suggest anything to try?

I’m adding 2 more cameras shortly, and hoping the system will scale reasonably; that it will be able to keep up with 8-9 in the end. I will post results. As an aside, these will be Hikvision domes – very impressed with value/features/quality of the Hikvision bullet camera I already have.

One thing I have struggled with is motion detection. Right now I have continuous recording going, and I have enough storage to stay with this even with 9 cameras, but it would be good to catch events to simplify going through footage. Is there any beginners’ guidance on tuning this aspect?

Another random question: video from my 2 cameras is being stored as a series of clips, but the lengths of these are not consistent. Most are 30 minutes - none exceed 3.8GB. Some are much shorter (480MB) - not sure why? (I've turned off motion detect for both cams.)

Anyway, I'm optimistic that I'm going in the right direction - and BI has already shown me that a mysterious damp area on one of my internal walls is going to be easy to resolve: I just need to strangle one of my cats!

 

[jsr182]

Download GPU-Z. It can tell you if there is usage on your Intel GPU. Mine shows 20% usage when quick sync is working in BI, and 0% otherwise.

AWESOME! Thank you for the direction!

 

[DLONG2]

I will admit I am not going to watch that video, but if they are using ubiquiti's proprietary NVR software, that will run on very little hardware and it would be a terrible reference for Blue Iris.

I found the video informative for the general use of surveillance recording of multiple cameras, featuring 7th generation i3, the WD Purple HDD, etc.

 

[Lil'John]

You are wrong...gaming machines are not run 24/7...the difference between an efficient machine and a powerhog can easily git 300-500 per year if you live on the east or west coast...

Even at a best case scenario of around $0.10/kwh it adds up to $88 per year for each additional 100W of power consumption with 24/7 operation...
So even best case in a couple of years a more efficient computer would pay for itself.

My gaming machine/desktop is a VERY high end setup. Two high end GPUs at 250W each and a 3930k CPU at 130W and those are stock numbers and not over clocked number... yeah, I think my gaming machine running for eight hours a day will easily pull more power than the AMD machine running 24 hours

If you do a forum search for 'NUC', seems like there are way more problems reported than success stories, and these are from folks running significantly less cameras than your planned 15 to 20.

Thank you VERY much for the partial search results. It looked like the common issue with most was they were using low end NUCs(celeron, pentium, and i3/i5 which are actually mobile... not equivalent of desktop)

1. The more powerful NUCs can handle the load, but they may run pretty hot. I was running Blue Iris on a gigabyte brix pro (similar to a NUC) and it got loud as a small vacuum cleaner and hot enough to thermal throttle, even on a fairly light constant load.
2. When you have a lighter load and a smaller budget. An i3 is basically half of an i7. i5 is somewhere in between. i3 can probably handle 300 megapixels per second without too much trouble. i5, maybe 450-550. i7, 600+ usually.
3. Quick Sync makes a very significant impact as long as your cameras are streaming h.264. Quick Sync doesn't affect video quality when decoding -- only when encoding, which Blue Iris does not use quick sync for.
4. 4 GB works for a few (6?) cameras, but 8 GB is better because Blue Iris isn't the only thing sucking up RAM on the system. Heavily loaded systems should have 12-16GB.
5. Depends entirely on how much recording you are going to do. Lets say you are going to continuously record 20 cameras at 4 Mbps each, that is 80 Mbps. Type into google "80 Mbps * 7 days" and you get 6.04800 terabytes. So a 6 TB drive would hold less than a week of video. But that same drive could hold months or years if you only record on motion detection.

Here is a page I wrote up a while ago discussing hardware needs. Choosing Hardware for Blue Iris | IP Cam Talk

More awesome information.
1) I haven't had a chance to fire up the SkullCanyon to see how bad the noise is.
2) I definitely am familiar with the i3/i5/i7 distinction. Intel has done some things to muddy the water between i3/i5/i7 depending upon application. Depending upon level of overclocking, some i3s will beat i5s.
3) To follow on this, Blue Iris doesn't use quick sync for encoding for write?
4) Good news. I wasn't sure if Blue Iris liked 32GB or more 16GB is my usual for machines.
5) Thank you for the analysis here. It looks like my plan for four or six 4TB drives in raid6 will be more than sufficient. My plan is to record only motion on about six or so cameras. At least two cameras will be 'buried' so only breaking in would result in motion.