High CPU Useage on 22 Camera build system

[cainrand]

I shopped around today. I can get a GA-Z270X-Gaming 7 motherboard with a i7-7700k and case for about $600. Any thoughts before I pull the trigger?

I was actually just looking at upgrading my 8yr old entertainment/gaming system and thought this was a pretty good deal, then thought, maybe this would be a great system for a new NVR (not that I need a new pc/nvr)
Amazon.com: Intel 7th Gen Intel Core Desktop Processor i7-7700K (BX80677I77700K) & ASUS LGA1151 DDR4 DP HDMI M.2 USB 3.1 ATX Motherboard ROG MAXIMUS IX HERO Bundle: Computers & Accessories

 

[Tolting Colt Acres]

That's an option too. Although, I may go with a GA-Z170X motherboard instead, because it offers 8 SATA ports. I'd prefer to have an isolated boot drive and then RAID my 6 6TB drives together. Not that Windows software RAID is that great, mind you, but I really do not want to invest in a separate SATA raid card. I wish there was a way to move my PERC H800 into the new system and just interchange the cables to use an Adaptec CBL-00079 (SFF-8087 to 4x7-Pin SATA Fanout Cable), but nothing is ever that easy. Looks like a Dell X4D23 cable will allow me to connect my H700 controller internally to the SATA drives.

 

[cainrand]

Just curious.. Why do you want/need RAID?

 

[vialentvia]

DTD made zero difference on my 13-camera system (PowerEdge R710 dual 6-core X5675 processors). I still hover around 75% CPU utilization. Even worse if I use BI as a service and then run the front-end periodically to see what is going on. Makes the system incredibly sluggish.

Good to know on the dual X5600 R710 I had often wondered how they would fare. I have an R710 with dual X5670s but it is for virtualization in our home network ( I do IT work and we have a multi-dwelling multi-business property. Decent size network).

On a Poweredge for that size system, all you really need to run 13 cams very well is an R220 with an E3-1246 v3 Haswell or a newer R230 with an E3-1245 v5 Sky Lake. These both have QuickSync and will really help the CPU out. I have been looking at both as I have a 12 3MP cam system myself virtualized on an I7-4790K (~23% CPU on host and ~50% on VM with 4 vCPU dedicated) and am getting tired of hearing the water cooler. I look to add about 4 more cams on the new server.

I compared the performance of the 4790K which to me is the King of the Haswells, and those two are the only ones that come close to the performance in a Xeon while maintaining onboard graphics.

The King of the Haswells allows me to game on the host (Arma 3 and Just Cause 3) while running BI in the VM and not exceeding ~75% of the CPU. To me, that is a pretty stout processor. I'm actually impressed.

 

[Tolting Colt Acres]

Just curious.. Why do you want/need RAID?

My NAS contains what my wife and I would classify as "mission critical" backups of our datasets. The additional cost for hardware RAID on the R710 was about $75 (at the time off eBay, not sure what they're going for now) for a PERC H700 internal raid card.

On a Poweredge for that size system, all you really need to run 13 cams very well is an R220 with an E3-1246 v3 Haswell or a newer R230 with an E3-1245 v5 Sky Lake. These both have QuickSync and will really help the CPU out.

I would lose 2 of my 6 drives though, so I'm somewhat limited to 2U solutions if I go that route (I need the storage - my system has a mere subset of the datasets I have at work, which encompass several petabytes of storage on my HPCC). And from the looks of things, the cost of the i7-7700 with motherboard, ram, and case are about the same as a 2U solution.

I had considered moving the 6 drives into an external MD1200 or MD1000 enclosure (even the 3G MD1000 should suffice) I'm still looking at ~200 minimum for the enclosure/H800/cables, and then I'd have to still build out a replacement system for the R710.... so I'm still looking at a $700 price tag, even with an older used i7 system often mentioned here (quoted about $500ish)

 

[vialentvia]

I meant in addition to letting your R710 manage your current data, making BI hardware dedicated on the 220/230. Two WD Purple drives are all you need, they only come in 3.5" though so you would be limited to that size drive chassis. Still looking at 700 bucks whether you build or buy a system. I was looking at Stallard Technologies who sells an R220 for $250 with no applicable processor. The E3-1246 v3 is about $280. Then about $100 each for WD Purple 2TB drives.

My two 6-bay 710s are in a VC6 setup with vNAS together to make all the 2TB drives one large volume from both hosts. Pretty sure the 710 is limited to 3TB max drive size. I've still not filled up the remaining 10 bays with 2TB drives. I have two 300GB 10K SAS drives that run my VMs and the rest is storage and file shares. My BI storage will always be separate dedicated drives.

 

[Tolting Colt Acres]

I have my 710 populated with 6 6TB drives. The H700 has two VDs, a 2TB OS volume, and the rest (~26TB) a data volume.

I do not recall what the 710 came with stock. Maybe a SAS 6I? (I forget TBH, its been so long) That may be where your limitation is on drive/volume size. First thing I did was swap out the stock controller for the H700. 6x2TB drives barely allows me to mirror my Alienware 18.

 

[vialentvia]

It was the PERC 6/i, yes. I won't see the benefit of an H700. 8TB is barely utilized for our current data. I buy a drive at a time, but a 6TB drive is a bit ridiculous for an enterprise drive price wise. I won't use a consumer grade drive, the MTBF is too low.

This thread actually has me excited about an R220 build honestly. I can't wait to stop hearing fans from the water cooler.

 

[Tolting Colt Acres]

Yeah, I have all ST6000NM0024's. Wasn't cheap but it was the only way I could get the storage I needed. I'm waiting for the 10TB units to come down in price and then I'll swap everything out.

 

[DarrenD]

Thank you to everyone in this thread and IPCamtalk in general. I have to conclude that IPCamtalk rocks a phat jam!

If it weren't for this forum, I would have given up on Blue Iris and ended up either having three different camera servers with three different free 16 channel systems (ACTi, Hikvision, and ExacqVision which we paid for 16 licenses years ago) OR I would have paid out the nose for 24 additional licenses to ExacqVision or equivalent.

Now I have 40 cameras (HikVision/ACTi mix) on one server (Win 7 desktop actually) running Blue Iris. The desktop is a Lenovo ThinkCentre (8 Gb RAM, Intel i7-4770S) and it's running at about 45%-55% CPU usage. Amazing!

At first, of course, my CPU was consistently hitting 100% and I just thought that there were too many cameras to handle the load. Thanks to all the tips on this forum, we are back in business!

 

[Tolting Colt Acres]

Well, I just finished migrating my system over to the new configuration I purchased, and I have to say, I'm worse off than I was before. This is very disappointing to say the least.

GIGABYTE AORUS GA-Z270X-Gaming 7 Gaming Motherboard w/ 16gb RAM and an Intel i7-7700K...

CPU is 92-95%, system is so sluggish I can barely get the mouse to move.

I was better off with the dual xeon configuration I had.

 

[Tolting Colt Acres]

Yes, I have verified all of my cameras record direct to disk:

 

[fenderman]

Well, I just finished migrating my system over to the new configuration I purchased, and I have to say, I'm worse off than I was before. This is very disappointing to say the least.

GIGABYTE AORUS GA-Z270X-Gaming 7 Gaming Motherboard w/ 16gb RAM and an Intel i7-7700K...

CPU is 92-95%, system is so sluggish I can barely get the mouse to move.

I was better off with the dual xeon configuration I had.

You are doing something wrong...what it is who knows..we know this is NOT normal...lots of folks running higher loads than you on weaker systems..
So tell us about the setup..are you remoting in? did you setup hardware acceleration etc...it may be related to server 2016...
your memory usage seems abnormally high...
What else is on this system..dont leave anything out regardless of how trivial...you should be at about 30-40 percent load...
old xeon processors have always been and continue to be terrible choices for Blue Iris machines..heck for any machine..

 

[Tolting Colt Acres]

I am not remoted in. The system is directly connected via an HDMI cable to my Samsung 4K SmartTV. It provides DNS and DHCP service to my network, and acts as a file server for backups (overnight) of my wife's laptop and my own. That's about it. I do not have anything else running on it, in TaskManager it looks like I have the standard Windows services running. I believe I have hardware acceleration turned on (for each camera, have "Hardware decoding" set to "Yes (no VPP)" under the Video tab). The cameras do the date/time overlay, so I have all that shut off. Have direct to disk enabled as well on all. Nothing else installed, other than an SSH client and RealTerm, which I use to connect to the managed switch from time to time if I'm working at the server. I do not really have anything else installed on this box, it mainly serves as a NAS and not as a workstation. 4 cores, hyperthreading and virtualization enabled. Anything else you might want to know?

I have a Win10Pro HDD I can throw into the machine tomorrow and see if that makes any difference. Windows Server was originally a necessity, as I had all my Hikvision cameras streaming to their own individual SMB shares set up on the server, and the desktop versions of Windows have a 20-concurrent connection limit.

 

[Tolting Colt Acres]

Fresh install of Windows 10 Pro. Applied all updates. Fresh install of Blue Iris. Added cameras. Stopped when I got to the 5th. Selected D2D during install, 10fps default, hardware acceleration (Yes-No VPP). Only 2 things changed during setup of each camera were turned off overlays (camera does them) and added 10 second lead-in for recording on motion. Cameras are configured as 10fps w/ exception of those on the street, which are 20fps.

Camera 1: 8% CPU 470MB RAM (10fps camera)
Camera 2: 17% 915MB RAM (10fps camera)
Camera 3: 33% 1325MB RAM (20fps camera)
Camera 4: 66% 1750MB RAM (20fps camera)
Camera 5: 97% 2090MB RAM (20fps camera)

 

[fenderman]

Fresh install of Windows 10 Pro. Applied all updates. Fresh install of Blue Iris. Added cameras. Stopped when I got to the 5th. Selected D2D during install, 10fps default, hardware acceleration (Yes-No VPP). Only 2 things changed during setup of each camera were turned off overlays (camera does them) and added 10 second lead-in for recording on motion. Cameras are configured as 10fps w/ exception of those on the street, which are 20fps.

Camera 1: 8% CPU 470MB RAM (10fps camera)
Camera 2: 17% 915MB RAM (10fps camera)
Camera 3: 33% 1325MB RAM (20fps camera)
Camera 4: 66% 1750MB RAM (20fps camera)
Camera 5: 97% 2090MB RAM (20fps camera)

Confirm you entered the license key for this new install. BI does not support d2d in demo mode. Try connecting to a 1080p display and see if there is a change...your post is unclear, did you go into each cameras record settings and change it to D2D? you need to do this for each camera independently. Also assuming there is no discrete card in there and you are using intel HD..

 

[fenderman]

@Tolting Colt Acres
See my edit above: Confirm you entered the license key for this new install. BI does not support d2d in demo mode. Make sure windows is set to high performance..

 

[Tolting Colt Acres]

Yes, I did enter the license key, and the system shows my license key with the xxxx's on the About screen. I am set for high performance. Downscaling to a 1080p resolution doesn't have an appreciable effect. I am using the Gigabit's on board video, Intel HD 620.

I set each camera up as follows: Add Camera -> Provided Full Name and Short name, type "network ip", and clicked on "direct to disc recording" checkbox (left enable motion detection clicked on as well, the default). Clicked Ok.
On network IP camera config screen, gave ip address, username and password, and clicked "find/inspect". once process completed, clicked close to return to network ip camera config screen. clicked ok.
brings me to new camera window, "video" index tab. left "max rate" at 10fps. Changed "hardware decoding" to "Yes (No VPP)" and un-checked the "enable" checkbox for "Text and graphic overlays".
clicked on "record" index tab. changed "pre-trigger video buffer" to "10" for 10 seconds.
clicked ok. camera added to video wall.

repeated this process for 5 cameras, making note of cpu and memory utilization after each, per previous post. stopped at 5 cameras when cpu utilization was in excess of 90%.

 

[Tolting Colt Acres]

Also have gone back in to each camera and verified that D2D is enabled for each camera of the 5.

 

[fenderman]

Also have gone back in to each camera and verified that D2D is enabled for each camera of the 5.

Pull and older update file that has been posted to the site and try with that...This is not making any sense..Post the record tab...