Suggestions for a video card to offset CPU video processing?

[crw030]

I only have one 8gb stick in the machine now.

Duh. I did have 16gb RAM DDR3 1200.
I'm going to try a video card anyway... suggestions?

I think what's important is if you have ONE memory stick or TWO. If you are running that chip with 16GB memory on a single DIMM you are running in single-channel-mode and your performance will likely be significantly hamstrung compared to 2x8GB sticks in the proper slots to run dual-channel-mode.

If you have ONE 16GB memory stick only, then bad news is you should get another 16GB memory stick (of similar type/speed as you have) before looking at graphics cards, or replace it with 2x8GB RAM and retest.

Please see @bp2008 very good analysis here: Ryzen 3950x for Blue Iris regarding effects of running fewer memory channels than your cpu/motherboard support. For comparison, Ryzen 3950x single-channel-mode memory is 24GiB/s and your i7-4790 is slightly less than that in dual-channel-mode, or less than HALF that in single-channel-mode. @bp2008 was able to achieve 650-880 MP/s single-channel, so for your computer to be struggling at HALF that would make a lot of sense if memory bandwidth is your issue.

Memory Channels Matter

To test the hypothesis that memory bandwidth was the limiting factor, I removed two 8 GB sticks from channel A, leaving only two 8 GB sticks in channel B. HWINFO confirmed that the memory was now running in single channel mode.

I then measured performance while running the baseline workload from before (1189 MP/s).

Single Channel Memory Results

Megapixels Per SecondNotesPower Consumption (Watts)Blue Iris CPU Usage %Overall CPU Usage %Nvidia RTX 2080 Ti Video Decode Usage %813Software Decode, NO GUINot Measured4853disabled835Software Decode, NO GUINot Measured3638disabled1013Software Decode, NO GUINot Measured3334disabled884Software Decode, NO GUINot Measured3334disabled654Software Decode, GUI @ 4KNot Measured8083disabled
Performance was all over the place, and frames were being dropped. During the GUI Open (at 4K resolution) test, the GUI was extremely sluggish and it was evident that many cameras' live views were delayed.

Next, I re-inserted the two RAM sticks I had removed, and turned off the XMP Profile. In effect, this reduces the memory speed from 3600 MHz to 2133 MHz, and slightly improves memory latency.

I re-ran the performance measurements.

Megapixels Per SecondNotesPower Consumption (Watts)Blue Iris CPU Usage %Overall CPU Usage %Nvidia RTX 2080 Ti Video Decode Usage %1089Software Decode, NO GUINot Measured1718disabled1089Software Decode, NO GUINot Measured1718disabled1087Software Decode, NO GUINot Measured1717disabled1086Software Decode, NO GUINot Measured2122disabled1089Software Decode, GUI @ 4KNot Measured3334disabled
Based on this, I think it is fair to say that Blue Iris is highly dependent on memory speed. Faster memory is better, and it is simply foolish to not run as many memory channels as your platform supports.

 

[crw030]

If this turns out to be the issue, please let us know. It is covered in the WIKI:

Choosing Hardware for Blue Iris
Memory speed matters, and it is important to utilize all the memory channels available on your chosen platform. Most systems are capable of running dual channel memory, and I highly recommend doing so if you are going to be pushing the limits with your system.

 

[marklyn]

After you have made the changes and rebooted your PC, post screen shots...

1) the Windows 10 task manager Process tab, include the GPU and GPU engine columns. sort with the most memory at the top (BI).
2) BI status cameras , make sure that the MP/s at the bottom left of the frame is in the screen shot.
3) BI status clip storage

SouthernYankee, those changes I made, based on your recommendations made a big difference. My CPU now runs 19-21% consistently. I've attached the three screen shots you asked for. 3 of the cameras I couldn't really do much with the frame/iframe rate (they are odd Reolink cams) but the rest fell in line. The storage clip may seem like I'm running out of space but I've tweaked BI storage settings over months to get it where it consistently leaves around 450Gb free on a 4 Tb drive. Hasn't been an issue for many months now and stays around to 450Gb free mark always.
I've been doing this for years and learned much about BI features/tweaks but my weakness is understanding the mechanics of the cameras better. I guess I'll always be learning from folks like you. Thanks so much for your recommendations, they made a big difference. Could you venture a guess what was the most significant change to bring down the CPU usage so much?

 

[marklyn]

If this turns out to be the issue, please let us know. It is covered in the WIKI:

For the record, and I forgot to elaborate, I have two 8Gb memory sticks, both DDR3 and one was a little faster than the other (as I recall). I was looking at something incorrectly earlier when I thought I had 8 Gb.

 

[Walrus]

SouthernYankee, those changes I made, based on your recommendations made a big difference. Could you venture a guess what was the most significant change to bring down the CPU usage so much?

SouthernYankee made suggestions based on what he uses, but we have no idea what changes you actually made. You already had h.264, hardware acceleration and direct to disk. The 'limit decoding' recommendation will significantly drop CPU. He also suggested lowering the FPS, but you haven't done it looking at your screenshot. Make one change at a time and note the CPU change.

Your bitrates still look out of balance for the MP and FPS of each cam.

 

[SouthernYankee]

You have an allocation problem, There should be no red on the clip storage. You have allocated 509 GB that you do not have.

screen shots of
Blue iris settings -> clips and archiving tab. Three screen shots, one each for the new stored and alert folders.

 

[mech]

As a fact-finding step, I did kidnap a Quadro P620 for the night, and saddled it with ten cameras, mostly 2MP. I'll attach a screenshot showing the result while displaying all the cameras. The interesting part is that CPU usage is actually not reduced. The workload on the Intel UHD Graphics (doing the Intel Quick Sync dedoding) drops substantially as the P620 picks up the slack, but CPU proper remains around 38%.



Playback of all my cams while continuing to record them (about 1600MP/sec) is still beyond the combined power of an i9, a GTX1660 and a P620. The Quadro goes right to the ceiling with its ten cams. That's always the acid test, can you play back the group of cams you want to view when it's showtime. In my neighborhood, "showtime" might mean gunshots and a car speeding away, or screaming and glass breaking. My "Perimeter" group consists of ten cams (it's an apartment building) and my system can do that, here's what that looks like:



One thing I do advise, is to have your hard drive defragment daily. I left my 10TB drive up to Windows to defrag, and the fragmentation level was so high that a week of non-stop Defrag could not recover it. A daily defrag has kept my mechanical drives at low levels of fragmentation and might be a modest improvement.

 

[marklyn]

SouthernYankee made suggestions based on what he uses, but we have no idea what changes you actually made. You already had h.264, hardware acceleration and direct to disk. The 'limit decoding' recommendation will significantly drop CPU. He also suggested lowering the FPS, but you haven't done it looking at your screenshot. Make one change at a time and note the CPU change.

Your bitrates still look out of balance for the MP and FPS of each cam.

First I set most of my camera FPS from what they were (mostly around 20-25fps) to around 12-15 fps (except two in my front of the house which I want higher fps). Then I matched the inter frame rate to the fps for those same cams.
That seemed to drop my average CPU usage down to around a steady 35-40% but I didn't monitor usage as long as I should have.
I turned on the limit decoding recommendation on all of my cameras. That's what seemed to drop the CPU usage down to the levels they are at now (around 15-19%).
My GPU also now hovers around 3%.
So as I'm happy with the changes, this gives me some good info to read up on and possibly do further testing to understand the differences related to these settings.
SouthernYankee, yes, I know I show an over-allocation and I don't understand why when I've got my BI storage/new settings tweaked tightly to keep around 450Gb free (been that way for months now after all the tweaking).
I will re-visit that again and research more as to why BI shows the over allocate, but it never goes lower than 400Gb of space.
Thanks all for your help and explanations. I've got lots to read up on and play around with, considering my 'developed' understanding.

 

[SouthernYankee]

Mech

Never defrag the video drive. All you are doing is decreasing the life of the drive. The video drive is high write low read rates. Defrag only helps with high read. Most video drives have a defrag value of 95+%. As you have multiple files writing at the same time, this will chop up the continuity

 

[mech]

Mech

Never defrag the video drive. All you are doing is decreasing the life of the drive. The video drive is high write low read rates. Defrag only helps with high read. Most video drives have a defrag value of 95+%. As you have multiple files writing at the same time, this will chop up the continuity

If a drive can't handle 10-15 minutes a day of defragging, then I'd say there's something wrong with it. My 10TB Seagate EXOS just wrapped up a full defrag in under 12 minutes. Its last defrag prior to that was the nightly one that ran around midnight.

Maybe it's not the end of the world for everyone, but with over 20 cams recording, playing back a set of them can be stressful. I ended up offloading the New recordings to a high-endurance 2TB nVME SSD, which is aces at reading while writing (and indifferent to fragmentation), and this further helps the mechanical drives since they receive the cascaded recordings as nice tidy 2TB files they can store in one chunk, not something they built as they went.

 

[SouthernYankee]

Look at the number of writes it takes to move one cluster in a defrag operation. Every video file that is written to a disk is fragmented if there is more than one camera on the system. Each new cluster write takes at least 3 write. A fragment move can take more than 8 writes. The life of a drive is dependent on the number of write. Most video is never looked at so there is near zero reads.

If you are moving from a new folder to a stored folder on a different device the complete file is moved in one operation, and there is not multiple different file writes at a time, so the file may not be very fragmented.

 

[mech]

That was part of the attraction of putting in the SSD, it is undoubtedly helping a lot since the files arrive on the mechanical drives in one chunk. Prior to that, most of the cams were being recorded to the 10TB drive, and when called upon to play back 15 hyper-fragmented files at once while continuing to record to them... ouch.

We'll see how long the SSD can take the punishment. I did leave about 25% of it unallocated so there's spare blocks to use for wear-levelling as it wears out. But I've never lost a mechanical drive so far, even my mediocre WD Reds that are now ?4 years? old. There, now I've jinxed myself for sure

 

[JNDATHP]

Will “Limit decoding unless required” affect motion detection?

 

[Walrus]

It used to, but a recent BI update in the recent months supposedly alleviated that.

 

[SouthernYankee]

Marklyn

As it appears that you are writing your BI data to an external drive (not sure what). The windows 10 write cache policy has changed for these drives, the write back cache may set to reliability, so write occur immediately, this improves reliability but decreases performance, In the case of a network drive it may increase network traffic.

NAG... you need to fix your allocation problem in BI.

 

[marklyn]

Marklyn

As it appears that you are writing your BI data to an external drive (not sure what). The windows 10 write cache policy has changed for these drives, the write back cache may set to reliability, so write occur immediately, this improves reliability but decreases performance, In the case of a network drive it may increase network traffic.

NAG... you need to fix your allocation problem in BI.

My BI Database for clips is on the same drive which is a SSD drive... the actual clips are going to a NAS drive (QNAP).