BI CPU utilization and number of processor cores

[mklemp]

I'm running 32 cameras (26 x 3 meg cams & 6 x 8 meg cams) and have BI running on a Dell Poweredge R720 with windows server 2019

The Dell Poweredge has dual E5-2690 processors (16 cores total), 128G of memory, OS running on a 1 TB SSD, and 24TB of mechanical storage for video.

My CPU utilization is pegged at 100% despite all of the usual tweeks, direct to disc, max 15 FPS, hw acceleration and limiting bandwidth to 4096 for each camera.

I can upgrade the CPUs to E5-2697 v2s which drops the clock speed from 2.9mhz to 2.7mhz but bumps the cores from 16 to 24 total.

Anyone have throughts on the realized benefit of adding eight more cores?

 

[wittaj]

I suspect you are not using the substreams.

Do EVERY optimization in the wiki and you will see a major drop in the CPU. Substreams are a must, especially once you get over 4MP cameras.

A member here runs 50 cameras on a 4th generation CPU at 30%, so I am sure your better processor will result in better CPU %.

And by EVERY, I mean EVERY. Too many people come here complaining of high CPU usage and claim they have done every optimization in the wiki and once they post screenshots, we see they are not using the substreams and that is probably one of the biggest CPU savers. Do not skip one because you think it isn't important or won't make that big of a deal. Even dropping frame rate a few FPS can make a big difference. No reason to run more than 15FPS, and many us have cams running at 10 to 12 FPS.

If you do not understand what something does in BI, then ask. Too many people also change a setting not realizing what it does and actually makes their performance worse.

Please post a screenshot of your BI camera status page that shows FPS, MP/s, etc.

Optimizing Blue Iris's CPU Usage

Blue Iris uses a lot of CPU time. In most Blue Iris installations, your CPU is the limiting factor which determines how many cameras you can have. As such, it is extremely important to optimize your configuration to ensure you can get the most out of your system while consuming the least...

ipcamtalk.com

 

[mklemp]

 

[wittaj]

Yep, you are not using the substreams. Do that and your CPU will drop tremendously.

Another issue, but your KEY frames are horrible and you are probably missing motion and do not even know it. KEY should be 1.00 which means FPS and iframes match.

A KEY of 0.14 means if an object can pass through your field of view in 7 seconds, you could miss it. BI recommends a KEY no less than 0.5 but suggests 1.00. I suspect you are using cameras that are changing it to what they see fit to try to give a bright static image. But that is another issue to tackle later LOL.

Go to substreams if the cameras allow and watch it drop.

 

[mklemp]

Thanks for the help and yes I am missing motion.

I'll see someone show up on a camera in the middle of the church and they passed three other cameras to get to that spot. No recording was captured on the other cameras. I literally just adjusted the motion settings on all of the cameras today.

 

[wittaj]

Motion settings isn't the problem, you need to match FPS with iframe. So if you are running 15FPS, then the iframes need to be 15.

But I see that many of your cameras are dropping down to 1, 2, 5 FPS. What FPS did you set in those cameras? If the cameras are dropping FPS and increasing iframes, not much you can do in the BI motion settings to catch all the motion.

What brand(s) of cameras are they?

 

[wittaj]

The most famous cameras that do this switching of FPS and iframes is Reolink.

Blue Iris and Reolinks do not work well together, but the same principles applies for almost any low end consumer grade camera. It is just Reolinks is one of the more consumer end cameras people buy and come to this site as to why it is pointed out often about. I have a cheapo camera for overview purposes so it doesn't matter, but it exhibits this same behavior even though in the settings I can set an iframe...

This was a screenshot of a member here where they had set these cameras to 15FPS within the cameras (and look some of the sub FPS were dropped to 5 and KEY of 0.25 which is a recipe for missed motion):

Now look at they key - that is the iframes ratio. Blue Iris works best when the FPS and the iframes match. Now this is a ratio, so it should be a 1.00 if it matches the FPS. The iframes not matching (that you cannot fix or change with a reolink) is why they miss motion in Blue Iris and why people have problems. This is mainly why people are having issues with these cameras and there are many threads showing the issues people have with this manufacturer and Blue Iris. It is these same games that make the camera look great as a still image or video but turn to crap once motion is introduced.

The Blue Iris developer has indicated that for best reliability, sub stream frame rate should be equal to the main stream frame rate and these cameras cannot do that and there is nothing you can do about that with these cameras... The iframe rates (something these cameras do not allow you to set) should equal the FPS, but at worse case be no more than double. This example shows the cameras going down to a keyrate of 0.25 means that the iframe rates are over 4 times the FPS and that is why motion detection is a disaster with these cameras and Blue Iris...A value of 0.5 or less is considered insufficient to trust for motion triggers reliably...try to do DeepStack and it will be useless...

Compounding the matter even worse...motion detection is based on the substream and look at the substream FPS - they dropped down to below 6 FPS with an iframe/key rate of 0.25 - you will miss motion most of the time with that issue...DeepStack probably won't work at all...

Blue Iris is great and works with probably more camera brands than most VMS programs, but there are brands that don't work well or not at all - Rings, Arlos, Nest, Some Zmodo cams use proprietary systems and cannot be used with Blue Iris, and for a lot of people Reolink doesn't work well either.

Now compare above to mine and cameras that follow industry standards that allow you to actually set parameters and they don't manipulate them. You will see that my FPS match what I set in the camera, and the 1.00 key means the iframe matches:

 

[mklemp]

My 3meg cams are Hikvision and the 8meg cams are Annke.

Thanks for the details on the iframe matching. I can make that adjustment when I enable the substreams.

I'll post an update with the results.

 

[wittaj]

Oh then yeah you should be able to adjust those!

 

[SouthernYankee]

Do the camera and bi optimization.

One of the problems with running a single app on a multi CPU chip machine is that it does not use the chip cache very efficiently. The processing threads can switch cpus and cause an internal cache flush. The apps must be custom written to work well on a system with multiple physical cpus.

 

[SouthernYankee]

look at passmark to compare CPUs.

PassMark - CPU Benchmarks - CPU Mega Page - Detailed List of Benchmarked CPUs

The E5-2690 has a rating of 9955
The E5-2697 v2 has a rating of 14165

I have a question ?? How is BI showing an "I" for intel "HA" hardware acceleration on the camera status when the E5-2690 does not have a GPU ?

 

[mklemp]

Big difference in CPU utilization by enabling substreams.

 

[jlindblo]

The most famous cameras that do this switching of FPS and iframes is Reolink.

Blue Iris and Reolinks do not work well together, but the same principles applies for almost any low end consumer grade camera. It is just Reolinks is one of the more consumer end cameras people buy and come to this site as to why it is pointed out often about. I have a cheapo camera for overview purposes so it doesn't matter, but it exhibits this same behavior even though in the settings I can set an iframe...

This was a screenshot of a member here where they had set these cameras to 15FPS within the cameras (and look some of the sub FPS were dropped to 5 and KEY of 0.25 which is a recipe for missed motion):

Now look at they key - that is the iframes ratio. Blue Iris works best when the FPS and the iframes match. Now this is a ratio, so it should be a 1.00 if it matches the FPS. The iframes not matching (that you cannot fix or change with a reolink) is why they miss motion in Blue Iris and why people have problems. This is mainly why people are having issues with these cameras and there are many threads showing the issues people have with this manufacturer and Blue Iris. It is these same games that make the camera look great as a still image or video but turn to crap once motion is introduced.

The Blue Iris developer has indicated that for best reliability, sub stream frame rate should be equal to the main stream frame rate and these cameras cannot do that and there is nothing you can do about that with these cameras... The iframe rates (something these cameras do not allow you to set) should equal the FPS, but at worse case be no more than double. This example shows the cameras going down to a keyrate of 0.25 means that the iframe rates are over 4 times the FPS and that is why motion detection is a disaster with these cameras and Blue Iris...A value of 0.5 or less is considered insufficient to trust for motion triggers reliably...try to do DeepStack and it will be useless...

Compounding the matter even worse...motion detection is based on the substream and look at the substream FPS - they dropped down to below 6 FPS with an iframe/key rate of 0.25 - you will miss motion most of the time with that issue...DeepStack probably won't work at all...

Blue Iris is great and works with probably more camera brands than most VMS programs, but there are brands that don't work well or not at all - Rings, Arlos, Nest, Some Zmodo cams use proprietary systems and cannot be used with Blue Iris, and for a lot of people Reolink doesn't work well either.

Now compare above to mine and cameras that follow industry standards that allow you to actually set parameters and they don't manipulate them. You will see that my FPS match what I set in the camera, and the 1.00 key means the iframe matches:

Hmmm ..... what program is generating these statistics? Within BI itself? It doesn't seem to be Blue Iris Tools.

 

[wittaj]

Hmmm ..... what program is generating these statistics? Within BI itself? It doesn't seem to be Blue Iris Tools.

It is within BI itself.

It is the camera status page when you select the lightning bolt/graph icon (depending on version) in the upper left. It has a tab that says "Cameras" and shows all of the main camera items - name, IP address, FPS, Main/Sub/KEY, etc. all in one place.

 

[jlindblo]

Thank you sir ... I appreciate. Is each individual ratio also shown in each camera "Properties/video" settings? I see that most of my sub streams are 50% .. and yep ... Reolink.

 

[wittaj]

Yes, they are shown in the individual cameras as well, but the snapshot is nice.

 

[jlindblo]

Ok ... I looked but I must be blind. On the Record Page? ....

 

[wittaj]

On the main camera page when you open up the camera settings in BI.

 

[jlindblo]

Ok ..... I was looking past. Thank you again ....

 

[mklemp]

Appreciate everyone's help and quick response with resolving my CPU utilization problem. My BI application is running amazingly better.
One of the items brought up was my FPS/key ratio. I've adjusted both the mainstream and substream to have a 1-1 FPS to iframe ratio.
All of the 3meg Hikvision cameras show good stats now but the 8meg Annke camera substream ratios still seem problematic and I'm missing motion.
I've played with the substream settings in an attempt to improve but I'm not getting the desired improvement.
Any suggestions here would be appreciated.