Does BI need to be rebooted often? or how to optimize stability? [RDP high CPU usage]

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
I've got a fresh new install of 5.7.1.3 running as a service that I started using yesterday with a single camera.
Played with some options to get to know BI and went to bed.
This morning I open it again and it's pretty laggy and behaving weird; empty logs list, takes a few seconds to redraw when changing any settings tabs, PTZ commands either don't work or take 10 seconds to execute, changing camera settings (removing substream config) didn't trigger that camera reset with the multicolor rectangles, etc.

Everything else (browser/OS) on that BI PC was working well at the time, so I rebooted it and it's now working well again, but now I'm worried about stability and worried BI could let me down precisely when I need it most.

CPU usage was 'normal' ~40-50% when in the BI GUI, none of the cores maxed out; this is an older HP Z400 workstation that I wanted to recycle as a BI machine. Xeon W3530 cpu with 12GB ram and SSD for the OS/BI and a P106-100 for CodeProjectAI. No intel Quicksync.

Are there any 'must have' settings that help with stability?
 
Joined
Aug 3, 2015
Messages
3,791
Reaction score
12,182
Location
Charlotte
1. Read the wiki page regarding sub-streams - I just did it this week, and it's been long overdue. CPU and RAM usage drop dramatically.
2. Get the right version of the Intel integrated video drivers, to prevent memory leaks.
3. Update to the latest version of CodeProject.AI if you're using it.
 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
1) Yep, substreams were enabled at that time. I noticed the BI stability problem precisely because it wouldn't toggle between mainstream/substream when solo'ing and de-solo'ing the camera.
2) No Quicksync or video graphics on this CPU - but thanks for pointing out that there's specific 'tested' versions.
3) I have the '2.0.8-Beta' version as downloaded by BI itself. Seems to be the latest on the CodeProjectAI website as well.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
Did you do a fresh install of Windows using the Media Creation Tool?

Which version of Windows?

I suspect it is a problem with the computer.

40-50% CPU with one camera is way too high, especially if you are using substreams. Folks run 50 cameras on a 4th gen at 30%.

Was the RAM high?

But no BI does not need a reboot.

BI allows for anonymous update of performance data. People have had it running nonstop for over 1900 days, or 5 years... And I suspect that the last time it rebooted was when they manually did it...

1676863587159.png
 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
It was a fresh install of Windows 10 LTSC. (Wanted to try it out for this install because it's an unbloated Win10 with many years of updates left. ISO from archive.org, MD5 hash matches MSDN info.)

I forgot to look at the ram usage before rebooting, but it's sitting at 3.5/12GB right now.

When the BI GUI is not running the CPU usage is about 10%.
When the BI GUI is running the CPU usage is ~33% when viewing the substream or ~40% when viewing the main stream. (30 fps cam. 0.3MP sub and 2.1MP main)
If I enable the 'limit live preview' to 5fps, it drops to ~15% for the substream or ~20% when viewing the main stream.
The above values are when managing BI through RDP so there's a bit of an overhead on the CPU for that.

It's a CPU from 2010 so definitely not the best option for BI maybe.

--

I have another spare HP Z240 workstation that I could use for BI which has an i5-7500 with quicksync. But only 4GB of ram in there so I'd need to upgrade that and it's an SFF chassis so can't use my full size GPU in there; wanted to try to avoid using it as I would need to host CodeProjectAI externally on a separate PC and I just wanted a self contained 1 PC solution for BI :)

How are you guys managing your BI installs? Through RDP as well, or got a dedicated monitor/mouse for your BI PC?
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
Not familar with capabilities of the Xeon W3530, but that still seems to high for one camera and adding any will be difficult.

A 7th gen will run CodeProject fine as a CPU version. You only need a GPU if you are doing some heaving lifting with a lot of cameras.

But if your camera has AI use it instead? Many of us find that is more than adequate.



Many of us use RDP to manage BI.


For surveillance cameras, 15FPS is more than enough. Movies are shot at 24FPS for the big screen, so more than that isn't needed for your monitor and phone LOL.

Plus running cameras at rated specs is like running your car at redline 24/7. Camera can't keep up.

Many here have come with troubles of jerky or lagging video or other issues and when they reluctantly drop the FPS to 15 all the problems go away.

Shutter speed is more important than FPS. I catch plates on cars going 45MPH at 8FPS because shutter speed is more important.

Drop the FPS to 15 and the system usage will drop.

Sure 30FPS can provide a smoother video but no police officer has said "wow that person really is running smooth". They want the ability to freeze frame and get a clean image. So be it if the video is a little choppy....and at 10-15FPS it won't be appreciable. My neighbor runs his at 60FPS, so the person or car goes by looking smooth, but it is a blur when trying to freeze frame it because the camera can't keep up. Meanwhile my camera at 15FPS with the proper shutter speed gets the clean shots.

Watch these, for most of us, it isn't annoying until below 10FPS


 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
I've got a fresh new install of 5.7.1.3 running as a service that I started using yesterday with a single camera.
Played with some options to get to know BI and went to bed.
This morning I open it again and it's pretty laggy and behaving weird; empty logs list, takes a few seconds to redraw when changing any settings tabs, PTZ commands either don't work or take 10 seconds to execute, changing camera settings (removing substream config) didn't trigger that camera reset with the multicolor rectangles, etc.

Everything else (browser/OS) on that BI PC was working well at the time, so I rebooted it and it's now working well again, but now I'm worried about stability and worried BI could let me down precisely when I need it most.

CPU usage was 'normal' ~40-50% when in the BI GUI, none of the cores maxed out; this is an older HP Z400 workstation that I wanted to recycle as a BI machine. Xeon W3530 cpu with 12GB ram and SSD for the OS/BI and a P106-100 for CodeProjectAI. No intel Quicksync.

Are there any 'must have' settings that help with stability?
That processor has a passmark of 3000 and very weak. When properly setup blue iris can run for as long as you leave it on or until a power outage or update. We are not rebooting our systems.
 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
That processor has a passmark of 3000 and very weak. When properly setup blue iris can run for as long as you leave it on or until a power outage or update. We are not rebooting our systems.
Indeed, it's pretty weak by modern standards, but I thought I could make it work for this install where I would only ever have 2-3 cameras max.

A 7th gen will run CodeProject fine as a CPU version. You only need a GPU if you are doing some heaving lifting with a lot of cameras.
This is an interesting suggestion. I assumed I needed a GPU for BI AI but if it's able to run directly on the i5-7500 then that might be the way to go. I'll give it a try.

But if your camera has AI use it instead? Many of us find that is more than adequate.
The one I'm playing with right now is the SD49225XA-HNR PTZ, and it looks like it has some person/car detection stuff but I haven't tried to make any of it work yet. Haven't touched the auto-tracking either yet.
I always assumed the built-in camera analytics were pretty rudimentary for consumer grade cams and that newer systems like deepstack or codeprojectai would yield better accuracy/results.
In what instances would you suggest someone to stick to the camera stuff and when to move to the external ai checkers?

For surveillance cameras, 15FPS is more than enough. Movies are shot at 24FPS for the big screen, so more than that isn't needed for your monitor and phone LOL.

Plus running cameras at rated specs is like running your car at redline 24/7. Camera can't keep up.

Many here have come with troubles of jerky or lagging video or other issues and when they reluctantly drop the FPS to 15 all the problems go away.

Shutter speed is more important than FPS. I catch plates on cars going 45MPH at 8FPS because shutter speed is more important.

Drop the FPS to 15 and the system usage will drop.

Sure 30FPS can provide a smoother video but no police officer has said "wow that person really is running smooth". They want the ability to freeze frame and get a clean image. So be it if the video is a little choppy....and at 10-15FPS it won't be appreciable. My neighbor runs his at 60FPS, so the person or car goes by looking smooth, but it is a blur when trying to freeze frame it because the camera can't keep up. Meanwhile my camera at 15FPS with the proper shutter speed gets the clean shots.
Will try your 15fps suggestion to see if that helps reduce ghosting/blur in my nighttime images :)
 

Flintstone61

Known around here
Joined
Feb 4, 2020
Messages
6,587
Reaction score
10,894
Location
Minnesota USA
The Xeon does not show on Intels website to have onboard graphics. So whatever graphics it's using could be the problem.... as well as he ancient processor. the LGA 1366 does not support desktop processors, you'd have to try a really stout Nvidia card and that system probably doesn't meet the requirements to support that card.
I'd use the other PC for a try,,,,then I'd say, maybe Save an HP elitedesk fom the scrap heap from Ebay and donate your old ones.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
Before I decided to just go primarily with using the camera AI, I was running BI AI CPU version on 7 cameras with a 4th gen without too much trouble as I played and tested it.

I have since backed down on most of them and simply use the camera AI as it works for my situation. But I still run BI AI on a few cams for unique situations.

The camera AI is useful to many people, but BI has way more motion setting granularity than the cameras, and some people need that additional detail, especially if wanting AI for more than a car or person. For folks that want AI and alerts on animals or specifically a UPS truck then they need the additional AI.

There isn't really a best practice because every field of view is different and use case and needs are different.

To many here, BI motion is more than adequate for what they do and they use zero AI.

To many here, camera AI is more than adequate for what they do.

To many here, using the BI AI adds additional functionality that the above alone can not do.

It comes down to testing with each field of view and which one gives you the most consistent results and accomplishes your goals.

I just want to be alerted if a person or vehicle is on my property and the camera AI does a fine job with that.

The true test....I have found the AI of the cameras to work even in a freakin blizzard....imagine how much the CPU would be maxing out sending all the snow pictures for analysis to CodeProject LOL. My non-AI cams in BI were triggering all night. This picture was ran through Deepstack (without the IVS or red lines on it) and it failed to recognize a person in the picture, but the camera AI did. This pic says it all and the video had the red box over it even in complete white out on the screen:


1613268961041.png


While some of that third party stuff is cool like tagging was it a dog or a bear, I don't need all that fancy stuff. If my camera triggers BI to tag an alert for human or vehicle and BI can accomplish what I need by way of a text or email or push or whatever, that is sufficient for my needs.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
Remember that the number of cameras is irrelevant its the MP per second. Blue iris uses the same processing power for 1 4k camera at 30fps as it does for 8 1080p cameras at 15 fps. The other major issue with old processors is power consumption. A newer system can pay for itself in as little as a year with the high cost of electricity in most areas.
 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
When the BI GUI is not running the CPU usage is about 10%.
When the BI GUI is running the CPU usage is ~33% when viewing the substream or ~40% when viewing the main stream. (30 fps cam. 0.3MP sub and 2.1MP main)
If I enable the 'limit live preview' to 5fps, it drops to ~15% for the substream or ~20% when viewing the main stream.
The above values are when managing BI through RDP so there's a bit of an overhead on the CPU for that.
So, got BI running on that other i5-7500 box.

When the BI GUI is not running the CPU usage is about 4%.
When the BI GUI is running the CPU usage is ~50% when viewing the main stream. (15 fps cam 2.1MP main)
If I enable the 'limit live preview' to 5fps, it drops to ~20% when viewing the main stream.
The above values are when managing BI through RDP, just the mainstream, with HA is set to I+.

So some stats are better, and some are now worse, but overall I feel like, those numbers are pretttty close to the previous W3530 box; I think it actually might be 4k RDP eating up all my resources when full screen live viewing.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
That seems high for that system.

Post the screenshot of the BI Camera Status page that shows stuff like KEY, FPS, bitrate, and at the bottom shows MP/s.

Around the time AI was introduced in BI, many here had their system become unstable with hardware acceleration on (even if not using DeepStack or CodeProject). Some have also been fine. I started to see that error when I was using hardware acceleration several updates into when AI was added.

This hits everyone at a different point. Some had their system go wonky immediately, some it was after a specific update, and some still don't have a problem, yet the trend is showing running hardware acceleration will result in a problem at some point.

However, with substreams being introduced, the CPU% needed to offload video to a GPU (internal or external) is more than the CPU% savings seen by offloading to a GPU. Especially after about 12 cameras, the CPU goes up by using hardware acceleration.

My CPU % went down by not using hardware acceleration. But if you use it, use plain intel.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
So, got BI running on that other i5-7500 box.

When the BI GUI is not running the CPU usage is about 4%.
When the BI GUI is running the CPU usage is ~50% when viewing the main stream. (15 fps cam 2.1MP main)
If I enable the 'limit live preview' to 5fps, it drops to ~20% when viewing the main stream.
The above values are when managing BI through RDP, just the mainstream, with HA is set to I+.

So some stats are better, and some are now worse, but overall I feel like, those numbers are pretttty close to the previous W3530 box; I think it actually might be 4k RDP eating up all my resources when full screen live viewing.
Something is wrong with your setup. Log in with a monitor.
 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
1679801213304.png
1679801280322.png

Yea, it is Remote Desktop :(

If I login directly it behaves much better;

1679802549635.png
1679802581099.png

When live viewing with the web UI3;

1679802756046.png

So looks like I'll need to do most of the viewing with the web UI if I want to watch at the recorded 15fps, and limit RDP to admin tasks.

I'm curious to see how it would behave using that remote administration feature built in BI, the one where you need an extra license.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
You are not using substreams for one, but it still should not be that high for one camera.

But you have something else going on.

Folks run 4th generation CPU with over 30 cameras using RDP and have lower CPU than you.

Have you changed things in BI thinking it would improve performance without knowing what it does?

Have you excluded anti-virus?

We mentioned it before - turn off hardware acceleration.

Have you followed every step in the wiki?

 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
The BI settings changes that I've done were all from the wiki, yes; direct to disk is set, limited to 15fps, BI is running as a service, overlays are disabled.
The only exception is I haven't enabled substreams yet on this new i5-7500 box.

If I limit my RDP resolution to 1080p, the total CPU usage hovers around 15% at the full 15fps live view.

1679805663310.png

But anything higher, and especially 4k and I'm back to ~50% cpu usage.

I'm guessing those who you are mentioning were on 1080p monitors?

---

1679806158592.png

I forced it to use the GPU for RDP encoding but that didn't have any impact. I'll search if there's any other RDP optimizations that I could do.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,428
Reaction score
47,544
Location
USA
I am one of those 4th gen users with a 4K display and I have not restricted RDP. You have something else going on.

 

Webfont

Pulling my weight
Joined
Sep 6, 2018
Messages
138
Reaction score
179
Location
Canada
Humm. I just tried Parsec and it's able to serve the 4k screen on a total of 24% CPU. Much better than RDP with it's ~57%.

1679844128882.png

But I don't want to use Parsec :( Would prefer to stick to RDP so I can lock down access to local systems only and not expose to the internet.

I'm really curious how you guys are able to make it work with RDP. Did you have to make any extra tweaks?
I tried playing a 4k youtube video via rdp and it's all choppy and laggy and CPU at ~70%.

And if I disable hardware acceleration for my single 2MP cam, BI then needs ~40% CPU to itself.

1679844790940.png

How on earth can you run 30 cams with no HA with a lower CPU usage on your 4th gen cpu. Are you running a specific version of BI?

I need to start over, this doesn't make sense :p
 
Top