Camera feed slowing down to 1-6fps

Z

zylogistik

n3wb
Feb 21, 2020
7
1
Hello there,

I recently installed my first BI5 environment. Sadly I do have some problems which I search help to fix.

Right after startup all 24 cameras (2-3MP) provide 29-30fps but after some seconds it dives down to around 1-10fps per camera. I provided a screenshot of the ressourcemanager as well as the camera tab of BI.

BI_Screen.png

I thought it could be a CPU bottleneck. So I disabled all but 8 cameras. CPU utilization is around 40-50% (with the 8).
I tried to diable recording and motion detection. No difference, not even a change in utilization beyond 5%.

I thought it could be a harddrive bottleneck, so I moved the New folder and database to an SSD. No difference.
I thought it could be a network bottleck, so I installed a 10Gbit card. As you can see in the screenshot, the network utilization switches between 16-60mbit, like up and down. No difference.

Is there anything coming to mind, which could cause this issue?
 
May not be relevant in your case but I had exactly this issue with a couple of Hikvision DS-2DE2A404IW-DE3 PTZ cams while my other PTZs maintained the set frame rate. Turned out to be an obscure setting I had enabled called Storage Lite which saves space when recording to SD cards by reducing the frame rate until or unless motion is detected. Trouble is, it's a buggy feature as once the frame rate dropped down, it would never recover even for motion, and the live view was then permanently restricted to between 1 and 8fps [vs the 25fps setting], whatever was or wasn't happening in the field of view. After disabling the Storage Lite feature, the cameras never dropped below 25fps.
 
What kind, brand, of cameras?
Why 30FPs, are you making movies?
What kind of CPU?
How much memory?
Have you looked in the WiKi and read "Optimizing Blue Iris CPU Useage"?

Too little information in the initial post to even guess.
 
Hey there thanks for your response.

20 cameras are Trendnet IP311 with hikvision firmware on them.
4 are foscam cameras Some FI99 dont have the exact modelname on hand.

I set it up with 30fps because the QNAP NAS which this BI5 Instance replaces had 30fps set. It does not mean that I am going to keep it that way. But I already tried changing it to 15fps. Same thing happens at first they are at 15fps then they go down and the video is almost like slowmotion. It tries to shows all frames but after it delayed the feed about 5-10 Minutes it jumps back to live and starts all over.

The System does have 2x Xeon-E5-2660 CPUs and 64GB RAM. (Please don't comment on how bad using old gen server hardware is, I had it on hand and did pay nothing for it)

I did read the wiki and the optimization page, but that doesn't help if I have the same problem with 1 or 24 cameras as the CPU should be plenty powerfull to watch a single video stream even if I don't optimize anything.
 
Hi @zylogistik. Your screenshot very clearly shows the CPU is pegged at 100% (good job with the screenshot, by the way!).

That indicates you have overloaded the system. Unlike many video recording systems, Blue Iris continuously decodes all the incoming video which gives it much bigger hardware requirements. Blue Iris cannot run properly with the CPU at 100%. Blue Iris needs to do all its work in real-time which it cannot do if the CPU is maxed out.

You need the steady-state CPU load to be around 50-60% or less for proper operation so that there is some headroom for other tasks like playing recordings, remote viewing, exporting clips, and installing updates. In this case, that will probably require reducing your frame rates across the board to about 10 FPS. Maybe even lower. This is in addition to all the other steps in the optimizing wiki, most especially direct-to-disk. Intel hardware acceleration isn't available on your system. If you can live with the "Limit decoding" setting on some of your cameras, that should save enough CPU that you can run with higher frame rates.

To run with fewer compromises, you will need a better computer. You could limp along by adding Nvidia graphics cards to the current one for hardware acceleration, but that would be expensive and very energy-inefficient, possibly overwhelming the server's power supply.

I've done a little math for you. 24 cameras at an average resolution of 2.5 megapixels, all at 30 FPS, this makes 1800 MP/s (megapixels per second). Based on my tests, 1800 MP/s is very close to exhausting the memory bandwidth available with dual channel DDR4-3600 on a latest-generation AMD platform. If you want to run all those cameras at 30 FPS then I recommend an HEDT system with quad-channel memory, running a latest-generation CPU of at least 10 cores. If that is too expensive, then go with an i7-8700K or i9-9900K system and start with 15 FPS. If desired, you can then increase the frame rates until you get to about 50-60% overall CPU usage with no remote viewing clients connected. You probably won't get to 30 FPS, but 20-25 should be achievable.
 
  • Like
Reactions: sebastiantombs and fenderman
Blue Iris does not support multiple CPUs.

Also describe your network. Does the camera traffic pass thru the Router.
 
Hey thanks for all of your responses.

I wanted to make a screenshot of the utilization with only one active camera, as I thought it would be impossible that viewing one camera can saturate the cpu.
Before I did that I made an update to 5.2.4.4 (which dates to 4th April instead of 3rd April as stated on the Blueiris Website) I don't know which version was installed before. I downloaded and installed BI yesterday.

Look what happened:

BI_Screen_2.png

All cores on both CPUs are in use, as they where before (I think BI can in fact utilize all cores even on dual socket systems at least in my setup), but the utilization dropped to 50 % with the Console open and around 40-45% when it is closed. UI3 works like a charm now even showing around 28-30fps if not limited, when before it was like 2-3. PERFECT!

CPU usage went up to around 60% when the console is closed but the UI is in use.


Many thanks for all of your help. I think the problem was the combination of BI Version and my system which had difficulties.
 
  • Like
Reactions: sebastiantombs
I wanted to come back to this topic as it is still current for me. I emaild BI Support but lets say that did not help, last thing I heard was "I will look into it".

To this day I was able to get it into working condition, like in my previous post, by randomly applying BI update packages till it runs at around 45% cpu utilization. It would stay like this till you need to restart or like today BI decides to die (First time it died since I use it. It lost connection to a camera which it seemed to be emotionally attached to and decided thats it.)

I am beyond my knowledge, the hardware can't be the problem as it is working some times and it works PERFECT if it works. Remains this to be a software issue which I can't solve on my own.
I think I have to look for something else then BI which is sad as it is perfect for our use case.
 
zylogistik said:
I wanted to come back to this topic as it is still current for me. I emaild BI Support but lets say that did not help, last thing I heard was "I will look into it".

To this day I was able to get it into working condition, like in my previous post, by randomly applying BI update packages till it runs at around 45% cpu utilization. It would stay like this till you need to restart or like today BI decides to die (First time it died since I use it. It lost connection to a camera which it seemed to be emotionally attached to and decided thats it.)

I am beyond my knowledge, the hardware can't be the problem as it is working some times and it works PERFECT if it works. Remains this to be a software issue which I can't solve on my own.
I think I have to look for something else then BI which is sad as it is perfect for our use case.
Click to expand...
It’s not the software or lots of others would have the same issue. Clean install windows and exclude bi program and storage folders from anti virus. Also you should be using 5.2.6.5 nothing newer.
 
bp2008 said:
Hey @zylogistik can you show a screenshot of a camera's video encoding settings?
Click to expand...
You mean of the camera itself?

Anmerkung 2020-06-20 082533.png

fenderman said:
It’s not the software or lots of others would have the same issue. Clean install windows and exclude bi program and storage folders from anti virus. Also you should be using 5.2.6.5 nothing newer.
Click to expand...

But the hardware isn't changing between installing the same update packages. There is noting on the system besides a Intel Networking Driver package and BI nothing else. Tried 5.2.6.5 again, but it is not more stable than the other versions. Currently it is working fine. Good for me for using server hardware behind an USV because I can basically run it till it dies on it's own. But that is what happend yesterday so I needed to look into it again.

It is possible that it is software related but not BI but how can I know. I might need a push in the right direction to find my problem.



BTW: This is my system, with the previous posts I got the feeling people think that the hardware is not fast enough. But here it is handling 1400MP/s with around 40% utilization.

1592635957891.png
 
zylogistik said:
Already did that(multiple times) as stated above there is nothing installed beside BI and a necessary networking driver.
Click to expand...
How is your network setup. Is data being passed through your router?
Are you running the demo?
 
Last edited:
fenderman said:
How is your network setup. Is data being passed through your router?
Are you running the demo?
Click to expand...

The cameras are connected to a 24 port POE Switch. The camera ports are all 100mbit. The switch does have 4 uplink connections which are 1gbit. 2 of those are connected to the system running BI (LACP). One is connected to the router for DHCP(cameras do have fixed IPs), DNS and internet connection. The system was connected with a single 1gbit connection which made no difference. The problem even occures with only 10 running (not even recording) cameras.

I do have a license active in BI.
 
You must log in or register to reply here.
Top Bottom