Memory creeping on Intel Graphics 530

[SouthernYankee]

@elektro255
What OS are you running and what tools are you using to generate the graphs.

Are you running virus scanner on bi folders ?

 

[looney2ns]

I do not think 526MB is high. Since BI V4 my system has used over 4000MB and under 5000MB for 14 cameras 700MP/s. As long as the memory is not growing every hour, every thing is OK. You have Task manager on the screen that is updating ever second, this is using lot graphics power. you are using 77% of the system memory, how much memory do you have ?

This is after 48 hrs of running. No, the BI memory is not high, but the overall system at 77% will continue to grow until it becomes unresponsive at 90%.
Task manager makes very little difference when open or closed, when looking at memory usage in other manners, such as UI3.
After a fresh reboot, System total memory will be 30%.
System has 8gb of ram.
System is running 41MB/s

 

[looney2ns]

DWM is the process that manages the Windows desktop and GUI and can make of hardware acceleration if available.

Any reason for running both TeamViewer and Ultraview? Or is your friend onto the “Amazon Support” team getting his refund

Ultra Viewer uses much less system resources when in use. At times, I can connect with it when I can't with TeamViewer due to system high memory usage.

 

[IAmATeaf]

I’d personally stick another 4gb stick, I originally ran my system with 8 but adding another 4 made such a difference. Currently have 16gb but that extra 4 had little to no impact.

 

[elektro255]

@elektro255
What OS are you running and what tools are you using to generate the graphs.

Are you running virus scanner on bi folders ?

Windows 10 for Workstations 10.0.19041 Build 19041. I use Telegraf that writes the stats into InfluxDB and Grafana to render the graphs.
I don't have a virus scanner on BI folders, it wouldn't matter since RAM usage is coming from the BI process, not the virus software.

It is leaking memory again 12GB now, it is interesting that the last 2 leaks started exactly at 2:25 pm PST I wonder if there any scheduled job at that time.

Looks like BlueIris isn't releasing some dynamicly allocated memory on heap:


The leaked memory doesn't look like any video buffers, it looks like some BI internal structures:

 

[wittaj]

DB repair maybe at 2:25?

 

[elektro255]

DB repair is at 2am

 

[wittaj]

Bummer - would be nice if that pinpointed it!

I changed mine to the afternoon since it doesn't record during DB repair, so thought maybe you did that as well.

 

[IAmATeaf]

Maybe it’s associated with when DB repair completes and not starts?

Like others here I only run a DB repair/rebuild once a week on Sunday afternoon.

 

[burgler2112]

Ok it's a 3 step process but quick. 1st step is really a system check and can be skipped or run alone without step 2 or 3. However, if your run step 2, either after step 1 or without step 1, you must run step 3.

Hope that makes sense. Step 1 is a more comprehensive version of the system check utility and sometimes fixes items the Windows run one can't. However, it's doesn't fix everything, and whilst worth trying, sometimes only a re-installation will do.

THANK YOU for this!!! Has made a huge improvement on my blue iris pc. Very grateful.

 

[elektro255]

My memory started leaking at 2:25pm again. I've checked Windows Events, Scheduler, and antivirus and nothing was running at this time so looks like BlueIris specific.
I was able to find the exact location in BlueIris code and forwarded that information to Ken, hopefully, we will see a fix soon. For now I'll schedule service restart at that time.

 

[elektro255]

I've looked into it deeper and in my case, it is related to audio resampling I have "Use FLAC audio compression" and after 14h 25m the audio stops recording, and the leak starts, the video is fine (I'm using 24h clips).
So I assume possible workarounds for now are:

• uncheck "Use FLAC audio compression"
• disable audio streams
• use clips shorter than 14h

 

[CCTVCam]

It definately sounds as if BI is failing to release memory after viewing a clip. It's good it's no longer creeping up on it's own, but is still problematic if it's creeping up due to failing to release after viewing as viewing is after all essential and will eventually lead to a crash if cumulative.

I'd suggest this is still an issue. Also a user earlier (apologies I lost the thread number) reported around 16GB of memory in use when viewing all the clips. This sounds a lot to me. 16 x substreams + the interface. I see 353mb for the interface alone with no cameras showing / imported / active. So 16x the sub streams = 15.7GB? Seems a lot. There may be some usage for the main streams but if not encoding it's just directing them to disc, if re-encoding then there's the processing and writing to disc. The latters going to be more memory intensive than the former. However, QuickSync has the reputation of being very efficient so I'm not seeing the huge memory usage as I believe it is able to encode on the fly even for multiple streams in which case surely there should be no memory usage as the videos not getting stored before being processed. No expert on this but 16gb sounds way high to me. I don't even see that when encoding video from a video editor and a video editor is both writing at much higher quality and encoding in effects, transitions and other manipulations committed to memory etc at the same time, even multiple timelines (streams at high quality). Video editing seem more CPU intensive in my experience.

@Looney, yes around 90% you'd expect a lock up. 10% of 8gb is 800mb which is what I'd expect Win 10 would require as a minimum to run. Win is running out of memory due to it being possibly in use by BI.

 

[VorlonFrog]

Interesting development this morning (good news, actually). Went into Blue Iris desktop via RDP. Memory usage was at 5.8 GB, or about .3GB lower than previous. I went in to adjust the motion detection zones on a few cameras, and this apparently triggered the memory cleanup. When I was done editing four cameras, memory usage was at 5.0 GB.

 

[SouthernYankee]

well my system just went to hell
Windows 10 PRO 10.0.1.19042 build 19042
BI 5.3.9.2
HP Elitedesk i7-4790
physical memory 16 GB
Virtual memory 48 GB
intel 4600 graphics - BI processing only - Intel driver 29.19.15.5126
NVIDIA Geforce 710 - display only


When I upgraded from BI 5.3.7.4 to BI 5.3.9.2 My system jumped to 14 GB used with a CPU at 100%. I did a shut down and a power down reboot. With in 15 minutes it was back to 100% CPU and 14 GB memory. Looking at the log file i had an errors "AddToBIDB failed 1" also "Clip: write error 80000020, undefined" AND "Clip: Cannot append; possilby corrupt"

So i disabled all the cameras, delete all the files in the DB folder, rebooted the BI PC. So BI had nothing to record. It just needed to rebuild the complete database, the rebuild took a little over 3 hours, for adding 4440 files.

BI memory used
0 cameras enable - 66 MB stable
enabled 4 cameras, 3 of which had sub streams, after 2.5 hours memory increased to 9.26 GB and continues to increase.

 

[IAmATeaf]

Maybe it was rebuilding the DB after the update?

 

[SouthernYankee]

I let it rebuild the DB without any activity, then turned on 4 cameras and the memory went to over 9 GB. On the earlier version 5.3.7.4 it never got above 5.9 GB with 14 cameras in operation. I am reporting this to KEN.

KEN need to add some memory debugging to a special log file so some of us could track the memory activity.

 

[elektro255]

You can debug memory issues with the standard windows debugging tools, steps are:
1. Install tools: Download Debugging Tools for Windows - WinDbg - Windows drivers (you only need one checkbox selected when installing)
2. Enable memory tracing for BlueIris.exe
2.1 Open cmd.exe as administrator
2.2 Execute command:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" /i BlueIris.exe +ust

3. Restart BlueIris
4. Take a memory trace before memory leak:
4.1 Find out BlueIris process id (PID) (go to Task Manager->Details it is second column)
4.2 Execute command on admin cmd:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\umdh.exe" -p:12345 -f:log1.txt

replace 12345 with BI PID
5. Wait for memory to leak
6. Take a second memory trace dump:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\umdh.exe" -p:12345 -f:log2.txt

7. Compare files to find out what addresses were allocated and not released:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\umdh.exe" log1.txt log2.txt > logcompare.txt

You can open logcompare.txt in

https://github.com/Nettention/UmdhViz/releases/download/1.00/UmdhViz.exe

:
It will something like:

It means that in time between steps 4 and 6 code at address 140F6D23E BlueIris.exe allocated 216MB of RAM that was not released, forward this information to Ken, using this address he can find the exact line of code that is causing the problem.


After you are done disable memory tracing for BI and restart BI:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" /i BlueIris.exe -ust

 

[gmpanazzolo]

Since creating my setup in late Dec 2020 I have been trying to work out why my BI memory usage was gradually ticking up from restart.

It would begin around the 900MB mark, and then gradually work its way up to about 4.5GB before I'd restart it again due to irritation! Interacting with the live views via UI3 or the BI console would cause an immediate increase in memory usage which would not drop down again once everything was closed. The only fix was restart and it was temporary.

My setup is:
Dell Optiplex 7040 SFF i7-6700.
16GB RAM.
HA = Intel+VPP
Intel(R) Graphics 530 - Driver Version 27.20.100.9168
9x 4MP HIKVISION ColorVu cams (approx 7,200 kB/s total).
10 sec pre-record on 8 of the cameras.

I was running BI 5.3.8.17 and I was getting the behaviour above. Last night I upgraded to 5.3.9.2 and now I notice that after restart BI's memory usage starts at about 950MB and stays there. If I open UI3 or the BI console and muck around with live views etc., memory usage goes up to about 1.3GB, but it drops down again when I close the interfaces.

So far so good!

 

[SouthernYankee]

@gmpanazzolo
4.5 GB of memory for blue iris is NOT a lot of memory, it depends on your system configuration. I have 14 cameras at about 700 mp/s. my system runs at 4.5GB to 5.5GB since I first install BI over 2 years ago. I use this much memory even when on BI 4. The memory has increased a little with the implementation do to sub streams, and other more advanced features.

A very simple understanding of memory, there are two major groups of memory for a program / system, there is memory allocated to the program and system memory. Memory is allocated in blocks, a block is about 4,096 bytes in size. As a program needs more memory, it looks to see if there is free memory in a local cache, if not then it requests multiple blocks of memory from the system. When the program is finished using the memory, it keeps the memory in a local (in the program) cache. So the next time it needs the memory, it get the memory from the local cache. Most programs seldom return the memory to the system. If a program needs the memory once it more than likely needs it again. In the back ground of a program it may check every once in a while to see if there is memory that can be returned to the system.

A memory leak is when the program asks for memory from the local cache, then when it is done using the memory it does not put it back into the local cache, then asks for a new block of memory to do the same job. So the program keeps getting bigger in memory.

A windows PC can use more memory than it actually has, so it is easy for a system to be using 32 GB of memory when it only has 16 GM of physical memory, this is what the system page file is used for, the memory blocks are stored on disk and when the system need the data in the memory it read it from the page file. A system when in high memory demand can put whole programs that are running on to disk. This is one of the reasons that as physical memory is in short supply the CPU goes up. The system is using the CPU to move memory around.

This is a very simple description of memory management, there a whole books written on memory management.