Windows 10 runs out of RAM

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
After ~1 week of uptime, my windows 10 system has started running out of memory. It doesn't look like Blue Iris is the culprit as its memory footprint stays relatively stable at 3.5GB of use. In fact Windows Task Manager just shows a red box around the memory column, and although Blue Iris is the top consumer, the rest of the processes don't add up to anywhere near the total memory of the system. When the system gets to this point it can take 5 minutes for Task Manager to come up, and i have never managed to get performance manager to show the system is so bogged (CPU isn't pegged).

Curious if anyone has tips on diagnosing a memory issue like this; is this the dreaded Intel memory leak? Its an old Devil's Canyon processor.
 

mat200

IPCT Contributor
Joined
Jan 17, 2017
Messages
13,668
Reaction score
22,771
After ~1 week of uptime, my windows 10 system has started running out of memory. It doesn't look like Blue Iris is the culprit as its memory footprint stays relatively stable at 3.5GB of use. In fact Windows Task Manager just shows a red box around the memory column, and although Blue Iris is the top consumer, the rest of the processes don't add up to anywhere near the total memory of the system. When the system gets to this point it can take 5 minutes for Task Manager to come up, and i have never managed to get performance manager to show the system is so bogged (CPU isn't pegged).

Curious if anyone has tips on diagnosing a memory issue like this; is this the dreaded Intel memory leak? Its an old Devil's Canyon processor.
Hi @Tygunn

Is the PC doing any other work?

Normally the quick fix is buying more memory, as tracking these leaks down can be time consuming and often buying more memory gives you a bit more time to figure it out imho.
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
Hi @Tygunn

Is the PC doing any other work?

Normally the quick fix is buying more memory, as tracking these leaks down can be time consuming and often buying more memory gives you a bit more time to figure it out imho.
The PC is dedicated to BlueIris. Its only other work is running the utility that changes camera profile at sunrise/sunset. I think my biggest problem right now is I don't know how to go about figuring out what is causing the memory leak. Task Manager and performance monitor in Windows don't seem to show which app is sucking up the memory. I've not found a definitive "better" tool for diagnosis. The other problem is when the issue occurs the computer bogs to the point I can't use it at all. I know its happening because Blue Iris loses connection to all the cameras.
 

mat200

IPCT Contributor
Joined
Jan 17, 2017
Messages
13,668
Reaction score
22,771
The PC is dedicated to BlueIris. Its only other work is running the utility that changes camera profile at sunrise/sunset. I think my biggest problem right now is I don't know how to go about figuring out what is causing the memory leak. Task Manager and performance monitor in Windows don't seem to show which app is sucking up the memory. I've not found a definitive "better" tool for diagnosis. The other problem is when the issue occurs the computer bogs to the point I can't use it at all. I know its happening because Blue Iris loses connection to all the cameras.
Hi @Tygunn

Yeah, once a system goes into swap you're in trouble... thus why SSDs and extra RAM are my "cheap" solutions when I am short on time to track down problems. ( cheap is relative to say a tech's time in a company )

Try this out:
ttps://windowsreport.com/memory-leak-windows-10/#6
 

bp2008

Staff member
Joined
Mar 10, 2014
Messages
12,666
Reaction score
14,006
Location
USA
Leave task manager open on the Performance tab so you can keep an eye on the CPU and memory graphs. This should let you easily see at a glance if any of them are slowly rising so you can investigate further before the system starts going unresponsive.
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
1) What are your system specs, CPU type/model, memory, disk drives.
2) On your windows 10 do you allow for windows update ? Have disabled windows updates.
3) Do you use GPU hardware acceleration for video processing ?
4) What is your driver version ?
see wiki , Memory Leak: Quick Sync (Hardware Acceleration) | IP Cam Talk
1) Intel Core i5-4690K (Haswell) Processor 3.5 GHz, 256GB SSH boot disk, WD Purple drive for active read/write, 16GB ram
2) I have updates disabled; it'll download them and ask me to install them.
3) I use the Intel hardware acceleration for video processing; I have to otherwise the machine can't handle the 12 cameras its got running.
4) I'm not sure on the driver version; perhaps I'll have to try rolling it back from what it is. Maybe it updated recently and when the power failed it installed the update.
 

SouthernYankee

IPCT Contributor
Joined
Feb 15, 2018
Messages
5,170
Reaction score
5,320
Location
Houston Tx
Read in detail the write up on the correct driver, this may be your problem. Some of the drives leak.
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
Read in detail the write up on the correct driver, this may be your problem. Some of the drives leak.
Haswell isn't listed. I'll have to experiment and see if I can roll this back.
 

mat200

IPCT Contributor
Joined
Jan 17, 2017
Messages
13,668
Reaction score
22,771
1) Intel Core i5-4690K (Haswell) Processor 3.5 GHz, 256GB SSH boot disk, WD Purple drive for active read/write, 16GB ram
2) I have updates disabled; it'll download them and ask me to install them.
3) I use the Intel hardware acceleration for video processing; I have to otherwise the machine can't handle the 12 cameras its got running.
4) I'm not sure on the driver version; perhaps I'll have to try rolling it back from what it is. Maybe it updated recently and when the power failed it installed the update.
16GB of RAM - system should be doing very well w/ that amount imho

hmmm... try this recommendation from MS to see if you can ID the leak ( this is for "user space" memory leaks, the kernel can also have memory leaks )
Using Performance Monitor to Find a User-Mode Memory Leak - Windows drivers
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
It appears that by continually observing memory usage with Performance Monitor I'm no longer encountering this leak... Of course. Haha.
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
Looks like the problem is back.

BlueIris is only using 4GB,and the rest don't add up to the fact that most of the system memory is in use.
upload_2019-6-7_6-59-30.png

Rammap seems to indicate its the non-paged pool, which from what I can determine means leaked memory; can't find a way to attribute this thus far.
upload_2019-6-7_7-0-22.png

Windows update history shows an intel graphics update happened in January. Seems odd I'd be seeing the issues now and not earlier.

...and this is why I wish BI ram on another platform. I forgot how much Windows annoys me.
 

morrisky

Getting the hang of it
Joined
Jun 30, 2017
Messages
95
Reaction score
46
Try sorting by Threads in Task Manager on the Details tab. See if anything has an abnormally high thread count maybe.
 

SouthernYankee

IPCT Contributor
Joined
Feb 15, 2018
Messages
5,170
Reaction score
5,320
Location
Houston Tx
Why is there chrome open on the bi PC?

Have you read the post listed above ?

Your drive is more than likely leaking ?

Another suggestion, turn off hard ware acceleration and see if it leaks .
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
Why is there chrome open on the bi PC?

Have you read the post listed above ?

Your drive is more than likely leaking ?

Another suggestion, turn off hard ware acceleration and see if it leaks .
Chrome was open when I took the screenshot because I was in the midst of searching for "solving Windows 10 memory leaks". :) Otherwise its just BI running.

I have read the post listed above; it mentions downgrading versions of the video drivers to certain well known states. Looks like the last known "good" video drivers was updated for my CPU. Yay. It used to be empty. I had planned on just going to the last version from what I've got now. Looks like I now have a known "good" state to start from. Crossing my fingers that it works. :)
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
Hmm, looks like the latest known stable Intel drivers are even worse; went from 40% to 67% CPU usage overnight. I guess I'll have to keep on trying other versions. :)

I've been considering a new BI build; I wonder if using the NVIDIA decoders would be more reliable...

Oh, and I considered turning off hardware acceleration, but then the computer would melt down into a pile of slag. It can't handle 12 cameras without the GPU acceleration.
 

Tygunn

Getting comfortable
Joined
Dec 24, 2016
Messages
390
Reaction score
312
FYI the following two firmwares are bunk; still getting terrible memory leaks which cripple the system in 1-2 days:

4th Generation (Haswell) / 5th Generation (Broadwell)
Driver Version 15.40.41.5058 is known to be NOT good: <Download Intel® Graphics Driver for Windows* [15.40]>
Driver Version 15.40.42.5063 is known to be NOT good: <Download Intel® Graphics Driver for Windows* [15.40]>

I'll keep downgrading to see if I can get to a good known state.

I'm glad I have a backup Dahua NVR to ensure I have consistent covering with how terribly flaky BI has been.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
FYI the following two firmwares are bunk; still getting terrible memory leaks which cripple the system in 1-2 days:

4th Generation (Haswell) / 5th Generation (Broadwell)
Driver Version 15.40.41.5058 is known to be NOT good: <Download Intel® Graphics Driver for Windows* [15.40]>
Driver Version 15.40.42.5063 is known to be NOT good: <Download Intel® Graphics Driver for Windows* [15.40]>

I'll keep downgrading to see if I can get to a good known state.

I'm glad I have a backup Dahua NVR to ensure I have consistent covering with how terribly flaky BI has been.
Your issue is likely not related to the driver particularly if you tried the known good drivers listed in the wiki. I personally have never seen a 4th gen haswell suffer from a memory leak.
Your issue is also not with blue iris. Blue iris itself does not suffer from a memory leak. Perhaps you are better off with the dahua NVR.
 
Top