Blue Iris computer poof in the middle of the night - any ideas what happened?

[ipcdal]

Thanks guys, as for ghosts, I honestly wouldn't be surprised at this point.

But in all seriousness, I did check, double-check, and triple-check the hard drives, files, filesystems, headroom, etc., database (it's all green, but just ran compact/repair -- still green -- and then for the heck of it I ran repair/regenerate... still green). And as mentioned I've gone the extra mile on the hardware testing/diag, etc...

Now I had not thought of removing the anti-virus before. That's a new one. I'm running Bitdefender Total Security, and it's never given me problems before (that I know of, the most famous caveat of all), BUT I am willing to give it a shot.

Thanks again.

So I'll uninstall Bitdefender, see how it goes...

Hope you all have a great day. Appreciate the help and feedback.

 

[mattp]

I remember seeing something about it before:
Here's from the wiki:

Optimizing Blue Iris's CPU Usage

Blue Iris uses a lot of CPU time. In most Blue Iris installations, your CPU is the limiting factor which determines how many cameras you can have. As such, it is extremely important to optimize your configuration to ensure you can get the most out of your system while consuming the least...

ipcamtalk.com

You may not have to disable it, you can just add an exclusion, like described in the link above.

 

[ipcdal]

I'll take a look. My CPU usage has been pretty good though, around 30%-ish average. But can't hurt to review those settings.

Also, just uninstalled Bitdefender.

 

[wittaj]

Remember though that unfortunately all the tests coming back fine doesn't mean there isn't a problem.

I had a similar bought a couple years ago and all the tests came back fine but when I opened up the computer to see if dust was collecting around the fan, somehow one of the RAM cards was loose. Reseated it and it has been fine since. The RAM tested fine with it loose.

Post a screenshot of your BI Camera status page - the one that shows cameras, IPs, bitrate, KEY, and at the bottom is MP/s. 30% for an 8500 seems high so maybe something else is going on as mentioned above like defrag or anitvirus or something that then when BI is heaving processing something it spazes out.

 

[Tinman]

Edit to add:
Speaking of defrag, check to see and make sure no tasks are occurring on a schedule at that time...especially, if you have it scheduled to defrag the hard drive, I could see BI choking.

I myself never for years have never defragged by BI data drives. I would think it would be almost impossible unless you stopped all recordings. Even though windows reports them fragmented I exclude them. I have never had any issues going this route. I do every now and then when I am sure nothing important will be lost just quick format them and start over to get a fresh start and new database.

 

[wpiman]

I had this issue last year.

I remember I tracked it down to windows or windows defending trying to force an update. I followed all of the settings for disabling it in windows 10 but there was one thing I had to add.

machine has been stable now for a while.

I’d love to be able to run this app in docker on a Linux box.

 

[Flintstone61]

I'd put money on a driver. most likely an intel graphics driver. That is the most common I've seen on the forums.
the other most common issue can be running the latest versions of BI. im running an 8th Gen Intel on a Dell.

 

[Flintstone61]

 

[ipcdal]

Thanks guys! All helpful comments! Really appreciate it!

BTW, wittaj, I looked again at my CPU usage and it's actually sitting at 22%. I had previously said 30%-ish but I must have been looking at it when it was consistently doing more stuff. Just staring at it now, it does fluctuate up to a peak of 30-32%, but currently it's somewhat steady around 20-24%.

As for looking at the Intel driver as suggested by Flinstone61, I'm hoping you are indeed on to something. When I installed Windows (fairly recently), I just let it install the default drivers for everything, ran update a few times, rebooted, updated, etc., you know the drill, until it really was up to date, and I left it at that. And just now I force-checked again, and it still says that it is up to date, and the Intel UHD Graphics 630 driver is at 30.0.100.9864.

But yours is at 30.0.101.1692....

So I looked inside Windows update in the "optional updates" section and clicked on driver updates, and lo and behold there's version 30.0.101.1692 sitting there. Windows never installed it previously, I assume because it was labeled as "optional," and I didn't think to look there. I just "trusted" Windows to properly update itself.

In any case, I just installed it, rebooted, now crossing fingers.

Wouldn't it be hilarious if this is all it took. One dumb "optional" driver. I really hope this is it.

So let's see what happens tonight now that I'm running 30.0.101.1692....

Thanks again for everyone's input. Feeling a little more optimistic.

 

[ipcdal]

P.S. As an added bonus after installing Intel driver 30.0.101.1692, my CPU usage has dropped a little bit to a fairly steady 19-21%.

 

[wittaj]

Yep a bad driver can certainly cause it. Causes a memory leak and it increases until it restarts.

 

[ipcdal]

Quick update that there were no crashes last night. System holding steady. Let's see how this weekend goes. Thanks again for the all the help here guys.

 

[wpiman]

Quick update that there were no crashes last night. System holding steady. Let's see how this weekend goes. Thanks again for the all the help here guys.

2 days of no posts? Sounds good.

 

[ipcdal]

2 days of no posts? Sounds good.

Haha, yep, I was waiting for the weekend to pass.

So the update is that the system has remained stable, thank goodness!

I think the main takeaways for anyone stumbling on this thread in the future, are IMO:

1. Look for advice and feedback in this forum!
2. Don't give up!
3. Thoroughly check your hardware!
4. In my case, it was most likely a driver update!
5. It may have been a conflict with my antivirus (which I removed, but I can't be 100% certain that actually helped since I did other things at the same time)
6. Spend some time optimizing your system by checking these tips: Optimizing Blue Iris's CPU Usage

Cheers, and thanks again to the folks here. So far so good. If things go south again, I'll post another update.

 

[ipcdal]

Quick update - I've had two crashes back to back and a corrupt database after updating to 5.7.7.12 (released June 15). That is the only change I made.

Thought it would be useful to post the ongoing saga for other people in the future who might be searching for help on a similar situation.

Note to self: Updating BI can potentially lead to crashes, be cautious about updating!

I "repaired/regenerated" the database, and it has been stable for one night so far.

But I honestly can't have such ongoing instability with a system that is so important. I may roll back to the "highly stable update" 5.7.6.8, and then if that still doesn't work, something more drastic. Will update again if there is another problem.

 

[wittaj]

That is why you do as most of us - DO NOT UPDATE until BI adds a feature you need or want.

I am still on a version over a year old because DeepStack flat out works for me and seeing all the horror stories of people trying to use CodeProject just isn't worth it to me.

There are a few nice things that have been added since then, but not enough I want to mess with a perfectly stable, great working system.

 

[ipcdal]

That is why you do as most of us - DO NOT UPDATE until BI adds a feature you need or want.

I am still on a version over a year old...

Noted, and I appreciate your feedback as always. But this is very frustrating, assuming this is the cause of the latest round of instability. I come from an IT background where security patches are very important to apply PLUS how is it that BI has a reputation for lousy patches where people are afraid to patch their systems! Seems like with software like this, updates from the developer should be few and far in between... but I see notifications all the time!

 

[wittaj]

Noted, and I appreciate your feedback as always. But this is very frustrating, assuming this is the cause of the latest round of instability. I come from an IT background where security patches are very important to apply PLUS how is it that BI has a reputation for lousy patches where people are afraid to patch their systems! Seems like with software like this, updates from the developer should be few and far in between... but I see notifications all the time!

Except in this case the updates are not addressing security vulnerabilities...the last update notes that mentioned security was version 5.2.5 - April 9, 2020

And if you isolated the system, the security vulnerability is minimized.

In this case, with the developer being ONE person, his approach is keep the cost of the software low and send out frequent updates (usually new goodies to try) and let those that choose to be beta testers do so.

When Deepstack first got introduced to BI, I was the beta tester and one of the first here updating the moment it came out to test it and help get it going. Then I found a stable system and have sat. And now we have people doing the same thing with CodeProject.

And we are very appreciative of those that live on the leading AKA bleeding edge as they find the bugs and issues that are not showing up on Ken's machine when he makes updates, but does show up in users systems when deployed.

That is why he has 3 options for updates - None, latest stable release, and newest AKA beta release.

 

[looney2ns]

Noted, and I appreciate your feedback as always. But this is very frustrating, assuming this is the cause of the latest round of instability. I come from an IT background where security patches are very important to apply PLUS how is it that BI has a reputation for lousy patches where people are afraid to patch their systems! Seems like with software like this, updates from the developer should be few and far in between... but I see notifications all the time!

He actively adds new features all the time, that's why you see so many updates.
BUT, if it's not broke, don't fix it. Turn off BI auto updates, and only update using those that are deemed stable. If not deemed stable, then consider them BETA.
If you install a BETA, it means just that, you are beta testing for the author, which by the way, Blue Iris is a one man shop.

If you have your local network setup correctly and secured, there is little to no need to worry about obtaining security updates.
See these:
VPN Primer for Noobs | IP Cam Talk
How to Secure Your Network (Don't Get Hacked!) | IP Cam Talk
Dual NIC setup on your Blue Iris Machine | IP Cam Talk

 

[ipcdal]

Thank you both! All sounds like good advice.

FYI, crashed again, so I'm rolling back to "highly stable update" 5.7.6.8!