Hi, My BI stoped recording?! Restarted it few times, and upgraded to 5.9.4.11 but it wont start recording? UI is working and I can see all cameras...And where alerts shoud be it's counting scanning alers... now it is on 70200....And one more thing, computer restarts randomly by itslef?! Only BI is on that server...any help is apriciated! I'm using BI for 5-7 years.....
My BI stoped recording?!
- Thread starter wrbex
- Start date
Vettester
Getting comfortable
- Feb 5, 2017
- 853
- 841
It's running a rebuild the database process. Be patient, alerts should come back and it will start recording when it finishes.
Yup, you've encountered one of Blue Iris's biggest flaws, its database. The biggest problem is it gets broken at the drop of a hat, frequently through no fault of yours or your computer's, but because of a programming error on their end, which happens alarmingly often (see this thread for a recent and relevant example). But it can also become corrupted when Blue Iris crashes or the computer crashes. So solving that power supply stability problem may be helpful.
Proper database engines are designed with resiliency as a very high priority, so data is not lost due to typical programming errors, crashes, etc. Blue Iris's database does not have any such protection. So when it gets broken, the only recovery method is a full rebuild which as you've discovered is dreadfully slow when you have terabytes of recordings for Blue Iris to scan through. The only way to speed that up is to delete the recordings before letting BI rebuild the database.
Understandably, Blue Iris relies on having the database be fully built and online in order to do any kind of recording management, so nothing can be recorded while a database rebuild is happening.
Proper database engines are designed with resiliency as a very high priority, so data is not lost due to typical programming errors, crashes, etc. Blue Iris's database does not have any such protection. So when it gets broken, the only recovery method is a full rebuild which as you've discovered is dreadfully slow when you have terabytes of recordings for Blue Iris to scan through. The only way to speed that up is to delete the recordings before letting BI rebuild the database.
Understandably, Blue Iris relies on having the database be fully built and online in order to do any kind of recording management, so nothing can be recorded while a database rebuild is happening.
The biggest thing you can do to help avoid the need for a database rebuild (besides replacing faulty hardware) is to not update Blue Iris as often. Make sure automatic updates are off and only update when you think you could tolerate some downtime in case the update causes problems. Many would say you should only update BI to versions they consider "stable" and well tested, but lets be honest here, 5.9.4.11 which you just updated to is one of those versions.
As for your log file, the "FORCE quitting CGadSocket thread" messages are sometimes seen when a camera instance crashes and restarts. I'm not sure of the cause but it may be related to Intel hardware acceleration. If you have that enabled and it is causing your camera windows to restart often, freeze, or otherwise show visual corruption, you might try turning it off (intel hardware acceleration). Also if you are utilizing sub streams, you should turn off hardware acceleration for those cameras because it has been observed to increase CPU usage in such cases on some systems.
aesterling
Getting comfortable
Do you think that BI is backed into a corner with the current BI database architecture and has no path forward to address these shortcomings? Has it not yet been addressed because it's such a massive undertaking to move to a new database structure, or does the dev not share your opinion that it's a problem in the first place?
I wouldn't say it is backed into a corner. There are no major external integrations to coordinate with; Blue Iris is effectively the only producer and consumer of its database files and they don't need to be sent to other systems and work there, so he would not need to worry about backwards compatibility.
I haven't spoken to Ken about it recently or in any kind of depth, but there's no way Ken hasn't thought about migrating the database out of this primitive home-grown format and into a mature and stable engine. There are a lot of challenges involved, starting with deciding it is worth the trouble in the first place. Blue Iris's homegrown DB system is no doubt easy for him to work with, since he's told me basically all it is is a simple dump of some in-memory structures to and from the disk. To work with a 3rd-party DB system properly, you are supposed to design a schema (tables, columns, relations, etc) and store all your data in it and query the database whenever you need to retrieve some of that data. (often, querying the database a lot is not very fast, so devs try to maintain an in-memory cache of the data, and then the cache gets out of sync, and bugs arise)
It can be a daunting prospect to decide to rip out something simple that (mostly) works and replace it with a very complex DB system designed and maintained by other people. Especially if he doesnt have any (or any recent) experience using that DB system, there would be a steep learning curve. He probably thinks it is better to just fix the bugs with the current DB system as they arise, never mind that it often means hours of downtime as the DB rebuilds. If it took too long to rebuild, that is partly the user's fault for having too many clips and alerts. Right?
Another challenge is making the right choice of which engine to migrate to. You can do all kinds of research on which DB engines are good at this or that, but ultimately you don't know until you've tried it and run with it for a while. As an example I have an open source project I wrote for gathering and organizing error reports. I originally built it to use the SQLite database engine through a particular C# wrapper library. The database solution was fully embedded in my app and it was easy to code for and it worked just fine..... Except every few weeks or months, the program would start up and just overwrite a database file with a blank one for no apparent reason, wiping out everything that was in it and forcing me to restore that file from a backup. Absolutely unacceptable. I could not find the cause or even reproduce the issue consistently, so I migrated the app to a totally different database system (postgres) which has been working with zero issues for a few years now. Luckily I was and still am the only user of this project (that I know of; it is on github after all) so I didn't have to worry about causing customers any grief with a clunky data migration experience and needing to install and configure a 3rd-party database server in order to run my app.
If Blue Iris migrated to a new DB system only to encounter problems like I did with SQLite, then it would have to migrate again to something else, and that would be a lot of sunk time and potential frustration for users. On the bright side, the second migration is typically easier.
aesterling
Getting comfortable
Useful insight, thanks @bp2008. I often read your criticisms of BI's database but didn't know if there was an obvious solution.