Upgraded 5.5.8.2 to 5.7.0.4 - Issues with Compact/Repair DB and Clip List

D

djmadfx

Getting the hang of it
Sep 29, 2014
106
20
As the title says, I upgraded from 5.5.8.2 to 5.7.0.4. I noticed issues with compact/repair DB and repair/regenerate DB failing. I also noticed a very high amount of time where it would take to view the list of recent clips via both UI3 and the mobile app. I downgraded back to 5.5.8.2 for now and everything is working perfectly again. Of course, I don't want to stay at this version as I did just pay the $35 maintenance fee as I was hoping to try out ALPR and Direct to Wire.

Is there an upgrade path that would be better to take in getting to version 5.7.0.4?
 
Is 5.7.0.4 listed as stable? If not, don’t use it and wait til 5.7.x stable.
 
I tested with the latest stable, 5.6.9.8, as well. Same issues.
 
Here are my findings thus far:

5.5.8.2 - original working
5.5.9.6 - working
5.6.0.11 - broken - DB corruption detected
5.6.1.3 - broken - DB corruption detected
5.6.2.10 - broken - DB corruption detected
5.6.3.7 - broken - DB corruption detected
5.6.4.2 - broken - DB corruption detected
5.6.5.9 - broken - DB corruption detected
5.6.6.1 - broken - DB corruption detected

Error starts popping up in the logs, "DB corruption detected" and my clips are partially missing from the clip list.
 
You lose the alert thumbnails, but not the actual files themselves.

Do a search here, once the move was made to 5.6.x.x. it pretty much has required a delete of the DB and let it rebuild. Losing the alert thumbnails is a sacrifice of updating to 5.6 and beyond.

You can bounce all around updates and anything above 5.6 will result in what you are experiencing, either go below 5.6 or recognize to run beyond 5.6 you need to delete the DB and let it rebuild.

It is a one time pain and others that have done that have then not had problems thru the latest updates.
 
  • Like
Reactions: Flintstone61 and djmadfx
wittaj said:
You lose the alert thumbnails, but not the actual files themselves.

Do a search here, once the move was made to 5.6.x.x. it pretty much has required a delete of the DB and let it rebuild. Losing the alert thumbnails is a sacrifice of updating to 5.6 and beyond.

You can bounce all around updates and anything above 5.6 will result in what you are experiencing, either go below 5.6 or recognize to run beyond 5.6 you need to delete the DB and let it rebuild.

It is a one time pain and others that have done that have then not had problems thru the latest updates.
Click to expand...
That surely explains a lot! Thank you for that info. I can effectively do a 'Database >> Delete & Regenerate' ?
 
Yeah, you can do that, but the best solution is to do what is in post 4 - shut down the service, delete the DB folder and then open.

If you do it under the option while BI is running, it can cause the problem to carry over.
 
  • Like
Reactions: djmadfx
wittaj said:
Yeah, you can do that, but the best solution is to do what is in post 4 - shut down the service, delete the DB folder and then open.

If you do it under the option while BI is running, it can cause the problem to carry over.
Click to expand...
Thank you. I'll go that route then.
 
I got everything up and running again. The rebuild took ~48 hours due to the amount saved. I have my cameras set, "Add to alerts list: Hi-res JPEG files", so those images were able to be used when rebuilding the database so nothing was lost. I really wish there was a way to rebuild while still recording, so all that downtime doesn't exist.
 
  • Wow
Reactions: Flintstone61 and looney2ns
djmadfx said:
I got everything up and running again. The rebuild took ~48 hours due to the amount saved. I have my cameras set, "Add to alerts list: Hi-res JPEG files", so those images were able to be used when rebuilding the database so nothing was lost. I really wish there was a way to rebuild while still recording, so all that downtime doesn't exist.
Click to expand...
The rebuild should never take 48 hours... You would have to have a building full of hard drives for that to happen .. something is wrong with your storage... Is your database on your SSD?
 
  • Like
Reactions: weigle2, Flintstone61 and looney2ns
Are you saying you have alert image thumbnails in the BI console on the left from before you deleted the database?

Are your drives on USB connections? 48 hours is crazy long.

1677296370581.png
 
  • Like
Reactions: Flintstone61
Seems like the 4th time I've had to delete the db folder in the past few months . . .
 
The database is on an SSD. I use StableBit CloudDrive to extend my storage so I can keep a big archive of clips (~19k BVR files) which is what slowed down the rebuilding process. Rebuilding the clips portion of the database took ~24 hours by itself. The other big chunk of time was BlueIris looking through the alerts folder.

Ideally the database rebuilding process should be revamped. I only needed a small portion rebuilt for immediate use instead of the whole thing. Ideally it should rebuild the database in the background so recording never stops. After it's done it should merge the databases together (currently recording and rebuilt) which would minimize downtime.

I had an issue a couple of days ago during the daily compact/repair where the process wasn't working. I woke up to find that my system did not record for HOURS. I ended up just disabling the compact/repair process.

I have been using Veeam for daily backups, which has saved me quite a few times, where I could just restore the database files in 15 minutes.
 
djmadfx said:
The database is on an SSD. I use StableBit CloudDrive to extend my storage
Click to expand...

OK so here lies the problem, the cloud drive will be orders of magnitude slower than even an old IDE hard disk, so is not going to be ideal at all. I'd recommend using only your local storage for BI and use your cloud storage only as a long term archive. Personally, I'd have a simple robocopy script set to run a few times a day to move clips and alerts images from the BI live local folders to the clound storage mount point that are older than a set number of days, maybe 14-30days held on local SATA drives and everything else moved to cloud, you would see fewer issues and DB rebuilds taking minutes not hours. I've got 30days of alerts and 24/7 clips from 8 cameras (about 6.5TB on SATA 5400rpm) and my DB rebuilds are about 40 seconds.
 
CrazyAsYou said:
OK so here lies the problem, the cloud drive will be orders of magnitude slower than even an old IDE hard disk, so is not going to be ideal at all. I'd recommend using only your local storage for BI and use your cloud storage only as a long term archive. Personally, I'd have a simple robocopy script set to run a few times a day to move clips and alerts images from the BI live local folders to the clound storage mount point that are older than a set number of days, maybe 14-30days held on local SATA drives and everything else moved to cloud, you would see fewer issues and DB rebuilds taking minutes not hours. I've got 30days of alerts and 24/7 clips from 8 cameras (about 6.5TB on SATA 5400rpm) and my DB rebuilds are about 40 seconds.
Click to expand...
Yeah, I know that was the issue. It's nice to keep everything within the BlueIris interface. I never imagined having to do a db rebuild. It was pretty much a one-time thing for me. Yeah, I can move the footage, but I won't have the indexing of the clips available. It's not a huge problem.
 
You must log in or register to reply here.
Top Bottom