Some cameras are just missing footage from the same window every day?

[Ri22o]

BI v5.7.7.14

I have 5 of my 30+ cameras that just decide to stop recording around 7am every morning, and then carry on like normal when the next hour segment hits.

I was wanting to post a video of the sound of the mini-PTZ R2D2ing. I had planned to use one of my driveway cams that is 30' away to show how the sound was/wasn't picked up from that distance. I had a good clip of walking back from the bus stop this morning at 7:51. I pulled it up on the 4K-T that is next to the PTZ and then decided to check one of the driveway cams. However, when I clicked on the 7:51:25 thumbnail in the alerts it would just take me to 7:30:00. Then I looked at the file length and it only showed 21:34 instead of 30:00 like the rest of the clips.

I checked a couple days worth of clips and the time varies, but it seems to always happen in the 7:30 hour, if it happens, because it doesn't happen every day, either.


I then checked all of my cams and found 5 others that show this behavior. They are all over/near the driveway except for one that is behind the house in the easement. 3 -ZE turrets, 1 4K-X, and 1 -AS turret.

Each camera is on its own home run.
The home runs are across 2 switches in the network rack.
The easement cam is on a switch with another camera (switch is in an enclosure on the back fence with a home run to the network rack). This other cam showed no issues. This cam seems to run about 18:00 minutes and then create an additional 12:00 clip to get to 8:00a.

The three driveway cams were 21:32, 21:34, 21:37. (These three always seem to be very close in clip length)
The cam in the front yard around the corner was 27:57.
The easement cam was 18:00.

Looking back through the calendar, it is the same window of the day. On 3-9 two -ZE driveway cams recorded for 5:54 at 7:00a and then went back to normal at 7:30a (pre-DST).

I rebuilt/repaired the database and it didn't help. Any thoughts?

Below you can see the Alert with the time stamp (7:51:25) and the clip lengths listed with 7:30 only recording for 21:34 and then starting over at 8:00.

 

[wittaj]

Is that when the DB maintenance is occurring? It wouldn't explain why it is only a couple of cameras unless it is stumbling on maint on just those cameras?

 

[Ri22o]

Settings say 2am for DB maintenance.

 

[wittaj]

I have had a few instances where I click on a thumbnail alert and it pulls up the wrong one (usually the beginning of the BVR file). I click it a few more times and then it pulls up the correct alert.

That could explain some of it, but why you are missing some of the 7:30 file is the question.

From the BI console at the bottom, how many clips/items does it say you have?

 

[Ri22o]

From the BI console at the bottom, how many clips/items does it say you have?

A lot...

 

[wittaj]

That could be it - BI used to actually give a number in the help file that stated after 200,000 items it could experience weird behavior.

And with CodeProject and other stuff, the number of items adds up quickly. Every .dat file is an item. Every burned in AI photo is an item. Every hi-rez picture is an item.

With older versions of BI, you could simply delete the .dat or burned in images or hi-rez image and do another compact/repair and the number of items would drop based on how many you deleted.

Now, as long as the .bvr file is still around associated with those items, the "number" of items remains and doesn't get reduced.

If that is the issue, you have a few options.

Shut down the service and delete the DB folder and every file in the Alerts folder and all the .bvr videos in your New and other folders if you have them. Restart and then see if it happens again. Unfortunately you lose all your videos.

Other option would be to go into the storage and say cut 25% off the storage limit and run compact/repair DB maint and see if the # of records drops below 200,000 and if so, then see if that fixes the problem.

I have quit saving hi-rez photos and stopped the burning in of AI and only turn on .dat files if CodeProject is having an issue because those all add up very quickly.

 

[Ri22o]

I have a lot of follow up questions, but let's start here.

I don't seem to be saving high res photos. Where would this be turned on?

Which folder do I need to cut the storage limit for? Alerts?

I am currently only running CPAI on my 2 LPRs and Easement. This is Easement, see anything I should change?

 

[wittaj]

OK the saving Hi rez photos is under alerts, so you probably have it marked as "database only". So that saves quite a bit.

You are saving the AI analysis files on that screen above and those can add up quickly. If you don't have to troubleshoot you can uncheck that. Open up the alerts folder and how many files are under that? That is probably the bulk of them.

To get below 200,000 records, you need to cut storage limits under NEW and any other folders you are saving video files to.

 

[Ri22o]

For both LPRs and Easement the Trigger tab is set to Database Only.

I just turned off analysis details for Easement.

Alerts has 176,220 items. (30GB)

The only thing I am seeing in the Alert folder are .dat files and AI photos for the LPRs and Easement. Is this normal? These go back to 12-26-23. My reviewable LPR video only goes back to 2-29-24 and Easement to 2-12-24. How do I manage/limit this in Alerts? Or do I just set an expiry date in Storage settings? Alerts is set to 30GB/no age

 

[SouthernYankee]

This is not a direct explanation for the problem occurring at 7 Am.
Tests that I have done indicate a noticeable drop in performance when There is Over 10,000 files in a folder, the search and adding files will eat CPU and disk performance. Look at using a sub folder per camera (see &CAM in bi help)

Also make sure there are no antivirus or disk defragmenter running on the BI disk drive.

 

[Ri22o]

This is not a direct explanation for the problem occurring at 7 Am.
Tests that I have done indicate a noticeable drop in performance when There is Over 10,000 files in a folder, the search and adding files will eat CPU and disk performance. Look at using a sub folder per camera (see &CAM in bi help)

Also make sure there are no antivirus or disk defragmenter running on the BI disk drive.

I was hoping it would maybe help the issue, but it's also something I need to fix regardless.

How do I save to the cam's subfolder? Since there are only 13 available folder options to re-map, would I address it as D:\BlueIris\New\&CAM and just make sure to have folders in the \New directory that matched with the camera's name?

No antivirus and no defragmenter running on the PC.

 

[wittaj]

For both LPRs and Easement the Trigger tab is set to Database Only.

I just turned off analysis details for Easement.

Alerts has 176,220 items. (30GB)

The only thing I am seeing in the Alert folder are .dat files and AI photos for the LPRs and Easement. Is this normal? These go back to 12-26-23. My reviewable LPR video only goes back to 2-29-24 and Easement to 2-12-24. How do I manage/limit this in Alerts? Or do I just set an expiry date in Storage settings? Alerts is set to 30GB/no age

Yeah I figured that is where they mostly were.

After 5.6.8, even if you set the Alerts to 0, it will still keep the "record" number until the original "alert" is gone from the BVR file. Up until then I could simply delete the .dat and AI photos every couple of days and keep the records number lower, but now it doesn't do that.

Turning off the .dat files will help. Then decide if you really need the burned in orange block around the AI.

I have about 20TB of storage and am under 10,000 clips.

 

[Ri22o]

Yeah I figured that is where they mostly were.

After 5.6.8, even if you set the Alerts to 0, it will still keep the "record" number until the original "alert" is gone from the BVR file. Up until then I could simply delete the .dat and AI photos every couple of days and keep the records number lower, but now it doesn't do that.

Turning off the .dat files will help. Then decide if you really need the burned in orange block around the AI.

I have about 20TB of storage and am under 10,000 clips.

What do I need to do to remove them? Cut Alerts size to 20GB? I could also set 30 days since that is all the further back the video will go?

If I am not saving high res photos, why are these being saved in the Alerts folder?

 

[wittaj]

As I said the only way to "remove" them from the records count is to delete the video files they are associated with.

You can manually delete the files in the Alerts folder, but it will still show as an "item" number until the associated video bvr file is deleted.

That is why I said to reduce your video storage by like 25% to get you down below 200,000, then after it shows records below that, then add the storage number back up and then hopefully thru attrition the remaining records will stay under 200,000 until they all cycle thru.

My Alerts folder is set to 1GB - if you aren't storing .dat and hi rez files you don't need anything but the minimum. And then set the delete time to correspond to however long you typically can get video for.

Under the Record tab, confirm you also are not saving jpegs there.

 

[Ri22o]

As I said the only way to "remove" them from the records count is to delete the video files they are associated with.

You can manually delete the files in the Alerts folder, but it will still show as an "item" number until the associated video bvr file is deleted.

That is why I said to reduce your video storage by like 25% to get you down below 200,000, then after it shows records below that, then add the storage number back up and then hopefully thru attrition the remaining records will stay under 200,000 until they all cycle thru.

My Alerts folder is set to 1GB - if you aren't storing .dat and hi rez files you don't need anything but the minimum. And then set the delete time to correspond to however long you typically can get video for.

Under the Record tab, confirm you also are not saving jpegs there.

That makes sense. Lower the thresholds, wait it out, and then bump them up. What doesn't make sense is there are files in Alerts from 12-26, but the oldest BVR file is 2-12.

No jpegs under Record tab, only video.

Are the .dat files coming from the AI analysis?

 

[wittaj]

I guess first thing is try changing the Alerts storage to 1GB and 30 days and then do the compact/repair database management and see if that clears out those going back to 12/26.

If it does, then the records count should drop and hopefully you don't have to do anymore.

If it doesn't drop, then go into the Alerts folder and delete what you don't want and then delete the DB folder and let it rebuild.

Yes the .dat files are coming from the AI analysis.

 

[Ri22o]

Just a heads up to check all profiles. I thought I had High Res and other stuff turned off, but ends up it was still turned on in my night profile.

I lowered the Alerts allocation and ran Compact/Repair and the file count is slowly going down.

 

[Ri22o]

After changing the Alerts allocation this is what I am down to.


I record in 30min clips, would it be beneficial to lengthen these to maybe an hour? 33 cameras x 48 clips a day x 30 days of footage = 47,520 clips.

 

[wittaj]

Wow that didn't take many records away, but removed many files.

I guess the next step is to drop down the storage capacity to maybe 31 TB to get the records number below 200,000?

 

[Ri22o]

That's what I thought. I messed with it some more and was going through looking at file counts and the numbers just didn't make sense. That's when I noticed it's now 188,000. I don't know if it just needed time to completely update, Compact/Repair took time to even start to show the number going down, or maybe I jiggled the handle with something.

I had added some additional drives a month ago to record the skycams to, but apparently never pointed them there. I updated their directories and then set the previous directory to 0GB and ran Compact/Repair. This is when I noticed the number drop to 188,000.

Going forward, do you think it would help to record in longer clips?