Issues with over-allocation, gaps in continuous recording

[theothernguyen]

Hello there!

Hoping to gather some input from you all.

Running BI 5 on a Windows 10 VM, with 9 IP cams over Ethernet, using SSD as my DB storage and a 3.6TB WD Purple for the BI clip storage.

Five of the 9 cams are set to continuous recording with triggers to alerts and was all working fine without any gaps for a few days; the rest of the cams are triggered only.

The issue I was having is that the cams set to continuous recording would eventually develop gaps and breaks where there would be seconds or minutes of unrecorded periods. Looking around, I found a Over-allocation error, I checked some settings and did basic troubleshooting including repairing the DB, but the error will fix itself after the DB is repaired and behave as expected with continuous recording/no breaks or gaps, but after a few days, the over-allocation error would come back and so would the gaps in continuous recording.I would have to repair the DB again. I have it so that both New and Alerts will dump into Stored, and then deleted in Stored eventually.

I am at a loss, the fix might be really simple but I cannot figure it out.

Please see attached screenshots.

Any thoughts would be appreciated! Let me know if you require further information.
After fresh DB repair:


After several days:


Thanks!
Theo

 

[SouthernYankee]

Never use age for any allocation in BI only use space.

1) Are your cameras wifi or hardwired ?
2) does the connect from BI to the cameras run through the router ?

 

[IAmATeaf]

I would move Alerts to the SSD, you don’t have much space on C so allocate 5gb for now and see how you get on.

Then I’d get rid of Stored, so set that to zero.

Then allocate all of the WD space (minus 10%) to New. Disable the age limit and select the Delete option so that BI will delete old clips as required. Then set all your cams to write to New.

Reason, Alerts are small so don’t need that much space and there’s no logical reason to move clips from New to Stored as they are on the same drive so doing this is just a waste of both time and CPU time.

 

[Rakin]

Also might check the settings for each cam in BI. Go to record and make sure you don’t have any set to protect.

Once you get your settings sorted out you may need to go and manually delete your files in your folders on the drive that keeps becoming over allocated then repair DB.


Sent from my iPhone using Tapatalk

 

[theothernguyen]

Thanks everyone for your replies! See below responses

Never use age for any allocation in BI only use space.

1) Are your cameras wifi or hardwired ?
2) does the connect from BI to the cameras run through the router ?

1. all POE hardwired
2. connects to a switch which the computer is connected to.

I would assume that BI should behave in such a way that once either the space allocation is full OR the clip has reached a certain age, that the clip is deleted. I suppose that is not the case?

I would move Alerts to the SSD, you don’t have much space on C so allocate 5gb for now and see how you get on.

Then I’d get rid of Stored, so set that to zero.

Then allocate all of the WD space (minus 10%) to New. Disable the age limit and select the Delete option so that BI will delete old clips as required. Then set all your cams to write to New.

Reason, Alerts are small so don’t need that much space and there’s no logical reason to move clips from New to Stored as they are on the same drive so doing this is just a waste of both time and CPU time.

Consolidating the new/stored sounds like a great idea! But, why do you suggest moving the Alerts to the SSD? If a continuous clip gets converted to alert, I figured it would have to move the whole file/clip to the SSD?

It was actually recording fine, continuously, if there was ample space on the WD Purple drive, I don't think it is the throughput to the WD Purple drive that is the problem. Current write speeds to that drive are about 10 MB/s, max.

Also might check the settings for each cam in BI. Go to record and make sure you don’t have any set to protect.

Once you get your settings sorted out you may need to go and manually delete your files in your folders on the drive that keeps becoming over allocated then repair DB

No cameras have the "protect" option selected. I will wipe the whole drive clear and retry the repair and see if that helps.

 

[IAmATeaf]

As far as I know Alerts are just pointers to the main clips which is why they don’t occupy that much space.

 

[Mikk36]

Alerts folder only gets used if you store high-quality snapshots. Those are useless, if you do continuous recording anyway.
Limiting by time is also propably useless, because you want to maximize the storage utilization and only want it to delete old clips if you're out of space.
Here are my settings as a reference:

I don't see your database/logging settings, do you have it set to log the move/delete details? If so, what do you see in the logs?

 

[theothernguyen]

Alerts folder only gets used if you store high-quality snapshots. Those are useless, if you do continuous recording anyway.
Limiting by time is also propably useless, because you want to maximize the storage utilization and only want it to delete old clips if you're out of space.
Here are my settings as a reference:
View attachment 48316 View attachment 48317 View attachment 48318
I don't see your database/logging settings, do you have it set to log the move/delete details? If so, what do you see in the logs?

Thanks for your reply. I changed it to limiting based on space only, not time as suggested by others. I do have it set on Log the move/delete details. I have to wait for the drive to fill up again and I can see what is happening in the logs.

With continuous recording, Is there a way to keep track of all the triggers, without having necessarily moving the file to the Alerts folder? I don't mind keeping all the clips in a single folder, so as long as clips are marked with when the trigger occurs.

Interestingly, after I cleared the whole WD Purple drive, eventually BI randomly shot up to 100% CPU usage and slowed down the whole computer, unclear why... I had to close the program down manually and restart it....

 

[fenderman]

Thanks for your reply. I changed it to limiting based on space only, not time as suggested by others. I do have it set on Log the move/delete details. I have to wait for the drive to fill up again and I can see what is happening in the logs.

With continuous recording, Is there a way to keep track of all the triggers, without having necessarily moving the file to the Alerts folder? I don't mind keeping all the clips in a single folder, so as long as clips are marked with when the trigger occurs.

Interestingly, after I cleared the whole WD Purple drive, eventually BI randomly shot up to 100% CPU usage and slowed down the whole computer, unclear why... I had to close the program down manually and restart it....

as noted, the alert folder is never used unless you choose to store high res alert images which is a waste of space as low resolution images are saved in the database to mark alerts. No files are moved to the alert folder, blue iris keeps track of everything, there is nothing you need to do.

 

[theothernguyen]

as noted, the alert folder is never used unless you choose to store high res alert images which is a waste of space as low resolution images are saved in the database to mark alerts. No files are moved to the alert folder, blue iris keeps track of everything, there is nothing you need to do.

Ohhhh I got it now. It didn't quite click with me right away.

Well hopefully the limitation by space rather than time will fix the weird over allocation problem.

 

[Mikk36]

Another question: how often do you split your video files? I mean size/time.

I've set mine to split every 8 hours so I get 3 files per day per camera.

 

[Walrus]

I do 1 hour files, anything longer I find difficult to review. When you open a clip and drag the slider bar, an 8 hour file is impossible to find motion/someone walking by if it wasn't flagged as an alert.

 

[SouthernYankee]

I do it at 1 hour intervals. So the files are a more manageable size. Faster to move. smaller file sizes can reduce the possibility of over allocation.

 

[Mikk36]

I do it at 1 hour intervals. So the files are a more manageable size. Faster to move. smaller file sizes can reduce the possibility of over allocation.

BI checks for over-allocation every 5 minutes, file size does not play a part in managing over-allocation.
You just get a little more out of the storage limit you've set with smaller files because you're deleting in smaller chunks.

 

[theothernguyen]

Another question: how often do you split your video files? I mean size/time.
View attachment 48349
I've set mine to split every 8 hours so I get 3 files per day per camera.

I split every 1 hour, or 4gb, one video file for each hour. I changed it from 8 hrs to 1 hr to see if that helps with the over allocation error.

 

[IAmATeaf]

I have mine set to 1 hour and/or 1Gb file size.