Recommended recordings file deletion in Blue Iris - Capacity or Age dependent

[agarwaldvk]

Hi All


I have read a fair bit on this forum about the recommendation of deletion of recordings files in Blue Iris. It is strongly suggested, from what I infer, that the recordings files should never be set up to be deleted by the system based on the age of the files but should be set up to be deleted by the system when the HDD reaches it nominated capacity.

Please bear with me for my ignorance and I would have thought that it would be better to delete files when they loose relevance, that is, when they become older than some defined time period. I am not even remotely suggesting that the files be deleted by the user manually but be deleted by the system (Blue Iris) by setting it up to be so deleted. But obviously, for some technical reason that I have so far failed to understand, is not advisable in this scenario.

Could someone please provide an answer as to why we set it up to deleted old files when the storage reaches its capacity and not by age?

The reason for this is that for my current setup with 8 cameras (3 off 4mp and 5 off 5 mp) on 24/7 recording, I exhaust about 400 Gb of storage everyday. Allowing for about 4.5 Tb of HDD storage, I would fill up the HDD in about 10 to 11 days.

So, when I go overseas, for example for a month, then I will only have access to the files for the last 10 last days as the older files will get deleted as the HDD gets full. If I would like to have the files for all of the 30 days to be saved, can someone please suggest what options should be considered for optimal system performance?

These are some of the options that comes to my mind :-
1. Add a 8 Tb drive to the system. If I were to do so, can the system be set up to record the footage on the existing HDD first and then on the additional drive as the existing HDD gets full and then start to delete files from the existing HDD as the additional HDD gets filled up
2. Move the older files to NAS - which doesn't work for me as my NAS is on a separate network not visible to the network that this camera security system is for data security reasons
3. Selectively manually delete files over the internet - not sure if I can do over the internet and will need to check with my IT guy who set up my network. Whilst I can view my footage over the internet securely, I am not sure if I can delete it over the internet


Best regards


Deepak

 

[wittaj]

If you set up storage by age, say 10 days, then all you will get at most is 10 days. But you may get less than 10 days if you don't have enough storage to record 10 days. But if there is still capacity for 2 more days, then you are only getting 10 days and not 12 days.

If you record based on storage, then you will get however many days of storage that available capacity will record. Maybe it is 9 days, maybe it is 11 days based on the amount of motion.

Saying to delete after 30 days when you only have storage capacity for 10 days only gets you 10 days, not 30 days.

Saying to delete when 4TB is full will get you 4TB worth of days however many that will be.

So deleting based on storage availability will get you the longest retention time.


Are you using substreams or recording just mainstream?

If you add another drive, best practice is to spread your cameras amongst the drives in the computer. Moving files from your 4.5TB to your 8TB is simply burning up the drives faster and using up CPU%. So say your 8 cameras are 2 on each side of the house, then put 1 camera from each side of the house on a different drive so if one drive fails, you still have some coverage on every side of the house.

You only move files if it is going to a NAS, not within the same computer box.

Manually deleting files is a recipe for disaster. The BI database is fragile and if you go in and start manually deleting stuff, your database could get corrupt quickly.

 

[agarwaldvk]

If you set up storage by age, say 10 days, then all you will get at most is 10 days. But you may get less than 10 days if you don't have enough storage to record 10 days. But if there is still capacity for 2 more days, then you are only getting 10 days and not 12 days.

If you record based on storage, then you will get however many days of storage that available capacity will record. Maybe it is 9 days, maybe it is 11 days based on the amount of motion.

Saying to delete after 30 days when you only have storage capacity for 10 days only gets you 10 days, not 30 days.

Saying to delete when 4TB is full will get you 4TB worth of days however many that will be.

So deleting based on storage availability will get you the longest retention time.


Are you using substreams or recording just mainstream?

If you add another drive, best practice is to spread your cameras amongst the drives in the computer. Moving files from your 4.5TB to your 8TB is simply burning up the drives faster and using up CPU%. So say your 8 cameras are 2 on each side of the house, then put 1 camera from each side of the house on a different drive so if one drive fails, you still have some coverage on every side of the house.

You only move files if it is going to a NAS, not within the same computer box.

Manually deleting files is a recipe for disaster. The BI database is fragile and if you go in and start manually deleting stuff, your database could get corrupt quickly.

Hi Wittag


Thanks for your response. Let that sit with until I get the hang of what you are trying to tell me.

I will then maybe comeback to you with possibly more questions, hopefully not!

I will need to check if I am recording mainstream or substream? How do I check that?


Deepak

 

[looney2ns]

Sub Stream Guide

.substreamGuide td { border: 1px solid #000000; } .substreamGuide a { color: #0080ff; } .substreamGuide code.inline { margin: 0; background: #f6f6f6; border: 1px solid #e0e0e0; border-radius: 0px; padding-top: 1px; padding-right: 3px; padding-bottom: 1px...

ipcamtalk.com

Blue Iris Storage configuration

 

[agarwaldvk]

If you set up storage by age, say 10 days, then all you will get at most is 10 days. But you may get less than 10 days if you don't have enough storage to record 10 days. But if there is still capacity for 2 more days, then you are only getting 10 days and not 12 days.

If you record based on storage, then you will get however many days of storage that available capacity will record. Maybe it is 9 days, maybe it is 11 days based on the amount of motion.

Saying to delete after 30 days when you only have storage capacity for 10 days only gets you 10 days, not 30 days.

Saying to delete when 4TB is full will get you 4TB worth of days however many that will be.

So deleting based on storage availability will get you the longest retention time.

Are you using substreams or recording just mainstream?

If you add another drive, best practice is to spread your cameras amongst the drives in the computer. Moving files from your 4.5TB to your 8TB is simply burning up the drives faster and using up CPU%. So say your 8 cameras are 2 on each side of the house, then put 1 camera from each side of the house on a different drive so if one drive fails, you still have some coverage on every side of the house.

You only move files if it is going to a NAS, not within the same computer box.

Manually deleting files is a recipe for disaster. The BI database is fragile and if you go in and start manually deleting stuff, your database could get corrupt quickly.

Hi Wittaj

I know its become kind of a longish post but I would greatly appreciate yours and others' views on these for my own knowledge and getting the best performance out of the system.

Agree with everything you have suggested.

As you have said that if I have storage capacity of about 11 days (this is on the premise that I have 4500 GB of storage allowing for 1 Tb capacity for 'other things' (as recommended on this forum after already discounting 500 GB initially) and I ask for deletion after 30, the system will delete after 11 days, right? So, either way, the system deletes the files after 11 days, no matter which option I pick, right? Please advise.

Out of interest, what is that spare capacity for anyway when I don't intend to store anything else on it anyway?

Further, just wondering, if I were to add another 8 Tb HDD and all the 8 cameras are set up to record on the same HDD at the same time (as it is now), can the system be set up such that the 'activity of saving the footage' SHIFTS from the original HDD to the new HDD?

Given that I do not intend to store anything else on this HDD (or even on the proposed additional HDD - 6 or 8 Tb), is it technically recommended to use about 90% of the rated capacity (5400 Gb for a 6 Tb or 7200 Gb for a 8 Tb HDD) for storing the footage and then deleting them cyclically (from the original HDD first and then the new HDD) as the drives progressively get full - the prerequisite being that 'the activity of saving the footage SHIFTS from one HDD to another as the original HDD gets full to the new HDD'. If this can be done, then I am all good as it would then give me a total capacity of about 30 days.

Here is the image of one the camera settings - I can't tell if I am recording the main stream or the sub-stream although I did follow the guide provided and the details look pretty much the same as shown on the guide




This is the screenshot provided on the guide



Best regards


Deepak

 

[wittaj]

That is correct - recording and deleting based on available storage will ALWAYS result in more days of storage than recording and deleting based on number of days.

As I said, it is burning up the HDD prematurely and using unnecessary CPU to do as you say "set up such that the 'activity of saving the footage' SHIFTS from the original HDD to the new HDD"

It is poor practice to simply move video from one HDD to another within the computer.

You can certainly do that if you want, but like I said moving files from one HDD to another HDD is just causing the HDD and CPU to work harder than it needs to and opens your system up to database errors and other issues.

You said you have 8 cameras, so put 4 on one drive and 4 on another. That then benefits you if one HDD goes out, you do not lose all of your cameras.

Or if the drives and MPs of the cameras are different, then allocate it such that the same Mbps are going to each drive - maybe it is 3 cameras to one drive and 5 to the other. Much better than just moving files around on HDDs.

You are using substreams, but it is the RECORD tab that says how you are saving the files and that dictates how much storage is being used.

Continuous recording will record 24/7 mainstream ONLY and will take up the most space and result in the least number of days.

Triggered events only recording will only record during triggered events and will result in the most number of days of storage. BUT you could miss events leading up to the trigger or even miss an event if the camera for some reason didn't trigger.

Continuous + triggered events results in the best overall video capture. It will record the substream when the camera is not being triggered (thus saving storage space as the substream takes up less storage than mainstream) but will switch to mainstream during triggered events.

 

[agarwaldvk]

Hi Wittaj

As I said, it is burning up the HDD prematurely and using unnecessary CPU to do as you say "set up such that the 'activity of saving the footage' SHIFTS from the original HDD to the new HDD"

It is poor practice to simply move video from one HDD to another within the computer.

I think this has been misinterpreted. I have never intended to say that I want to MOVE files - I don't.

Let's say that the current set up records all the footage on the only existing drive - Drive A (6 Tb)
I now add a new drive - Drive B (8 Tb)

When I said "set up such that the 'activity of saving the footage' SHIFTS from the original HDD to the new HDD", what I meant was this :-

When, after 11 days, the system realizes that "Drive A" is full, at that point, it moves the process of recording the footage, from Drive A (the original drive that the footage was being recorded on) to Drive B (the proposed additional drive). As at this point, the system does not delete or move any files from anywhere to anywhere. All files sit where they were - only new files get added to Drive B. Then from day 12 onward, all the recording is done on Drive B. After another 18 days, when this Drive B gets full, the system realizes that "Drive B" is full and it needs more room, at that point, it deletes files from Drive A and NOT Drive B and continues to do so until Drive A is full with new footage. Then it goes to to Drive B and records the footage there and repeats the process cyclically. So, there is no file movement at all - only file deletion - it is just the timing which is different

I hope the question that I originally, in this scenario, is clearer now. Can you please advise the system can be set up to do so?


Alternatively, as you suggested, I can always, allocate the 3 off (4 MP cameras) to record on Drive A - giving me about 36 days of storage and allocate the other 5 off (5 MP cameras) to record on the Drive B - giving me about 29 days worth of storage. This on the presumption of allocating 90% of the nameplate capacity of the drive - 5400 GB for 6 Tb drive & 7200 GB for 8 Tb drive.


Best regards


Deepak