BI Storage issues....

[sdkid]

OK. My BI machine (running 5.3.3.14) has a 250Gb EVO SSD for the OS drive, a 1Tb drive for "New", and a 4TB WD Purple for "Storage".
It is an old Dell, 8GB, with an I5-4460

Look at this screenshot from UI below... Note the clips--- a handful from this morning, then it jumps 3 weeks BACK to October 24th...
Questions:
HOW can it save MORE than I allow?
Why is it NOT moving files over to Storage from New as it needs space on NEW?
If I marked some alert clips to protect--- does that prevent them from being moved from New to Storage?
I know it recommends NEW is on local drive with DB. Does it really matter than I am using a 1TB HD for that? Should I move the DB to that same HD as the NEW folder? DB is set to "Compact/Repair" each night at 2am.

I recently switched some cameras to continuous record.
My record path and filename structure to NEW looks like this: &CAM.%Y%m%d%\&CAM.%Y%m%d_%H%M%S (note the slash-- making folders for each day for each cam.
EDIT: Using substreams for most cams, and recording direct to disk.

Alright boys--- tell me where I screwed up...

 

[Flintstone61]

I may know less than you.....My dell is running a single 5.0 TB 3.5 5400rpm drive, and everything is at defaults for "new" and Db. I don't compact anything. Maybe bypass the 1Tb drive and put everything on the big drive?

 

[sebastiantombs]

As humans we tend to want to "compartmentalize" things, but there is no reason to move things around from New to Stored. Set the limit to size only, don't use days. Use the 1TB drive for 20% of your cameras and the 4TB for the remaining 80%. You can use the same, default BI, directory structure on both drives. Not recording continuously on all cameras is just asking to miss something important, generally speaking.

If you can, delete all the files and the files in the BI database, reformat both storage drives and set the block size to 1024, and start fresh.

What are you storing on the C drive? Given that it's an SSD it isn't a good idea to write any video to it at all because it will significantly shorten the drive life.

 

[whoami ™]

Also if possible set cams to stream @H256 EDIT: @H265 VBR @ 15fps with a iFrame interval of 15. I was amazed at how much more I was able to fit on a 8TB WD Purple and what it did for CPU usage.

 

[looney2ns]

See here

 

[sebastiantombs]

I think you typoed, that's H264 and even 10FPS is fine for video surveillance with iframe set to 10.

 

[sdkid]

As humans we tend to want to "compartmentalize" things, but there is no reason to move things around from New to Stored. Set the limit to size only, don't use days. Use the 1TB drive for 20% of your cameras and the 4TB for the remaining 80%. You can use the same, default BI, directory structure on both drives. Not recording continuously on all cameras is just asking to miss something important, generally speaking.

If you can, delete all the files and the files in the BI database, reformat both storage drives and set the block size to 1024, and start fresh.

What are you storing on the C drive? Given that it's an SSD it isn't a good idea to write any video to it at all because it will significantly shorten the drive life.

UPDATE:

I initiated a manual "Compact/Repair Database" that started chugging along. I decided to just let that do it's thing while I paid attention to my grandson, watched a little football, and snoozed in my recliner... Ahhhhh-- Sundays. I will be changing my storage structure to use ONE folder for the storage based on SIZE (not TIME), and utilize the 2 hard drives for different camera groups--- was getting ready to do just that, Then I checked on things.....

and there was a massive change in disk usage.... It apparently decided to delete MOST of my video files?? Looks like around 2TB of files went bye bye while I snoozed....
Uhhhhh.... that's a bit scary.

 

[whoami ™]

I think you typoed, that's H264 and even 10FPS is fine for video surveillance with iframe set to 10.

which codec takes up less storage? H264 or H265? I was under the impression H265 is more compressed. Im a long way from expert on codec so dont take my question the wrong way.

The difference between H.264 and H.265 is primarily one of streaming bandwidth and storage requirements There is great interest in H.265 recording, because the new video coding standard brings huge bandwidth savings of approximate 40-50% over H.264 encoded content with similar quality

 

[sebastiantombs]

You originally typed something other than H265.

H265 does provide more compression and can save some storage space and bandwidith. It is kind of a minimal saving so either H264 or H265 will work just fine.

 

[sebastiantombs]

@sdkid Have you looked "back" to see how far back your clips go? Once again, what are you writing to the C drive?

 

[sdkid]

@sdkid Have you looked "back" to see how far back your clips go? Once again, what are you writing to the C drive?

C drive had minimal space set aside initially just for Alert images, which were really tiny in size. Was going to move that off the SSD, but didn't get around to it. Looking at that folder now, it is filled with empty folders from different cams & dates.... All are empty....

 

[sebastiantombs]

Never save anything, other than the database itself, on the C drive. When recording 24/7 alerts are nothing more than markers in the database to locate the video, not actually separate recordings.

 

[sdkid]

@sdkid Have you looked "back" to see how far back your clips go? Once again, what are you writing to the C drive?

Here is part of the Alerts Clips history on one cam..... look at the jump in dates....

 

[sebastiantombs]

If there's nothing you really need in the clips, I'd just delete the database files and let BI re-create it. Worst case, reformat the storage drives, delete the database and start fresh. Personally, I keep a weeks worth of 24/7 video on a 4TB drive. I review the alerts every day and if I see something I want to save I export it using UI3. 14 cameras with a total of about 50Mps.

 

[sdkid]

If there's nothing you really need in the clips, I'd just delete the database files and let BI re-create it. Worst case, reformat the storage drives, delete the database and start fresh. Personally, I keep a weeks worth of 24/7 video on a 4TB drive. I review the alerts every day and if I see something I want to save I export it using UI3. 14 cameras with a total of about 50Mps.

I like this plan.... Anything REALLY interesting to me-- I do the same thing already-- download the clip via UI

I am only running 6 cams... two 5mp, two 4mp, one 2mp ptz, and 1 old 1.3mp from 2012...
Not sure what their combined bandwidth is.... need to look at each and add them up. I am likely Well below your bandwidth, unless you have some seriously minimal settings going.

 

[sebastiantombs]

I am running at 10FPS, it is surveillance and not "home video". I have 12 2MP running and 2 4MP running, all at full resolution. Bit rates are 2048 on the 2MP and 8192 on the 4MP. Using substreams keeps the CPU under 10% and those are set as low as I can get them on all the cameras.

 

[Flintstone61]

Where do I select to display substreams over mainstreams?

 

[whoami ™]

Never save anything, other than the database itself, on the C drive. When recording 24/7 alerts are nothing more than markers in the database to locate the video, not actually separate recordings.

Are those markers saved in [NEW]? Always wondered about that. My [ALERTS] folder appears to always be empty. So am I wasting space by giving Alerts 32GB in Clips and Archiving?

 

[sebastiantombs]

@Flintstone61 You don't select substreams. They are configured for each camera in BI. They need to be entered in the video configuration page. They also need to be enabled on each camera through the web GUI of the camera itself. There you can select the resolution of the substream and set the bitrate for the substream. Ken, BI developer, says to keep the ifame rate the same in both sub and main streams. I think that is related to motion detection and video continuity since that way they both start at the same point in time.

5.2.7.0 - RTSP sub-streams for high-MP cameras

5.2.7.0 - RTSP sub-streams for high-MP cameras The help file has not been updated with an explanation of this new feature. There is a new box to enter the substream path in the video config settings. EDIT with release notes. 5.2.7 - May 1, 2020 You may now specify a second “sub” stream for an...

ipcamtalk.com

@whoami The markers are in the database itself and are not saved elsewhere as files. They are data points in the database only that "point" to the location in the continuous recorded video file. This assumes 24/7 recording. If you are set to only record on triggers, video files are saved to the Alerts folder.

 

[Flintstone61]

@Flintstone61 You don't select substreams. They are configured for each camera in BI. They need to be entered in the video configuration page. They also need to be enabled on each camera through the web GUI of the camera itself. There you can select the resolution of the substream and set the bitrate for the substream. Ken, BI developer, says to keep the ifame rate the same in both sub and main streams. I think that is related to motion detection and video continuity since that way they both start at the same point in time.

5.2.7.0 - RTSP sub-streams for high-MP cameras

5.2.7.0 - RTSP sub-streams for high-MP cameras The help file has not been updated with an explanation of this new feature. There is a new box to enter the substream path in the video config settings. EDIT with release notes. 5.2.7 - May 1, 2020 You may now specify a second “sub” stream for an...

ipcamtalk.com

@whoami The markers are in the database itself and are not saved elsewhere as files. They are data points in the database only that "point" to the location in the continuous recorded video file. This assumes 24/7 recording. If you are set to only record on triggers, video files are saved to the Alerts folder.

Yeah they're enabled on all web gui's....just not sure if you have to write code or script or something in config page....