Blue Iris Status Log Errors

[danlorenzo]

I cannot seem to find out in the documentation or in the forums what the Alerts folder errors shown in the status log are from. I recently enabled GPU Deepstack, and they seem to originate from that timeframe. I turned off the "Save DeepStack analysis details" thinking that the .dat files were somehow causing this, but that was not the issue. Would this be from the cancelled alerts? I have done several database repairs after stopping saving the analysis details, and deleting the .dat files, to no avail. You can see that there are a significant amount of them.

 

[wittaj]

You will lose the alerts in the timeline and to the left, but if you stop BI service, delete the DB folder, and re-open BI, it will rebuild the DB and remove those errors.

 

[danlorenzo]

Thanks, I will give that a try.

 

[danlorenzo]

So I did as you said, and it cleared the errors. But, errors have started to re-appear in the successive database activities. You can see the errors starting to appear a few hundred at a time after the first database activity after the rebuild. I did not seem to have this issue prior to using DeepStack - do you think that the errors related to that?

 

[sebastiantombs]

We need a screen shot of the BI storage tab in the status window, the graph with a lightning bolt, in the upper left of the console.

Best practice is to allocate 90-95% of a drive to BI use. Write all new video to the New folder and set that folder by size, not by date, and leave the files there. Moving them around is pointless and wastes CPU and disk cycles.

 

[SouthernYankee]

You have and allocation problem, and a Deep stack motion detection problem. You need to work on your system configuration first before evening attempting deepstack.
What is your CPU, memory and graphics processor ?

Screen shots:
1) windows task manager process tab sorted by memory (most at the top), Must contain, memory, disk, network, GPU, GPU engine columns
2) Blue Iris Setting about tab (black out your license key)
3) Blue iris status (lighting bolt graph,upper left corner) clip storage tab
4) blue Iris status cameras tab
5) Blue iris settings clips and archiving tab , for the NEW folder, stored folder, alerts folder. (three screen shots)
===================================

My Standard allocation post.

1) Do not use time (limit clip age)to determine when BI video files are moved or deleted, only use space. Using time wastes disk space.
2) If New and stored are on the same disk drive do not used stored, set the stored size to zero, set the new folder to delete, not move. All it does is waste CPU time and increase the number of disk writes. You can leave the stored folder on the drive just do not use it.
3) Never allocate over 90% of the total disk drive to BI.
4) if using continuous recording on the BI camera settings, record tab, set the combine and cut video to 1 hour or 3 GB. Really big files are difficult to transfer and review.
5) it is recommend to NOT store video on an SSD (the C: drive).
6) Do not run the disk defragmenter on the video storage disk drives.
7) Do not run virus scanners on BI folders
8) an alternate way to allocate space on multiple drives is to assign different cameras to different drives, so there is no file movement between new and stored.
9) Never use an External USB drive for the NEW folder. Never use a network drive for the NEW folder.
10) for performance do not put more than about 10,000 files in a folder, the search and adding files will eat CPU and disk performance. Look at using a sub folder per camera (see &CAM in bi help)


Advanced storage:
If you are using a complete disk for large video file storage (BVR) continuous recording, I recommend formatting the disk, with a windows cluster size of 1024K (1 Megabyte). This is a increase from the 4K default. This will reduce the physical number of disk write, decrease the disk fragmentation, speed up access.
Hint:
On the Blue iris status (lighting bolt graph) clip storage tab, if there is any red on the bars you have a allocation problem. If there is no Green, you have no free space, this is bad.
======================================

 

[danlorenzo]

I have been using BI since about 2016, and do feel that I have tuned the system to some extent. Obviously, I am still doing something wrong. I only added Deepstack a couple of weeks ago.
Ok, here is my setup.
Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
Windows 10, Blue Iris 5.5.1.3
64 GByte RAM
GPU Nvidia GeForce GTX 1080
Deepstack: GPU
Drives: New and Alerts on Q drive, a dedicated SAMSUNG 970 EVO Plus SSD 1TB - M.2 NVMe
C drive is a Samsung 2 TB SSD with BI program installed.
Lots of storage to get 5-6 weeks of recordings.
Internal: 1 8TB drive, 1 16 TB drive
DAS: 3 8TB drives, 1 16TB drive
Answers to the standard allocation post

• I use size as the limit for the folder, with a large time to cause it to limit based upon space.
• New and stored are on different drives. Only Alerts and New are on the same (Q) drive.
• I keep the drive freespace at minimum of 10% (no red on disk management). Most drives are dedicated to BI storage, but a couple are not.
• I have one hour clip maximums, with the size limit set to allow up to one hour. Can be slightly more space for 4K cameras. Continuous recordings are used.
• I save New and Alerts on a dedicated SSD. I monitor the TBW, and understand when it fails that it will have to be replaced. I moved from the C drive SSD to the Q drive fairly recently.
• I did have defrag set up for BI drives, but now have turned off.
• I have excluded BI BVR files, the BI program folder, and the BI storage drives from Windows Defender.
• New and Stored are on different drives.
• External DAS is used for folders other than New. New and Alerts are non NVMe drive.
• I have not looked at using subfolders per camera – and I do have some large drives. Something I will look into.

Advanced: I have not looked at formatting my BI only drives to larger cluster sized. I will look into that for the dedicated ones that I have.

 

[sebastiantombs]

Again, uncheck time there is no need for it when allocating by space allows BI to do its' thing managing the files. Using a small drive, the 1TB, for new and alerts insures things will have to be moved somewhere else. The simplest solution would be to allocate new style storage across the platters drives with some camera writing to each drive rather than new to stored to long term stored. Alerts can be sent to one platter drive as well with the amount of space you have, no need to write to that 1TB at all. Remember that the lifespan of an SSD drive is based on write operations and writing never stops with video surveillance.

 

[danlorenzo]

Ok I understand about unchecking versus setting the time out of bounds. I will look at writing certain cameras to certain drives rather than moving them all through each storage drive. I never thought about doing it that way. It will require me to think a bit more about the camera to drive mapping. I am guessing that rebuilding the database allows BI to map the location of each file, regardless of which drive/folder that it is in. I just never thought that much about the database mapping.

 

[sebastiantombs]

I do that by listing the cameras and bit rates, some are 2MP and some are 4MP, and balance the total bits to each drive while keeping cameras from the same area on different drives. That way if a drive fails, at the wrong time, you don't lose every camera on, say, the front of the house for example.

 

[danlorenzo]

So I partitioned the folders into a subfolder per camera. I went ahead and moved files for the older data also. I was surprised to see that a full database rebuild went from about 9 hours to 3-4 hours. Unless I did something else, that seemed to have made the rebuild/regen process a fair amount faster. I did various other cleanups prior to the deletion, and now do not seem to have any more errors in the database. So, now I seem to be good. I will continue to refine the DeepStack, and ponder on the camera per folder versus the cascade approach. Thanks for all of your help.