Disk full. Disk can’t write fast enough. Is my hard drive shot?

[Jacksonspop1]

Disk can’t write fast enough, buffer overflow, disk full. What in the world is going on with my blue iris? For the last month I have been getting several red X messages in the log. The past week it has gone insane. Yesterday the computer magically turned itself off. When I rebooted it, everything is working fine. It is updated to newest critical software. I’m wandering if my hard drive is full and can’t delete the older files? Or maybe it is something else? Who all knows what I need to do next? Thanks in advance!!!!

 

[wittaj]

Best practice is to only update BI when the update includes something you want or need.

Are you flagging alerts and thus the drive is now full with protected files that will not delete on the BI delete schedule?

Have you allocated the entire HDD to BI? You need to leave some headroom (about 10%) so that there is room for the drive to write, override, and delete.

So in the example below it is a 2TB drive (2,000GB).

You don't put 2000GB there as the drive needs some "free" space for when it is full and is overwriting and saving at the same time.

So 90% of 2000GB would mean do not put more than 1,800GB in that location, and in this example, the person allocated a little extra free space and put 1,750GB

 

[Jacksonspop1]

Best practice is to only update BI when the update includes something you want or need.

Are you flagging alerts and thus the drive is now full with protected files that will not delete on the BI delete schedule?

Have you allocated the entire HDD to BI? You need to leave some headroom (about 10%) so that there is room for the drive to write, override, and delete.

So in the example below it is a 2TB drive (2,000GB).

You don't put 2000GB there as the drive needs some "free" space for when it is full and is overwriting and saving at the same time.

So 90% of 2000GB would mean do not put more than 1,800GB in that location, and in this example, the person allocated a little extra free space and put 1,750GB

i usually dont do updates unless its having some issues. ive been having BI for probaly 5 years now. yes it keeps flagged items but its on auto delete once those folders reach a certain limit. yes there should be 10% free on the hard drive. ive had to replace the hard drive a few years back. i hate to do it again. unless i need to. and i remember one time i had to repair the disk or defrag it or something like that. i dont remember exactly how i did it.

 

[wittaj]

Auto delete won't delete the flagged items, that is why they are flagged so that they are not deleted per the schedule. You have to manually delete them.

And some people have it checked to flag every alert and eventually the drive simply gets full of flagged alerts.

Post the screenshot above to confirm you have 10% available.

What drive and capacity do you have?

It might be a good idea to go thru each camera and make sure one or more isn't set to auto-flag alerts.

 

[Jacksonspop1]

Auto delete won't delete the flagged items, that is why they are flagged so that they are not deleted per the schedule. You have to manually delete them.

And some people have it checked to flag every alert and eventually the drive simply gets full of flagged alerts.

Post the screenshot above to confirm you have 10% available.

What drive and capacity do you have?

It might be a good idea to go thru each camera and make sure one or more isn't set to auto-flag alerts.

Ok. So the way I have the recordings set up is the full hour clips go to the “new” folder. After 3 days they go to “stored”. After 14 days in stored they get get deleted. Anything that gets motion on certain cams I get alerts. Those short clips go to “alert” folder. 21 days later they moved to “aux1”. After 70 days in “aux” they get deleted. I honestly don’t remember setting up a 70 day hold on those alerts in aux but I guess I did for some reason. I am currently running a “DB repair” on the storage tab. I seem to remember something like this happening a few years back and that solved one issue.

 

[wittaj]

Part of your problem is also moving all those files around on the same drive.

All you are doing is using up CPU and wearing out the drives faster moving it from one folder to another on the same drive.

The STORED drive should only be used if offloading to a NAS.

The BI database is fragile and all that moving around files is bound to cause issues.

Most of us here only put video in the NEW folder and it deletes based on storage limits.

And unless you are saving all hi-rez alerts for the thumbnails, 1GB for the ALERTS folder is more than enough.

 

[Flintstone61]

Simplify your storage scheme.
When your disk writes a stream of lets say 10 or so cameras it gets plenty busy....
If you then you " rewrite" the same 10 cameras data on a file transfer to another folder, whiles it's trying to write 10 new streams,
the drive gets at least 20 read/write requests, and they may start becoming pending.....then it may write to " virtual memory" which is " free space on a disk"
if the drive can't find space for virtual memory and/or cannot move the data as fast as the requests., this will cause some of your error messages.
And mathematically, if you hit a point in your scheme where 3 of your transfer schedules happen at the same time, that is exceeding the "OH fuck it's a SHITton" limit on the drive.
which will cause all or most of the problems, esp. when the disk starts getting bad sectors.
Maxing out your physical ram to whatever your system can handle can help store overflows of data temporarily as well.

As long as your have 24-30 days of viable storage for clips, you can let BI drop them off like @wittaj explains.

Here is a look at the way im using my storage. There isn't an absolute hard and fast rule here,
but you don't need to move anything if your hard drive(s) can hold a month of data.
I'm running 2 -8TB wd drives, and half of 1 is partitioned for daily computer backups and other junk I save....
i can share the work of streaming to 2 drives, lightening the load for the drives, SO then when they get tasked with writing a backup of C:\ or a Clone of C:\ they can handle it....

View attachment 20250222-0944-32.6916584.mp4

 

[Flintstone61]

Long story longer of why i want 4 weeks or more of storage....
When I had a BI system at the condo... with a supporting NVR for the legacy network of coax cams (about 8 cams),
I had the luxury of 5-6 weeks of storage on the NVR.
This proved useful when during the George Floyd riots,
when they broke into a local post office and stole some mail trucks and the " master keys" to the Blue boxes and apartment buildings( & multi-unit dwellings) city wide.

We had a crew visiting our mail room in the evening and using the " master key" would unlock banks of mail boxes and parse through everything.
This included the " outgoing mail", in which older folks were still mailing checks in to pay bills. These became checks which the amounts were changed and the recipient was changed.
Additionally another resident had a $5000.00 insurance check stolen and transacted upon.
We couldn't figure out WTF was going on.

One evening the thieves got spooked and made a hasty retreat, leaving a bank of mail boxes slightly askew.
A board member noticed this and called me, and I came over and started reviewing footage.
Once I saw what was going on, I sent of a Board wide text alert telling them to post notices building wide for the residents NOT to use the outbox for mail.
I sat down at the NVR console and was able to see back to 3 other events across about 5 weeks.
Including the $5000 check theft taken from Unit 318's mail box.
This footage was key in having the insurance company believing the guy wasn't scamming them or whatever,
and explained why people writing checks to Comcast were getting non-payment notices and calls from banks etc.

I went to the hardware store and put a hasp and padlock on the outgoing mail box that night.
next day I rigged up some more padlock and hasp devices over the " master key" hole so they couldn't get in.

The mother fuckers came back a few nights later and got a surprise. No Outgoing mail for you tonight JAckASSes!
This time upon exiting he was scanning for, and found/noticed all 4 cameras watching his ass. They never came back.

This fuckery was enough to have the Postmaster come and investigate the matter and issue us new " master locks" for the mail room.

so now I like to have at least 4 weeks of data....because if a pattern of crime has developed, you have a chance of catching it.
unfortunately the Blue Iris machine in the property didn't have more than about 23-25 days of storage. and wasn't able to see some of the supporting views surrounding the mail room heists.

Unfortunately, I did not have an LPR cam setup at this time. It might've given the PoPo something to go on for locating some of the crew.

 

[Flintstone61]

View attachment Video.mov

 

[jaydeel]

I can shed a little light on the Disk can’t write fast enough error.

I also see these errors occasionally. However, unlike your situation, my Purple drive is definitely not overallocated and has 10% free space. I continuously record 6 dual-stream cameras during the day, and 9 cameras at night (7 dual-stream, 2 full-stream).

Per Ken (referencing the screenshot below)...

These numbers say there was no Windows error (0), 1900+frames were being buffered waiting to be written for more than 30 seconds. The last number is a timecode difference that's not too relevant.​

So ... it does look like the disk was struggling for some reason. To prevent memory from running out and system crashing, Blue Iris closed the files and abandoned those buffers.​

Logfile entry examples:


My experience is that EVERY time this error shows up in my BI logfile, and despite Ken's comment above, a catastrophic memory leak shortly follows. And shortly after that the Blue Iris webserver becomes sluggish and eventually unresponsive. For my system, the only fix is to reboot the server. Unlike your experience, I don't think the server has ever self-rebooted.

How I know that a memory leak occurs is that I run a Powershell script from Task Scheduler that logs the system RAM usage, the Blue Iris RAM usage, and BI server uptime every 3 minutes. The chart below is an example. This event occurred after I turned in for the night and I did not catch until early in the morning. Note that after 12:30a the script stopped recording data (implying that the server hung). Also note (from the tooltip) that Blue Iris had been running a little over eight hours before the leak started. This is pretty much a worse case scenario.


To give me a heads-up on these types of events, I've created another Powershell script (bi_logfile_wdog) that interrogates all new messages in the Blue Iris logfile with specific entries (like those above) and sends a notification (Pushover or Telegram). If I receive this notification AND I have access to the server, this gives me a few minutes to access the server and verify & take action before bad sh** starts happening. Otherwise, I remotely reboot the server using an MSNSwitch.

To avoid the worse case scenario like the above chart, I'm also testing a new Powershell utility to automatically reboot the BI server when this happens. If it proves successful for future events, I'll share it.

Finally, I should note that that another logfile entry that I've discovered precipitates a catastrophic memory leak is live.sem stuck.