Foscam is creating huge ".dat" files. Screens attached.

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
I have 9 cams set up and operating very well. We store images only and only if motion is detected at our premise. All images go neatly in their respective folders with date and camera name etc. Everything is very clean and running super smooth. HOWEVER the software seems to be leaving "dirt" behind. The camera folder contains hundreds of small and sometimes very huge .dat files. See image attached. I deleted the huge files of 7GB but they are created right away and keep growing very slowly.

See screenshot attached. Please help. Thank you.

Clipboard03ca.jpg dfswa.png
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
Those are the database files. They are not 7GB the are tiny files...i dont see any large files on that list.....just leave them...there is no need to delete them.
 

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
Thank you for replying. I mentioned that I deleted the 7GB and a new files has been created and is growing at 50kb per second.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
What version of blue iris are you running? Something is wrong, i just checked my system and the dat file is 66mb...it should not get to 7gb...but even 7gb is nothing today.....
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
Go to options >clips and archiving and make sure compact and repair database is checked. In the main view, right click on the clip view on the right and select database first repair and regenerate it then compact it.

Bump to the newest version and see if it helps...maybe others can chime in about their database file size.
 

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
sasdsa.pngClipboard03.jpg

Yeah this is getting ridiculous. The files are getting bigger everyday and Blue Iris is getting slower. Take a look at the screenshots. How do I disable clippings and the creation of these files. All I want for the software to do is simple save images when motion is triggered in the specific folder. It does it greatly however it leaves a lot of trash behind. Can someone help please!
 

icerabbit

Getting the hang of it
Joined
Apr 14, 2014
Messages
583
Reaction score
74
Location
FL <~> ME
Note that you should let the software do what it is designed to do. Most of those files you have shown are tiny KB and MB sized. You show one single GB file now. In the original shot the largest files were MB size.

What are your motion settings?
How much motion activity do you have?
How long are you retaining video clips for?
How many GB or TB of video data is stored?

As that last images indicates >800000 video clips, if I see that correctly.

If so, no wonder you'd have a bit of a database file to keep track of all those clips and the system runs consequently slower. The database will grow relative to how many data points and clips there are. Few files = small db. as you add more data, it grows. Completely normal.

That's not to say you can't have a bug or anything, but you may need to do a thorough settings review first.

And what are you recording? Motion in a commercial setting (supermarket, 24hr gas station, ...) or residential looking outside can be semi continuous motion where all the cameras record people walking through a store or see leaves and clouds move, and each time record 10sec clips, pretty constantly ... thats a lot of clips over the course of a day, week, month. So to keep the database and number of files smaller, longer clips or some changes towards more continuous recording on some cameras may be better.
 
Last edited by a moderator:

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
Did you repair the database? Blue iris should be saving the database files in a folder called db..yours is not and maybe that is why its not deleting the old files. Make sure you are using the latest version of the software.
 

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
View attachment Processing Entrance 20140826 91059.bmpView attachment Processing Entrance 20140826 91051.bmp

Thank you for replying. My motion settings are attached. I hope that helps.

It is true that the files were tiny but some of them grow out of control to several gigabytes. I have deleted them all yesterday. It has no effect on anything. All my pictures which were taken are still stored nicely in their folders.

It is a clinical processing plant. We have motion in some areas all day long and in other areas we rarely have motion. The thing is that our processing plant is huge and sometimes staff are not visible on one camera for several hours. That is why I can't have the cams record 24h every 10 seconds. I just need the images when motion is there. Which works very fine, I am actually amazed that is works so smoothly.

I have no intention and I also have no use for VIDEO clips. I thought I disabled the video clips. If these huge files are video clips then this might be the problem.

For one camera I get approx 50 MB of images of motion a day, which is pretty small and all I need. So if I have 10 cams x 50MB = 500 MB a day. All images are stored in subfolders and sorted by dates.

If BlueIris is trying to add these pictures to its own "viewer" or "archiver" then I need to stop it. The files are on an external server drive and they should stay there.
 

icerabbit

Getting the hang of it
Joined
Apr 14, 2014
Messages
583
Reaction score
74
Location
FL <~> ME
Also, have you emailed the developer of blueiris?

I understand you only want to record still images, which is fine, but somehow BI treats what you have as a setup as close to 1 million clips. And so you maybe pushing something the software is not optimzed for.

To me the snapshot every three min is a bit suspect, but I don't use BI.

I can only suggest to contact the developer, maybe even link him to thisbthread for the discussion and images to bring him up to speed.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
Blue iris is not deleting the old database files because there is something wrong with you folder structure. The .dat files should be in the DB folder. Like i said try repairing the database. You may have to reinstall.
 

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
fdfsdfsd.png

Fenderman, the database in in the folder and everything is in its subfolders, I store the images inside "New". However even if I click on repair database it uses 12% of my CPU and scans through images and it keeps counting but after a while it simply restarts. How can I make it stop and simply focus on taking images and saving them.
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,907
Reaction score
21,290
put the database in its own folder called "db" that is the default and maybe that is causing your issues. The database is not storing images...but for some reason its not deleting old files...try the folder, if it doesnt work then either email support or reinstall.
 

bp2008

Staff member
Joined
Mar 10, 2014
Messages
12,681
Reaction score
14,043
Location
USA
I recommend you:

1. Close Blue Iris. If it is running as a background service, stop the service also.
2. Move, rename, or delete your "cam" folder (the one with Alerts, New, and Stored inside it).
3. Open Blue Iris.

This will start you fresh with a new database and 0 clips. The 800,000 clips you have already are going to be a serious slowdown and make the database huge. My otherwise speedy system got sluggish with just 30,000 clips. If you want, we can add a step 4:

4. Configure Blue Iris to delete old recordings after some period of time. 1 to 6 months or whatever you are comfortable with. This would help keep things running more smoothly.
 

uzai

n3wb
Joined
Jun 16, 2014
Messages
9
Reaction score
0
bp2008 I did what you recommended and indeed the whole database is new and the clips started to appear on the rigth side of BlueIris the way I have never seen it before. Thank you.

Now I believe that the data is for the images on the right side. Kind of a buffer or cache. How can I disable that? It is using CPU and wasting unnessesary hard disk space. One does not need a picture viewer for the pictures I am taking. I can simply go to the folder itself and browse.

Does anyone know how to disable the right side preview thing?fewrgfwrg.png
 

bp2008

Staff member
Joined
Mar 10, 2014
Messages
12,681
Reaction score
14,043
Location
USA
I don't think you can disable that. Is that clip-creation rate normal? If that keeps up, it may not be long before your clip database starts causing problems again.

Maybe if you tweak the Record settings for each camera, you can get the desired behavior without much overhead in the clip database?
 

icerabbit

Getting the hang of it
Joined
Apr 14, 2014
Messages
583
Reaction score
74
Location
FL <~> ME
Which goes back to emailing the developer. There may be some room for optimization do deal with his usage scenario, as in purging the database older than nn days but keeping the jpgs/clips, not having any previews, ...

But, it sounds like the OP basically wants:
- motion triggered jpgs
- stored in camera folders on a drive
- long term
- no database
- only review them as needed in Windows Explorer

... at which point you really don't need surveillance software anymore.

You can just let the camera do that and write them to a folder on a shared drive, somewhere.
 
Top