IP Cam Talk Custom Community DeepStack Model
- Thread starter MikeLud1
- Start date
At this time I am keep both updated
@MikeLud1 asked me to publish performance data using the new 'combined.pt' custom model using the methodology I published in August.
ipcamtalk.com
This post compares processing times for Blue Iris running Deepstack:
Please note that my observations are system-specific. My relevant system specs:
'COMBINED.PT' CUSTOM MODEL
Full vertical scale
Total data points: 1275
Points >250 msec: 110 (9.4%) ... these points were discarded for the statistical analysis
0-250 msec vertical scale + stats
Total data points: 1165
Performance: 102 ±19 msec (1 stdev)
Note: on the 1st day of the study I forgot to configure all profiles to use the custom model.
DS CORE 'OBJECTS' MODEL
Full vertical scale
Total data points: 2216
Points >250 msec: 289 (15%) ... these points were discarded for the statistical analysis
0-250 msec vertical scale + stats
Total data points: 1927
Performance: 142 ±18msec (1 stdev)
DeepStack Case Study: Performance from CPU to GPU version
I'm posting this thread for anyone considering making the transition from the DeepStack CPU version to the GPU version. Hopefully this will give you an impression of what you can expect in terms of enhanced performance. Please note some of my observations are system-specific. My system specs...
This post compares processing times for Blue Iris running Deepstack:
- with only the 'combined.pt' model for 1 week
- with only the DS core 'objects' model for the prior week.
- DS core 'objects' model performance: 142 ±18msec (1 stdev)
- 'combined.pt' model performance: 102 ±19 msec (1 stdev) ... 29% faster
- Indistinguishable results for the 'combined.pt' model v2 vs v1
Please note that my observations are system-specific. My relevant system specs:
- I7-4770 processor
- 16 GB RAM
- PNY NVIDIA Quadro P400 V2
- 9 x 2MP cameras, all continuously dual streaming
- EDIT: All cameras are sending sub stream resolution D1 (704x576) to DeepStack.
- 5/9 cameras using DeepStack (all Dahua and triggered via ONVIF using IVS tripwires)
'COMBINED.PT' CUSTOM MODEL
Full vertical scale
Total data points: 1275
Points >250 msec: 110 (9.4%) ... these points were discarded for the statistical analysis
0-250 msec vertical scale + stats
Total data points: 1165
Performance: 102 ±19 msec (1 stdev)
Note: on the 1st day of the study I forgot to configure all profiles to use the custom model.
DS CORE 'OBJECTS' MODEL
Full vertical scale
Total data points: 2216
Points >250 msec: 289 (15%) ... these points were discarded for the statistical analysis
0-250 msec vertical scale + stats
Total data points: 1927
Performance: 142 ±18msec (1 stdev)
Last edited:
Part II compares confidence levels for Blue Iris running Deepstack:
My system specs: Please see the previous post (#223).
My camera DeepStack settings for the study:
Driveway cams (2 total)
Notes:
Entry cams (3 total)
Notes:
'COMBINED.PT' CUSTOM MODEL
Total data points: 824
Confidence (all data points): 74 ±14% (1 stdev)
DS CORE 'OBJECTS' MODEL
Total data points: 1268
Confidence (all data points): 72±16% (1 stdev)
- with only the 'combined.pt' model for 1 week
- with only the DS core 'objects' model for the prior week.
- Indistinguishable results for the 'combined.pt' model vs the DS core 'objects' model
- DS core 'objects' model performance: 72±16% (1 stdev)
- 'combined.pt' model performance: 74 ±14% (1 stdev)
My system specs: Please see the previous post (#223).
My camera DeepStack settings for the study:
Driveway cams (2 total)
Notes:
- I do not confirm cars (too many) - just large vehicles.
- I analyze every 200 ms because objects move faster than my 'entry' cams (below)
Entry cams (3 total)
Notes:
- I include cell phones as I've noticed they are a good proxy for people.
- I've attempted to use 'mouse' as a possible proxy for critters smaller than a cat (skunks, possums, squirrels). It's never worked yet.
'COMBINED.PT' CUSTOM MODEL
Total data points: 824
Confidence (all data points): 74 ±14% (1 stdev)
DS CORE 'OBJECTS' MODEL
Total data points: 1268
Confidence (all data points): 72±16% (1 stdev)
Last edited:
sebastiantombs
Known around here
I'm wondering if "zebra" in "to cancel" isn't effecting things. I don't think there is a zebra in the combined model unless Mike added it recently. Does DS keep looking for an object that doesn't exist under these circumstances is the question.
BI will keep on sending images to DS because BI does not know what labels (objects) are in the model
sebastiantombs
Known around here
So even if DS detects an object from the "to confirm" list it still keeps analyzing. I'd have thought it would stop once the "confirm" criteria was met.
My mistake, you are correct it would stop once confirmed.
That's why I was asking. So adding an object to the Cancel field that doesn't exist, serves no purpose.
Nope. It helps in certain fields of view, and it helps get a better alert image.
At night, my one camera, that has a straight on angle of the street to get a side profile of a car, would either find a car but the alert image would be the lightshine on the street or just a part of the vehicle, or it would trigger out nothing found due to headlight bounce off the street.
Once I added a cancel banana in the field, it now will go thru all the images and select the best one, which gives me the whole vehicle in the frame. It makes for scrubbing video much quicker as I can skip looking at video of known vehicles.
scp3000
n3wb
Well I must say after reading this thread from page start to finish, I thoroughly enjoyed the thread very constructive and a lot of work by Mike have made a few changes now to my BI be interesting to see what happens I hope I have understood it correctley I have put in (objects:0,combined,animal) for daytime and (objects:0,dark,combined,animal) for nightime, is this correct ? again great thread cheers guys..Ricky
If you are using combined all you need is (objects:0,combined) for daytime and (objects:0,dark,combined) for nightime all of the new animals are in the combined model. In the next several days I am going to add the dark images to the new models so the new models will not need dark any more.
Last edited:
I'm liking adding "banana" to the cancel field. It's doing exactly as you said by sending all 10 images to deep stack and using the one with the highest percentage. I have a feeling that if I had this setting the other day, it would have identified the deer as a deer instead of a bird.
scp3000
n3wb
Thanks for that any idea when I play back deepstack log I get three frames around all detected items like my cars myself etc ?
Attachments
I use it for the same reason as @wittaj in post #231.
I may not need it for my non-street cams, but I’ve not run a study.
I chose ‘zebra’ on the off chance the DS software looped through the labels in alphabetical order and it might matter.
In my experience it does delay the confirmation by a second or two.
I do not use the cancel option on my time critical cams.
I started working on adding the images from the dark custom model to the new models.
One question is should I add them as a separate label for example (car and Car) or should I combine them into one label (car)?
My thought is to have two separate label because the image count for each label in the dark model is low compared to new models and if I add them under one label it would be just noise for that label and degrade the accuracy for both.
One question is should I add them as a separate label for example (car and Car) or should I combine them into one label (car)?
My thought is to have two separate label because the image count for each label in the dark model is low compared to new models and if I add them under one label it would be just noise for that label and degrade the accuracy for both.
Last edited:
sebastiantombs
Known around here
I'd say "car" and "Car" or even "car" and "Vehicle". That way you could add trucks as well and just use a single label.