[tool] [tutorial] Free AI Person Detection for Blue Iris

jseybert

n3wb
Joined
Aug 23, 2016
Messages
12
Reaction score
2
Still trying different settings to optimize this, but I get randomly high detection times occassionally, sometimes for hours at a time. Not sure what is causing the super high "Thread Queue Time" results. Here are an "average" log entry, and a high queueu time entry from the AITool log. I am running the VorlonCD fork of AITool on a Docker install of DeepQuest.

Average Detection:

[14.08.2020, 17:48:04.944]: DetectObjects> Starting analysis of X:\BlueIris\AI_Input\AI_LR.20200814_174804887.jpg...
[14.08.2020, 17:48:04.945]: DetectObjects> (1/6) Uploading image to DeepQuestAI Server at [14.08.2020, 17:48:06.785]: DetectObjects> (2/6) Posted in 1838ms, Received a 492 byte response.
[14.08.2020, 17:48:06.787]: DetectObjects> (3/6) Processing results...
[14.08.2020, 17:48:06.788]: DetectObjects> Detected objects:couch (66.71%), tv (99.66%), tv (92.99%), microwave (74.5%), refrigerator (46.53%),
[14.08.2020, 17:48:06.789]: DetectObjects> (4/6) Checking if detected object is relevant and within confidence limits:
[14.08.2020, 17:48:06.823]: DetectObjects> couch (66.71%) is irrelevant.
[14.08.2020, 17:48:06.850]: DetectObjects> tv (99.66%) is irrelevant.
[14.08.2020, 17:48:06.877]: DetectObjects> tv (92.99%) is irrelevant.
[14.08.2020, 17:48:06.903]: DetectObjects> microwave (74.5%) is irrelevant.
[14.08.2020, 17:48:06.929]: DetectObjects> refrigerator (46.53%) is irrelevant.
[14.08.2020, 17:48:06.937]: Save> Settings saved to C:\AI Tool 1.65\Release\aitool.Settings.json
[14.08.2020, 17:48:06.937]: DetectObjects> (6/6) Camera Living Room caused an irrelevant alert.
[14.08.2020, 17:48:06.940]: DetectObjects> 5x irrelevant, so it's an irrelevant alert.
[14.08.2020, 17:48:06.942]: DetectObjects> ...Object detection finished:
[14.08.2020, 17:48:06.943]: DetectObjects> Total Time: 2053ms (Count=9592, Min=1542ms, Max=147548ms, Avg=2982ms)
[14.08.2020, 17:48:06.945]: DetectObjects> DeepStack Time: 1838ms (Count=9592, Min=1480ms, Max=5388ms, Avg=1949ms)
[14.08.2020, 17:48:06.947]: DetectObjects> File lock Time: 56ms (Count=4018, Min=19ms, Max=213ms, Avg=59ms)
[14.08.2020, 17:48:06.950]: DetectObjects> Thread Queue Time: 0ms (Count=5606, Min=1ms, Max=144927ms, Avg=2184ms)
[14.08.2020, 17:48:08.942]: DetectObjects>

Long Detection

[14.08.2020, 10:57:26.541]: DetectObjects>
[14.08.2020, 10:57:26.542]: DetectObjects> Starting analysis of X:\BlueIris\AI_Input\AI_FrtDrv.20200814_105550277.jpg...
[14.08.2020, 10:57:26.543]: DetectObjects> (1/6) Uploading image to DeepQuestAI Server at [14.08.2020, 10:57:28.498]: DetectObjects> (2/6) Posted in 1953ms, Received a 304 byte response.
[14.08.2020, 10:57:28.500]: DetectObjects> (3/6) Processing results...
[14.08.2020, 10:57:28.501]: DetectObjects> Detected objects: person (43.22%), car (99.56%), car (64.41%),
[14.08.2020, 10:57:28.502]: DetectObjects> (4/6) Checking if detected object is relevant and within confidence limits:
[14.08.2020, 10:57:28.536]: DetectObjects> person (43.22%) is irrelevant.
[14.08.2020, 10:57:28.570]: DetectObjects> car (99.56%) is irrelevant.
[14.08.2020, 10:57:28.608]: DetectObjects> car (64.41%) is irrelevant.
[14.08.2020, 10:57:28.614]: Save> Settings saved to C:\AI Tool 1.65\Release\aitool.Settings.json
[14.08.2020, 10:57:28.614]: DetectObjects> (6/6) Camera Front Driveway caused an irrelevant alert.
[14.08.2020, 10:57:28.616]: DetectObjects> 1x not in confidence range; 2x irrelevant, so it's an irrelevant alert.
[14.08.2020, 10:57:28.617]: DetectObjects> ...Object detection finished:
[14.08.2020, 10:57:28.618]: DetectObjects> Total Time: 98271ms (Count=2331, Min=1542ms, Max=98944ms, Avg=42505ms)
[14.08.2020, 10:57:28.620]: DetectObjects> DeepStack Time: 1953ms (Count=2331, Min=1480ms, Max=4881ms, Avg=1874ms)
[14.08.2020, 10:57:28.621]: DetectObjects> File lock Time: 0ms (Count=1289, Min=20ms, Max=198ms, Avg=60ms)
[14.08.2020, 10:57:28.623]: DetectObjects> Thread Queue Time: 96197ms (Count=1050, Min=1ms, Max=96923ms, Avg=41315ms)
[14.08.2020, 10:57:28.625]: DetectObjects>

Any thoughts on what might cause this to vary so much? When there is an instance like this it ties everything up and no more alerts come in until it settles back down. Thanks for any direction anyone might have!
 

kosh42efg

n3wb
Joined
Aug 14, 2020
Messages
29
Reaction score
13
This is the Docker for Deepstack:
Code:
sudo docker run --restart=always --gpus all -e VISION-SCENE=True -v localstorage:/datastore -p 83:5000 --name deepstackgpu2 deepquestai/deepstack:gpu
Issue 1 - I was using VISION-SCENE not VISION-DETECTION. Numpty.
 

Chris Dodge

Pulling my weight
Joined
Aug 9, 2019
Messages
90
Reaction score
114
Location
massachusetts
Still trying different settings to optimize this, but I get randomly high detection times occassionally, sometimes for hours at a time. Not sure what is causing the super high "Thread Queue Time" results. Here are an "average" log entry, and a high queueu time entry from the AITool log. I am running the VorlonCD fork of AITool on a Docker install of DeepQuest.

[14.08.2020, 10:57:28.618]: DetectObjects> Total Time: 98271ms (Count=2331, Min=1542ms, Max=98944ms, Avg=42505ms)
[14.08.2020, 10:57:28.620]: DetectObjects> DeepStack Time: 1953ms (Count=2331, Min=1480ms, Max=4881ms, Avg=1874ms)
[14.08.2020, 10:57:28.621]: DetectObjects> File lock Time: 0ms (Count=1289, Min=20ms, Max=198ms, Avg=60ms)
[14.08.2020, 10:57:28.623]: DetectObjects> Thread Queue Time: 96197ms (Count=1050, Min=1ms, Max=96923ms, Avg=41315ms)
[14.08.2020, 10:57:28.625]: DetectObjects>

Any thoughts on what might cause this to vary so much? When there is an instance like this it ties everything up and no more alerts come in until it settles back down. Thanks for any direction anyone might have!
Only one file can be processed at a time by DeepStack. I'm guessing too many images are being generated so they all have to wait in line. The thread queue wait is how long it took for the previous images to be processed. Are you sure BI is set to generate images "Only when triggered"? 2000ms for deepstack is a bit long. If you could put it on faster hardware or a solid state drive that may help reduce time.
 

warbux

n3wb
Joined
Aug 16, 2020
Messages
17
Reaction score
1
Location
United States
Hi Guys, I would like to run my cameras and deepstack virtually via my ESXI server. Currently it is running on 2 e5-2690 v4 CPUs and 500GB of RAM. The CPUs do not support QuickSync (adding a single camera spikes my CPU usage on 6 cores to 50%) so I am hoping I can add a couple GTX 1080s for Blue Iris and for DeepStack AI and offload all of the compute to the GPUs. Has anyone tried this approach? If so how did it go?
 

Andygds007

n3wb
Joined
Aug 16, 2020
Messages
8
Reaction score
1
Location
US
I use Blue Iris with the BI mobile app (for receiving alerts and playback) and tested this AI tool and it's been working nicely in "day time". However there only three issues i came across:
  • When i get visitors, their cars gets parked in my front yard and if the BI motion detection triggers randomly because of the lightning or shadows(caused by weather) it automatically accepts those alerts as positive since the car is there. Naturally the AI is doing his job since it's telling you what you configured it to look for in the picture and it's BI motion detection is at fault since it's being inaccurate. (i believe cannot be done much about this).
  • AI tool does not work at all if you have "highlight" selected in triggers under motion detection in BI. <-- this might be good to put in tut.
  • Night time in black and white is.... not working at all for me. AI tool cannot seem to trigger/intercept anything in a black and white image. So i have the AI set to work only in day time for now.
1597626959019.png
 
Last edited:

meissen

Young grasshopper
Joined
Jun 28, 2017
Messages
61
Reaction score
10
Funny you mention that, Andygds007, I was coming here just to post similar about night time in black and white. I really think I need to have mine set up so that there's a daytime schedule which uses DeepStackAI and then black and white night time does regular motion detection...

The one picture it's falsely triggering on a house window saying it's a car, but then the next photo which CLEARLY has people walking by and it doesn't even see them.
 

Attachments

Andygds007

n3wb
Joined
Aug 16, 2020
Messages
8
Reaction score
1
Location
US
Funny you mention that, Andygds007, I was coming here just to post similar about night time in black and white. I really think I need to have mine set up so that there's a daytime schedule which uses DeepStackAI and then black and white night time does regular motion detection...

The one picture it's falsely triggering on a house window saying it's a car, but then the next photo which CLEARLY has people walking by and it doesn't even see them.
Glad it's not only me having this issue. As i can understand/ learn from your scenario, it does work on cars (well at least 1 of them all which is also not so accurate) but not on people while in black and white image.
 

cjowers

Getting the hang of it
Joined
Jan 28, 2020
Messages
107
Reaction score
36
Location
AUS
Hi Guys, I would like to run my cameras and deepstack virtually via my ESXI server. Currently it is running on 2 e5-2690 v4 CPUs and 500GB of RAM. The CPUs do not support QuickSync (adding a single camera spikes my CPU usage on 6 cores to 50%) so I am hoping I can add a couple GTX 1080s for Blue Iris and for DeepStack AI and offload all of the compute to the GPUs. Has anyone tried this approach? If so how did it go?
I tried briefly with a quadro P1800 or something along those lines, but didn't have any luck with using the deepstack:gpu version. not sure if anyone has succeeded, but would also like to know. I am on windows though, possibly its better on linux?

I expect you're not buying 2 x 1080s just for this? Probably for other reasons. Someone else can probably chime in, but a cheap quicksync cpu should be much cheaper and faster and less overall watt usage, especially if most your current usage is due to the cameras. Unless you're going crazy with several cameras, triggering nonstop, the AI doesn't seem too taxing comparatively. You can always run deep stack on low / med / high, and lots of BI / camera settings to better optimize the workload.
 

Village Guy

Pulling my weight
Joined
May 6, 2020
Messages
291
Reaction score
161
Location
UK
I did follow the guide but 640x480 was too crappy for me so I switched back and used full 1080p resolution.
Do you have object detection enabled? If yes, check if it is set to only trigger when an object travels more than 100 pixels. This is an issue for some camera configurations, especially if the camera's are low resolution.
 

Netwalker

Getting the hang of it
Joined
Aug 8, 2017
Messages
46
Reaction score
28
I’ve skipped the sub streams completely and have the cameras send a snapshot to my server via FTP (via private lan) for AI analysis. Works amazing and I’m down to 10% cpu (AI temporally spikes it’s to 50%) on an old i5. Recording 8 2mp cameras 24x7 using DD and now can turn on “limit decoding unless required” because all the heavy lifting is outside of BI.

Hikivision cameras were simple to setup with this because they put their snapshots in the directory exactly like bi but Dahua were harder because they upload their images to /CameraName/Year/Month/Day/Hour/minute/ and that has to get sorted with bp2008’s command line program before processing.
 

nortish

n3wb
Joined
Apr 22, 2019
Messages
2
Reaction score
1
Location
Sweden
Hi everyone! First, amazing stuff this. Great work GentlePumpkin!
The question:
Im using this with Home Assistant and would love to be able to use the flagged pictures in automations. I saw the option to send the picture with telegram, but if I can avoid using Telegram for this it would be better.
Is the pictures with the red boxes stored somewhere? If so I could just send them over somehow to my Home Assistant VM.
 

jseybert

n3wb
Joined
Aug 23, 2016
Messages
12
Reaction score
2
Only one file can be processed at a time by DeepStack. I'm guessing too many images are being generated so they all have to wait in line. The thread queue wait is how long it took for the previous images to be processed. Are you sure BI is set to generate images "Only when triggered"? 2000ms for deepstack is a bit long. If you could put it on faster hardware or a solid state drive that may help reduce time.
Thank you! That makes sense. I found one of my cameras was generating a flood of images - motion detector on the clone was set way too sensitive. Fixed that and now Thread queue time is now down to 0 most of the time, but Deepstack is still running in the 1500-2300ms range. I am running this on i7-7700K at 4.2GHz with a Toshiba RD400 SSD. Is there a way to dig into Deepstack and see exactly where the delay is coming in? Anything that could help it short of the hardware change? What is an average good time that I should be striving for?

Thanks!
 

Sintei

n3wb
Joined
Sep 19, 2018
Messages
7
Reaction score
2
Location
Sweden
Running BI 5 and AITools 1.67 preview 7 on win 10 computer.
After windows update I now get a .net error everytime I start AItools.
If I click continue AItools another dialogbox comes stating ERROR LoadCameras() failed.
I click OK AItools start and work normally, but If I restart the computer the service won't start automagically as I have to press continue and OK due to the errors.
Anyone else with the same problem?
Attached the logfile of the .net error.
 

Attachments

Joined
Aug 18, 2020
Messages
2
Reaction score
1
Location
USA
Running BI 5 and AITools 1.67 preview 7 on win 10 computer.
After windows update I now get a .net error everytime I start AItools.
If I click continue AItools another dialogbox comes stating ERROR LoadCameras() failed.
I click OK AItools start and work normally, but If I restart the computer the service won't start automagically as I have to press continue and OK due to the errors.
Anyone else with the same problem?
Attached the logfile of the .net error.
Have you checked that "Read Only" is disabled on all AI-tool related files?

I had a few strange errors which went away after disabling "Read Only" on all files.
 
Joined
Aug 18, 2020
Messages
2
Reaction score
1
Location
USA
I’ve skipped the sub streams completely and have the cameras send a snapshot to my server via FTP (via private lan) for AI analysis. Works amazing and I’m down to 10% cpu (AI temporally spikes it’s to 50%) on an old i5. Recording 8 2mp cameras 24x7 using DD and now can turn on “limit decoding unless required” because all the heavy lifting is outside of BI.

Hikivision cameras were simple to setup with this because they put their snapshots in the directory exactly like bi but Dahua were harder because they upload their images to /CameraName/Year/Month/Day/Hour/minute/ and that has to get sorted with bp2008’s command line program before processing.
Seems promising! I've had the exact same idea with my HiK cameras, due to problems with BI when it gets dark.
If it is also possible to trigger the cameras to record via a http-command/trigger, BI could be closed completely to save cpu power.
 

Chris Dodge

Pulling my weight
Joined
Aug 9, 2019
Messages
90
Reaction score
114
Location
massachusetts
Thank you! That makes sense. I found one of my cameras was generating a flood of images - motion detector on the clone was set way too sensitive. Fixed that and now Thread queue time is now down to 0 most of the time, but Deepstack is still running in the 1500-2300ms range. I am running this on i7-7700K at 4.2GHz with a Toshiba RD400 SSD. Is there a way to dig into Deepstack and see exactly where the delay is coming in? Anything that could help it short of the hardware change? What is an average good time that I should be striving for?

Thanks!
In my limited experience with the Windows version of Deepstack, I average around 600ms. Thats on a Threadripper 1950x with 16 cores, 64 GB ram and Samsung M2 drives that are like 3000mb read speed. If you had less than 16 GB ram that appears like it could be a factor since I'm seeing over 8GB ram usage in all the deepstack processes. And I'm not 100% sure but I think it tries to use all of my cores, so that certainly is a factor. I could never get it to work, but if you have a good nvidia card you could try the "GPU" version in Docker.
 

Chris Dodge

Pulling my weight
Joined
Aug 9, 2019
Messages
90
Reaction score
114
Location
massachusetts
Running BI 5 and AITools 1.67 preview 7 on win 10 computer.
After windows update I now get a .net error everytime I start AItools.
If I click continue AItools another dialogbox comes stating ERROR LoadCameras() failed.
I click OK AItools start and work normally, but If I restart the computer the service won't start automagically as I have to press continue and OK due to the errors.
Anyone else with the same problem?
Attached the logfile of the .net error.
Open each file in \Cameras\ subfolder in Notepad. If any are empty or have null's, weird charactors, etc, they are corrupt. Each file should be human readable. If not, delete them and recreate the camera.
 

Chris Dodge

Pulling my weight
Joined
Aug 9, 2019
Messages
90
Reaction score
114
Location
massachusetts
Speaking of lack of night time detection. Deepstack didn't even consider this an object - i'm surprised.

View attachment 68907
Same, things at night are almost never detected for me. And your contrast is a lot better than mine. I have a camera ordered that does color night vision if the light is 1 lux or brighter, so I'll let you know if it helps: Amazon.com : Lorex LNB9272S 4K 8MP 30FPS Vari-Focal 4X Zoom Bullet Camera w/Listen-in Audio … : Camera & Photo
 
Joined
May 27, 2020
Messages
13
Reaction score
1
Location
Idaho Springs, Colorado
Same, things at night are almost never detected for me. And your contrast is a lot better than mine. I have a camera ordered that does color night vision if the light is 1 lux or brighter, so I'll let you know if it helps: Amazon.com : Lorex LNB9272S 4K 8MP 30FPS Vari-Focal 4X Zoom Bullet Camera w/Listen-in Audio … : Camera & Photo
Interesting camera - i might have to try that as well. I just started using IR floods for better contrast and was hoping it would help with AI. It didn't. But it sure improves night vision and most importantly NO MORE SPIDERS with the the camera's LEDs turned off!!!!!
 
Top