A prototype for alerting based on object detection

Alan KK

n3wb
Joined
Jan 4, 2020
Messages
3
Reaction score
3
Location
Washington, DC
I have a solution for object detection in videos I'd like to put in front of you all. I have been very frustrated with the false positives in my home 2-cam + BI setup. I can't seem to tune the triggers to account for wind, rain, the sun behind clouds and other things like that. My alerts are basically useless because of all the false positives.

Recently, I have discovered Amazon's Rekognition and Google's Vision AI services that do image recognition in videos and images. Both of these return labels such as "Car", "Person" and "Animal" when they scan images. I thought I could generate alerts only when objects I care about are detected. I do a little bit of software development and I know the AWS cloud services pretty well and decided to give it a go. What I ended up with is pretty useful to me and may be useful for others as well.

What you see below is delivered to me in email 3 minutes after a Blue Iris trigger is done. It shows the trigger image, clickable to view the video, all the objects detected by Amazon and Google and their confidence, 0-100, in their object detection. The numbers in the brackets are object detection confidence for each of the images being analyzed. Since video analysis is expensive at $0.10 per minute, I set up BI to capture images every 5 seconds along with the video during an event. Using a series of images rather than just one dramatically improved the object detection for the event.

What you don't see is that both Google and Amazon detect 20-30 objects in this view even when nothing is going on. I have set up an ignore list so the normal things like "asphalt" and "trees" are ignored. Alerts are emailed to me only when there is an object detected that isn't on my ignore list.

Screen Shot 2020-01-05 at 2.23.36 PM.png


It was a fun hobby project and is useful for me. I am tending to use these more for a summary of activity like a morning report of what happened at my house last night than for actionable alerts. Nevertheless, I can now actually see what is happening rather than pouring through false positives.

I build this for myself so it is not really usable by anyone else without doing some work to make it more generic but if there is any interest, I can answer questions, share code, or just talk more about how I implemented this.

Cheers!
 

Steppi

n3wb
Joined
Dec 3, 2019
Messages
7
Reaction score
8
Location
Genova, Italy
Hello ! Congratulations on the idea .. I've been working on something like this for about a month .. I installed this script on an ubuntu server with a RX 1080ti GPU.


I am very happy because it works very well .. I receive a Telegram message every time an object is detected ..
I'd like to improve the script .. could we work on it together?
 

Alan KK

n3wb
Joined
Jan 4, 2020
Messages
3
Reaction score
3
Location
Washington, DC
Hello ! Congratulations on the idea .. I've been working on something like this for about a month .. I installed this script on an ubuntu server with a RX 1080ti GPU.


I am very happy because it works very well .. I receive a Telegram message every time an object is detected ..
I'd like to improve the script .. could we work on it together?
Thank you. It seems we have taken very different approaches to solving a similar problem, which is interesting. You have documented your project very well. I will do the same and share my githup repo when it is ready.
 

jrf

Getting the hang of it
Joined
Sep 12, 2017
Messages
169
Reaction score
93
The technology that's available now is fun to mess around with. I did this with some python scripts and the sighthound API a few years ago. Had it tuned pretty good to then go through my home automation system to handle the alerts and tracking of the same vehicle. It worked great for eliminating the false positives. Then when Sentry came out I Beta tested it and haven't looked back. I'm sure all of these will get easier to use and integrate. I actually did my own Reckognition integration back then too! Great job! Have fun.

Some old threads to read through if you are interetsed:

 
Top