Object (person) detection

iandrews

Young grasshopper
Joined
Oct 30, 2017
Messages
30
Reaction score
10
Hi,

Despite tuning different parameters etc I still get false positives, not a lot, but some none the less. Inspired by this thread Script to Detect Colour, Make and Model of Car from IP Camera I was maybe going to try and use Sighthound's Cloud API for people / face recognition. However I wasn't sure on uploading my "pictures" to them, and what there privacy policy was and if they store the images you upload and if so what they do with them.

I thought "local processing" may be better, and started to look around and came across this:

OlafenwaMoses/ImageAI

I cant say I fully understand some (well most of it, and I work in IT), but it uses Tensorflow which seems to be a AI framework originally developed by Google.

I created a new Windows 2016 VM server on my main server, and followed the instructions. Had a couple of gotchers. First I had installed the 32bit version of Python, so needed to go back and install the 64bit version. Also, think I forgot to tick the box to update PATH variables when I installed, so I needed to do that.

I created the Object Detection script and ran it but I got an error, which after googleing around it related to needing to install the C++ redistributable.

Once that was installed it all worked. Put a picture called "image2" (or something else if you modify the script for a different name) into the folder where the code is, and it analyses it, and tells you what it found, and also creates a new version of the image with boxes around what its found.

I have attached a couple of images showing the output.

As I say, I was running on VM, and only 1 cpu (of about 1000 passmark score) was assigned to it, and it took about a min to parse the photo, with about 20 to 30 secs of that being the time the script seemed to start up. I them noticed that you could give the script different speed parameters (the faster it is the less accurate / sure it is about what it detects), so selected fast, and that seemed to speed it up to about 30 secs (with most of that again being script start up time).

So, its a work in progress, but may be able to do something where when motions is detected a photo is saved, passed to the script, and it comes back with what its found, and then I can tack action on that (i.e send e-mail if person).
 

Attachments

Last edited:

bp2008

Staff member
Joined
Mar 10, 2014
Messages
12,666
Reaction score
14,006
Location
USA
Neat. If you could get that to the point it could process one 2MP frame per second or so (obviously would take a more advanced method of input than starting the script once for each frame) then it could have potential. I wonder if it could be GPU-accelerated. I wonder if its own inaccuracies would end up being a problem, too. Like if a person in shadow isn't detected, or if someone's garden gnome is detected as a person all the time.
 

iandrews

Young grasshopper
Joined
Oct 30, 2017
Messages
30
Reaction score
10
Good points re its own inaccuracies, could maybe go from 1 set of false alerts to another. Tensorflow has a GPU accelerated install (Nvidia cards), so assume as the scrip sits on top, if you have the GPU version installed it would use that.
 
Last edited:

awsum140

Known around here
Joined
Nov 14, 2017
Messages
1,254
Reaction score
1,128
Location
Southern NJ
Knowing the "horsepower" that the newer Nvidia cards have, I would expect much higher processing speeds than a CPU is capable of.
 

jrf

Getting the hang of it
Joined
Sep 12, 2017
Messages
169
Reaction score
93
Neat. If you could get that to the point it could process one 2MP frame per second or so (obviously would take a more advanced method of input than starting the script once for each frame) then it could have potential. I wonder if it could be GPU-accelerated. I wonder if its own inaccuracies would end up being a problem, too. Like if a person in shadow isn't detected, or if someone's garden gnome is detected as a person all the time.
Reading that site it looks like it can process video in real time.
 

xdq

Young grasshopper
Joined
Jun 12, 2017
Messages
48
Reaction score
19
Location
m
Great work! I have though about using Sighthound but as with a lot of things I'll never get time to do it. Have you seen Amazon's Rekognition? Amazon Rekognition – Video and Image - AWS They have a free Tier but the way I read it you only get 12 months before having to pay. Taking the system offline has the benefits of being (relatively) free but the drawback of not having the raw processing power of Amazon, Google etc.
 

jrf

Getting the hang of it
Joined
Sep 12, 2017
Messages
169
Reaction score
93
Thanks for bringing that up! It's only $1.00 per 1,000 images per month after the first free year. Which really isn't that expensive for the amount of images I upload (probably less than that number) and the decrease of false positives is DEFINITELY worth that to me.

Is it better than sighthound? No idea. I really only have the need for people and vehicle recognition. If anyone writes some code for this I'd try it out to compare both.
 
  • Like
Reactions: xdq

Rob M

n3wb
Joined
May 1, 2018
Messages
3
Reaction score
2
Hey everyone, this is one of the most valuable posts in the entire IPCamTalk site. I have been reading the forums for ages here and can tell you that anyone that is not interested in sending their pictures to "Sentry" or some other third party for analysis should implement this. The image recognition is absolutely the best, the "resnet50_coco_best_v2.0.1.h5" model is the most accurate and can identify all types of objects, from cars to trucks etc... You can limit detection to ONLY people, cars, trucks, etc... by using the "custom = detector.CustomObjects(person=True)" line to limit to persons only. You can also use the GPU version of tensorflow to implement the analysis for much faster processing. All of this is open source and free to use, it would be a shame if Blue Iris didn't implement this natively. I'm glad I got everything going, you will quite literally have a ZERO percent false rate when using the AI image recognition, if privacy matters to you, and you don't want some third party looking at your images.... this is the way to go.

EDIT: By the way, it works with Black and White (IR images) just as well as color ones.
 

jrf

Getting the hang of it
Joined
Sep 12, 2017
Messages
169
Reaction score
93
Just to update from my end I did actually build an integration with AWS Reckognition. It did not work as well for me as sighthound. While tensorflow and reckognition did Identify a lot more things...just not what I wanted. I just really cared to know people and faces and cars. Sighthound does this acceptably for me (I'm sure we would all like 100% hit ratio...). I'm sure all of this will get better and better as time goes on.
 

hayrone

n3wb
Joined
Nov 22, 2015
Messages
18
Reaction score
4
Hey everyone, this is one of the most valuable posts in the entire IPCamTalk site. I have been reading the forums for ages here and can tell you that anyone that is not interested in sending their pictures to "Sentry" or some other third party for analysis should implement this. The image recognition is absolutely the best, the "resnet50_coco_best_v2.0.1.h5" model is the most accurate and can identify all types of objects, from cars to trucks etc... You can limit detection to ONLY people, cars, trucks, etc... by using the "custom = detector.CustomObjects(person=True)" line to limit to persons only. You can also use the GPU version of tensorflow to implement the analysis for much faster processing. All of this is open source and free to use, it would be a shame if Blue Iris didn't implement this natively. I'm glad I got everything going, you will quite literally have a ZERO percent false rate when using the AI image recognition, if privacy matters to you, and you don't want some third party looking at your images.... this is the way to go.

EDIT: By the way, it works with Black and White (IR images) just as well as color ones.

Can to share your scripts and setup?
 
Top