Yet Another Free Extension for Blue Iris Adding AI Object Dectection/Reduction in False Alarms/Enhanced Notification of Activity/On Guard

[silencery]

Hey, just found this and gave it a test drive. Seems to work quite nicely! Thanks for putting this out there. I plan to give it a longer term trial to see how things go.

If you're open to feature requests, here are a few off the top of my head:

• Add support for MQTT protocol in order to connect with home automation platforms (although I'm using legacy URL calls, so it works as is for me for now)
• Personally I use pushover for mobile notifications, but I have it setup so my home automation handles the messaging actions (i.e. send GIF to phone when someone is at the door).
• Ability to identify specific people (i know this is probably a big ask. It appears from the DeepStack docs that it supports this)?

 

[Ken98045]

Hey, just found this and gave it a test drive. Seems to work quite nicely! Thanks for putting this out there. I plan to give it a longer term trial to see how things go.

If you're open to feature requests, here are a few off the top of my head:

• Add support for MQTT protocol in order to connect with home automation platforms (although I'm using legacy URL calls, so it works as is for me for now)
• Personally I use pushover for mobile notifications, but I have it setup so my home automation handles the messaging actions (i.e. send GIF to phone when someone is at the door).
• Ability to identify specific people (i know this is probably a big ask. It appears from the DeepStack docs that it supports this)?

I have thought a little about MQTT. I'll look into it further. I'll put at least exploring the idea on the TODO list.
If you want pictures on your phone you can use the email feature and use your phone number plus your carrier's email address. There is a carrier helper in the Tools/Add/Edit Email addresses menu. The only disadvantage of this is that some carriers are rather slow to put MMS messages through. You can also use IFTTT to trigger home automation actions by sending it an URL. IFTTT actually works pretty well now. For years it was so slow it was unusable for me.
I'm not considering using face recognition for the foreseeable future. While in theory making the call to Deepstack should be pretty easy, I'm not anxious to take on the supporting UI to make this user friendly.

 

[silencery]

Thanks for the reply. Makes sense to me

Yes, email works. In my specific case, mqtt opens up a whole range of possibilities for further integration, but URLs are a good workaround for now.

Just thought of one more:
Add ability to identify negative activity (i.e. turn off a home automation sensor when people have left the frame). Don't know if this is possible within the current architecture

 

[nibblerrick]

Just to say @silencery is not alone: I would like to see mqtt, too. I'm using pushover, too, so that would also be a nice add, but mqtt is more relevant I think. And of course: thanks for the work!

 

[Ken98045]

Thanks for the reply. Makes sense to me

Yes, email works. In my specific case, mqtt opens up a whole range of possibilities for further integration, but URLs are a good workaround for now.

Just thought of one more:
Add ability to identify negative activity (i.e. turn off a home automation sensor when people have left the frame). Don't know if this is possible within the current architecture

 

[MolsonB]

This app is what I'm looking for! To have mult areas defined in 1 picture, which can call different MQTT (I'll use URL's for now). Thanks !

Example...
-Detect motion on the road = Turn lights on low
-Detect motion by house / doors = Turn lights on full

 

[Ken98045]

OK, I've looked a little at MQTT. I am thinking about publishing something like: "OnGuard/YourCameraName/YourZone/FoundObjectType. Do you want me to pass file name (I assume yes), confidence (?), anything else?

 

[Ken98045]

Do you want On Guard to connect to a third party server/broker or do you want me to run one locally? At this point I think I'll start with publishing the activity. Your client can subscribe to messages. However, I don't know what the "standard" practice for something like this is. I sort of assume that there may be free server/brokers you are already using.

 

[Ken98045]

Negative activity is a little tricky. For instance, if a car pulls into your driveway and sits there the object/car will still be recognized. That isn't a problem if there isn't motion, but if some other motion happens (say a cat moving) to trigger Blue Iris images then I would get false triggers.

In order to get around this I need to see if that car is moving. That isn't just a matter of checking the object outline/size since lighting changes and cause the outline to shift a little. So, you need to check to see if the change is significant. I want to do this anyway to make sure that the object found in the image is in fact significant. I do have some code in there to do this, but right now it isn't hooked up yet. I do want to do that fairly soon. I'll look into making that an event type, so negative motion is possible, just not within the next couple of weeks.

 

[fenderman]

Do you want On Guard to connect to a third party server/broker or do you want me to run one locally? At this point I think I'll start with publishing the activity. Your client can subscribe to messages. However, I don't know what the "standard" practice for something like this is. I sort of assume that there may be free server/brokers you are already using.

BI suggests Download in the help file so that may be a good starting point.

 

[silencery]

I am thinking about publishing something like: "OnGuard/YourCameraName/YourZone/FoundObjectType. Do you want me to pass file name (I assume yes), confidence (?), anything else?

That would be great, thanks! Yes, I think this would be a good start. This would already cover my own use case, and I believe it should cover 80% of users. The only thing I would MAYBE add is to also send the binary data (the image file)? I don't use that at the moment, and I wouldn't want it if there was a high performance hit, but I could imagine some really neat possibilities.

Do you want On Guard to connect to a third party server/broker or do you want me to run one locally? At this point I think I'll start with publishing the activity. Your client can subscribe to messages. However, I don't know what the "standard" practice for something like this is. I sort of assume that there may be free server/brokers you are already using.

I can't speak for anyone else, but I host my own MQTT broker locally so no need to include one. I guess this is the case for most people?
In the architecture for most other applications, MQTT is a client treated as another connectivity option (similar to email/sms) in each app, and it's meant to connect to a server.

Here's an example of the config page from tasmota which represents a pretty typical MQTT client setup

 

[Ken98045]

Negative activity is a little tricky. For instance, if a car pulls into your driveway and sits there the object/car will still be recognized. That isn't a problem if there isn't motion, but if some other motion happens (say a cat moving) to trigger Blue Iris images then I would get false triggers.

In order to get around this I need to see if that car is moving. That isn't just a matter of checking the object outline/size since lighting changes and cause the outline to shift a little. So, you need to check to see if the change is significant. I want to do this anyway to make sure that the object found in the image is in fact significant. I do have some code in there to do this, but right now it isn't hooked up yet. I do want to do that fairly soon. I'll look into making that an event type, so negative motion is possible, just not within the next couple of weeks.

Also, cars/trucks are a little tricky. Blue Iris can shift between recognizing an SUV as a car or as a truck. Simple lighting changes can shift that definition, along with a considerable shift in outline. Often you will see a vehicle in an image recognized as both a car and a truck. The car/truck definition can have a considerably different outline/size. My only way around this is to look at the size/location and determine if the object is a dual definition as defined by an overlapping outlines.

 

[Ken98045]

That would be great, thanks! Yes, I think this would be a good start. This would already cover my own use case, and I believe it should cover 80% of users. The only thing I would MAYBE add is to also send the binary data (the image file)? I don't use that at the moment, and I wouldn't want it if there was a high performance hit, but I could imagine some really neat possibilities.



I can't speak for anyone else, but I host my own MQTT broker locally so no need to include one. I guess this is the case for most people?
In the architecture for most other applications, MQTT is a client treated as another connectivity option (similar to email/sms) in each app, and it's meant to connect to a server.

Here's an example of the config page from tasmota which represents a pretty typical MQTT client setup

View attachment 76345

In the grand scheme of things sending binary image data is cheap. DeepStack is so slow that the data is instant in comparison. However, I think I'll start with just the notification of activity. I think I can get that going within 2 - 4 weeks. It doesn't look too hard, but since I haven't used it before I want to be sure I do it right. Everything looks easy from the outside, but....

 

[tripp396]

In the grand scheme of things sending binary image data is cheap. DeepStack is so slow that the data is instant in comparison. However, I think I'll start with just the notification of activity. I think I can get that going within 2 - 4 weeks. It doesn't look too hard, but since I haven't used it before I want to be sure I do it right. Everything looks easy from the outside, but....

Forgot to respond here. I like this way of detecting things. I have a specific use case for multiple zones on one camera so that’s great. I’ll tack on that MQTT would be great. Also I think that telegram would suit my needs better than email but I’m just one user.

Keep up the great work! Can’t wait for this to be the daily driver for me

 

[tripp396]

Forgot to respond here. I like this way of detecting things. I have a specific use case for multiple zones on one camera so that’s great. I’ll tack on that MQTT would be great. Also I think that telegram would suit my needs better than email but I’m just one user.

Keep up the great work! Can’t wait for this to be the daily driver for me

replied to the wrong post, whoops. Meant to reply to one post up

 

[Ken98045]

Thanks for the reply. Makes sense to me

Yes, email works. In my specific case, mqtt opens up a whole range of possibilities for further integration, but URLs are a good workaround for now.

Just thought of one more:
Add ability to identify negative activity (i.e. turn off a home automation sensor when people have left the frame). Don't know if this is possible within the current architecture

That would be great, thanks! Yes, I think this would be a good start. This would already cover my own use case, and I believe it should cover 80% of users. The only thing I would MAYBE add is to also send the binary data (the image file)? I don't use that at the moment, and I wouldn't want it if there was a high performance hit, but I could imagine some really neat possibilities.



I can't speak for anyone else, but I host my own MQTT broker locally so no need to include one. I guess this is the case for most people?
In the architecture for most other applications, MQTT is a client treated as another connectivity option (similar to email/sms) in each app, and it's meant to connect to a server.

Here's an example of the config page from tasmota which represents a pretty typical MQTT client setup

View attachment 76345

I do have basic MQTT working locally. The vast majority of that work was choosing what library to use for the MQTT interface and getting a general feel for how things work.

It would be very helpful to know how people are using MQTT so I can make sure that I meet your needs. Just out of general curiosity I'd be interested to know what kind of devices you are working with.

Do you have a need for an MQTT cool-down period? Right now I think that this may be useful, but maybe with the way MQTT works that isn't too important.

I am working on some additional enhancements I'd like to add along with MQTT. This includes ruling out static/non moving objects (mostly vehicles).

With any luck I'll try to put out a new release next week.

 

[MolsonB]

So for me, I'm using Home Assistant to handle the MQTT messages and actions. From one camera, I want to trigger different area's on the screen. Example, if it detects motion on the road, and I'm not home, then turn on the outdoor lights on low. If it detects something in driveway, turn on the lights brighter and send an alert to my phone. Detects motion at doorway, sound alarm.

I don't think cool-down period is necessary with mqtt, but I'm not sure what others will use it for.

 

[silencery]

On a high level, I use MQTT as a universal bus to communicate between platforms.

In practice, that means I connect my home automation server to send updates to devices including cameras, switches, wall mounted tablets, etc and vice versa (i.e. if a tablet detect motion, send activity to home automation server and do something).

As for cooldown, I agree with @MolsonB. MQTT is designed to be extra lightweight, so it can handle the noise. For instance, my network includes PIR motion detectors which generate a lot of chatter. If anything, you could maybe use similar cooldown rules as what you already have for email, but it wouldn't be a must-have.

 

[Ken98045]

Thanks for the reply. Makes sense to me

Yes, email works. In my specific case, mqtt opens up a whole range of possibilities for further integration, but URLs are a good workaround for now.

Just thought of one more:
Add ability to identify negative activity (i.e. turn off a home automation sensor when people have left the frame). Don't know if this is possible within the current architecture

So for me, I'm using Home Assistant to handle the MQTT messages and actions. From one camera, I want to trigger different area's on the screen. Example, if it detects motion on the road, and I'm not home, then turn on the outdoor lights on low. If it detects something in driveway, turn on the lights brighter and send an alert to my phone. Detects motion at doorway, sound alarm.

I don't think cool-down period is necessary with mqtt, but I'm not sure what others will use it for.

If all you want is lighting control then the contents of the data I publish to MQTT isn't too important. Right now (locally) I'm just passing the motion file name as the payload. I assume that you just need the fact that it was published, not what was published. If people really care about the contents, I could pass things like confidence and the binary picture data. IF I was going do do that I'd need the payload in a format that people could use. XML seems like a likely format, but that would require parsing on the part of a client. I'm open to suggestions, but I'll work on just getting the event published and worry about the rest later.

An example of a cool down period might be if people want to do things like sound a chime in the house if there is someone coming to the door. You wouldn't want the chime to keep going off constantly. Similarly, you don't want 6 notifications going to your phone for 1 person walking up to your door. I'll think more about that. The cool down could be the same for all zones, or it could be specific to a zone. That is, the driveway may have a cool down of 5 minutes while door activity might be 1 minute. However, it would obviously be easier if it were the same for all zones.

 

[silencery]

Good questions and points.

Ultimately, my goal is to find a reliable way to use face detection to do more advanced automations such as setting home occupancy, recording when unknown people/vehicles come by, or managing locks. That's more a long term item though, so yes, you're right, in my own use case I don't care what the content is, just that something was published.

For the cooldown question, yes, that is definitely a great point too. Only speaking for myself, my home automation platform (openHAB) has rules in place to handle de-bouncing triggers, so I don't mind if there's a lot of noise.