Yet Another Free Extension for Blue Iris Adding AI Object Dectection/Reduction in False Alarms/Enhanced Notification of Activity/On Guard

[Ken98045]

There is a new release Version 1.7.3 at: Release On Guard Security Assistant Version 1.7.3 · Ken98045/On-Guard
This fixes bugs where:
1. Areas were not delete properly
2. Objects to search for in an area were not deleted properly
3. Urls to notify when motion stopped were saved but not editable.
4. A couple of other minor things.

Note that this is all related to the move from savings settings in a file vs. the registry. This has proven more difficult than I thought, but I think that in the long (and even short) term that this is worth it. I will try to be more methodical in releases in the future to reduce the number of releases. However, when I do find something that would affect many users I will release an update as soon as possible.

 

[jz3082]

In the second and third log postings above the car was moving. This was working correctly in your earlier versions so I thought it may be related to adding *any vehicle in the AIO choices. This first image below corresponds to the last 2 log entries in the previous post. The car was parked at the mailbox but is now moving toward the driveway exit. The car entered the driveway in the upper left corner and stopped at the mailbox. The mail carrier got out of the car and walked up the sidewalk on the left middle of the image. A doorbell camera identified the vehicle as a car in the success box. All 3 cameras identified the mail carrier as a person and the car but no alerts were sent for the car or person. I was able to get a person alert via MQTT which I explain below.

The next images identify the car and person. No MQTT alerts were sent for person or car.


With further testing I sent this image a second time and got an MQTT alert for the person but not the car.

Notice I received the MQTT alert for the person but still nothing for the car.

I did not run another image with the person for the SW camera and notice no MQTT alerts were received for the car or person

I then renamed that image an copied into the monitored folder expecting to see a person MQTT alert. The log is below along with the current state in HomeSeer with no notification for a person or car. I did receive a MQTT alert at 2:52:08 AM which shows I am getting some MQTT alerts.


Even if we don't consider the missing MQTT alerts for the person and the car OnGuard missed the vehicle in all instances because it thought it was parked according to the log even though there were 6 pictures of the moving vehicle from the SE camera and 9 images of the moving vehicle from the SW camera with confidence levels above the trigger values.

 

[Ken98045]

The logic for defining a car as parked needs to be worked on. It seemed to be better for lower frame rate situations. That is, if you are taking 10fps the vehicle may only move slightly between frames. At 1fps, the moves further, and therefore isn't considered parked. There may also be some outright bugs in there too.

In the longer term (probably a few days or so) I would like to examine positions over time (say 5+ seconds) to see if vehicles are moving. I'll take a look later today.

I have taken a look at this and you are correct. The logic that determines a vehicle is parked needs to be re-visited. In particular, I think that it is mandatory to look at a longer term history than just the previous frame. However, please note that this will probably never be perfect. The outlines of a car can shift slightly as the light changes. Headlights at night make this worse. Also, the outlines of a vehicle can change as people walk in front of it. The same real world object that is considered a car can appear and disappear too, largely because of lighting (car, nothing, car, nothing, truck). Thus, determining if a vehicle has moved is very tricky. For the purposes of starting a video this isn't too critical. For the purposes of email or MQTT false notifications it can be more of a problem.

I'm going to try to start taking a look at the recent history of vehicle locations to try to solve these problems. This make take a few days to a week or more depending on how many bugs are reported.

 

[Ken98045]

In the second and third log postings above the car was moving. This was working correctly in your earlier versions so I thought it may be related to adding *any vehicle in the AIO choices. This first image below corresponds to the last 2 log entries in the previous post. The car was parked at the mailbox but is now moving toward the driveway exit. The car entered the driveway in the upper left corner and stopped at the mailbox. The mail carrier got out of the car and walked up the sidewalk on the left middle of the image. A doorbell camera identified the vehicle as a car in the success box. All 3 cameras identified the mail carrier as a person and the car but no alerts were sent for the car or person. I was able to get a person alert via MQTT which I explain below.
View attachment 80650
The next images identify the car and person. No MQTT alerts were sent for person or car.
View attachment 80655View attachment 80656View attachment 80657

With further testing I sent this image a second time and got an MQTT alert for the person but not the car.
View attachment 80658
Notice I received the MQTT alert for the person but still nothing for the car.
View attachment 80661
I did not run another image with the person for the SW camera and notice no MQTT alerts were received for the car or person
View attachment 80662
I then renamed that image an copied into the monitored folder expecting to see a person MQTT alert. The log is below along with the current state in HomeSeer with no notification for a person or car. I did receive a MQTT alert at 2:52:08 AM which shows I am getting some MQTT alerts.
View attachment 80669View attachment 80670

Even if we don't consider the missing MQTT alerts for the person and the car OnGuard missed the vehicle in all instances because it thought it was parked according to the log even though there were 6 pictures of the moving vehicle from the SE camera and 9 images of the moving vehicle from the SW camera with confidence levels above the trigger values.

Please see my other post regarding parked vehicles. There may be something related to the any vehicle stuff. However, in some test photos I used it will detect a car even with "any vehicle", but more often than not it considers it falsely parked if it is in the distance even if moving. If there is an easy fix for at least a good portion of the problem I will get a fix up soon. More than likely though this is a few days work.

 

[jz3082]

The problem was seen with cars not being identified from images that are taken of the car as as it travels across the yard I understand it is difficult to detect if a car is not moving but the 3 cameras showed 16 images of the vehicle traveling about 300 feet.

I think I found something that may help with your debugging. I installed 1.7.3 and then posted the image from the SW camera with the person and car identified as qualifying objects. I received an MQTT alert for both the car and person. I then posted the image from the SE camera with the car and person identified as interesting objects. I did not get any MQTT alerts until I stopped OnGuard, started it again and posted the same picture again. It looks like everything is working correctly after it first starts. The logic may not be 100% for the vehicles but the person did not generate the MQTT alert until after the restart.

I am trying to provide as much information as I can to help you find the problem. I appreciate all of the work you have put into this.

 

[jz3082]

I forgot to mention I am still using car and truck specifically but removed them and chose any vehicle for testing.

 

[Vettester]

The logic that determines a vehicle is parked needs to be re-visited. In particular, I think that it is mandatory to look at a longer term history than just the previous frame.

One thing I've noticed is that sometimes DeepStack recognizes a single parked vehicle as both a car and a truck. When this occurs the logic count of the total number of vehicles is off so the moving vehicle doesn't trigger motion event. Here's an example that illustrates this issue. The SUV across the street is parked and was recognized as both a car and a truck.

 

[Ken98045]

One thing I've noticed is that sometimes DeepStack recognizes a single parked vehicle as both a car and a truck. When this occurs the logic count of the total number of vehicles is off so the moving vehicle doesn't trigger motion event. Here's an example that illustrates this issue. The SUV across the street is parked and was recognized as both a car and a truck.

View attachment 80672

Yes, I have seen this multiple times. There is logic in their to reduce the multiple objects into one object that has the highest confidence. As the documentation mentions the confidence of the object with the highest confidence is artificially boosted. That is because the AI seems to say: "I don't know if it is a car or truck, so I'll give both definitions with a low confidence. Since On Guard is oriented toward security cameras we care more about the fact that it is some sort of vehicle than the exact type. In future versions this may be limited only toward requests for "all vehicles". I'm not sure if that is actually better though.

 

[Ken98045]

I forgot to mention I am still using car and truck specifically but removed them and chose any vehicle for testing.

Thanks for the information.

I am looking at all the logic related to this. It is definitely over estimating that a vehicle is parked. I think that if I look at the history of objects in a location for about a minute or so I should be able to figure this out better. One of the problems is that right now it only looks back to the one prior frame. I've been working toward using the history for this purpose as well as movement direction (say vehicles coming into your driveway instead of leaving).

 

[jz3082]

I do not believe it is the history that is causing a problem in my instance. I have had 840 images posted to the alerts folder for 4 cameras over the last 24 hours.

I copied and reposted images from this morning for my test this evening. OnGuard was restarted twice since those events this morning. An alert was not sent when I posted those images from this morning. I then restarted OnGuard and posted the exact same image again changing the name but after the restart the alert is sent. After the initial alert was sent I then posted images from earlier in the day from the other 3 cameras where cars were identified with confidence levels exceeding the threshold and no alerts were sent.

What I can get to repeat is that an alert for a car is sent the first time after a restart and not again until another restart even though images and cameras where used that were different than the image and camera that triggers the first alert. Alerts for animals and people without cars are working without any issues.

Can these settings be causing the problem?

 

[Vettester]

Since On Guard is oriented toward security cameras we care more about the fact that it is some sort of vehicle than the exact type.

Understood, but my point is that On Guard did not trigger an alert for this event.

 

[Holbs]

ooof. The disadvantage of having OnGuard/Deepstack on a different machine. OnGuard was offline (it crashed) for 5 hours. I just happen to notice when neighbor vehicle drove by and I didn't get alert.
To the network folks out there... is there a way to see the services status on a different machine? My Blue Iris machine is downstairs in the network cabinet...my Main PC is upstairs. Would love to see a small indicator on my Main PC (green and/or red traffic light for example) with OnGuard and Deepstack services/server online status.
2021.01.26 13:03:17:7064 - 1 interesting objects found in file: D:\AI\AI_Front.20210126_130155611.jpg
2021.01.26 13:03:20:6216 - MainWindow - AddMotionFramesTable SQL Exception - Violation of PRIMARY KEY constraint 'PK__tmp_ms_x__C1BC802ADB9BBDC9'. Cannot insert duplicate key in object 'dbo.tblMotionFiles'. The duplicate key value is (132561685156160921).
The statement has been terminated.
2021.01.26 19:57:54:1245 - On Guard started at: 1/26/2021 7:57:54 PM

 

[OldBobcat]

I do not believe it is the history that is causing a problem in my instance. I have had 840 images posted to the alerts folder for 4 cameras over the last 24 hours.

I copied and reposted images from this morning for my test this evening. OnGuard was restarted twice since those events this morning. An alert was not sent when I posted those images from this morning. I then restarted OnGuard and posted the exact same image again changing the name but after the restart the alert is sent. After the initial alert was sent I then posted images from earlier in the day from the other 3 cameras where cars were identified with confidence levels exceeding the threshold and no alerts were sent.

What I can get to repeat is that an alert for a car is sent the first time after a restart and not again until another restart even though images and cameras where used that were different than the image and camera that triggers the first alert. Alerts for animals and people without cars are working without any issues.

Can these settings be causing the problem?

View attachment 80684

View attachment 80685

I think I’m seeing this same thing. When I first start OnGuard I get one notification of a car, but then no more. If I review and analyze the images with vehicles they show that the vehicle was recognized, just no notification.

It does seem that I still get notifications if a person passes by.

 

[Vettester]

To the network folks out there... is there a way to see the services status on a different machine?

I've used NetXMS to monitor windows services in the past. It worked well and it is free. I know there are others out there that may be more popular. (Spiceworks, Nagios, etc.)

NetXMS — enabling networks. Highly customizable, scalable, secure.

Open-source network and infrastructure monitoring and management system — a product of choice for banking, Telcos, MSPs and production lines. Enterprise edition.

www.netxms.org

 

[Holbs]

I've used NetXMS to monitor windows services in the past. It worked well and it is free. I know there are others out there that may be more popular. (Spiceworks, Nagios, etc.)

NetXMS — enabling networks. Highly customizable, scalable, secure.

Open-source network and infrastructure monitoring and management system — a product of choice for banking, Telcos, MSPs and production lines. Enterprise edition.

www.netxms.org

ah thanks. So..it is possible. Will look into it

 

[Ken98045]

I think I’m seeing this same thing. When I first start OnGuard I get one notification of a car, but then no more. If I review and analyze the images with vehicles they show that the vehicle was recognized, just no notification.

It does seem that I still get notifications if a person passes by.

Yes, I am trying to filter out parked cars since cars are interesting when moving, but not so much when they are just sitting there. I am actively working on making this more reliable. When you review the picture in the UI they will still be shown as interesting if you use the Analyze button. Filtering out parked cars is necessary because you don't want an alert for car movement in your driveway if the car was just sitting there and a dog wandered through the scene causing motion to occur and the frame to be re-analyzed in real time.

 

[Ken98045]

Understood, but my point is that On Guard did not trigger an alert for this event.

Yes, I am working on this as a top priority.

 

[Holbs]

Ken.. should migrate this into Blue Iris somehow in the future Would become huge like UI3 has become. Though it has flaws, it works enough to give me peace of mind with all this tire slashing event stuff I have going on, mostly. And is cool you keep at it to make it better.

 

[Ken98045]

I do not believe it is the history that is causing a problem in my instance. I have had 840 images posted to the alerts folder for 4 cameras over the last 24 hours.

I copied and reposted images from this morning for my test this evening. OnGuard was restarted twice since those events this morning. An alert was not sent when I posted those images from this morning. I then restarted OnGuard and posted the exact same image again changing the name but after the restart the alert is sent. After the initial alert was sent I then posted images from earlier in the day from the other 3 cameras where cars were identified with confidence levels exceeding the threshold and no alerts were sent.

What I can get to repeat is that an alert for a car is sent the first time after a restart and not again until another restart even though images and cameras where used that were different than the image and camera that triggers the first alert. Alerts for animals and people without cars are working without any issues.

Can these settings be causing the problem?

View attachment 80684

View attachment 80685

This is probably all related to the parked car problem (I think). You can turn on "Log Detailed Information" and maybe you can get a better idea of what is going on. There is a lot of spew when this is on, and not everything captured in the log will necessarily make sense to you, but if you have the patience to work through it it may help.

Since I am trying to actively work on the parked vs. moving car problem I don't have the time to work through it for you. Once I feel that the parked car stuff is under control I'll work with you on it if it is still happening.

 

[wittaj]

Ken.. should migrate this into Blue Iris somehow in the future Would become huge like UI3 has become. Though it has flaws, it works enough to give me peace of mind with all this tire slashing event stuff I have going on, mostly. And is cool you keep at it to make it better.

Especially since Sentry has seemed to drop the ball...when was the last time someone posted here that it works right or they get a return email from them?