[tool] [tutorial] Free AI Person Detection for Blue Iris

Wrong file type by accident?
No, I learned a long time ago to check all the obvious stuff first LOL. It is saved as a png. I just thought about this. I have 2 of that model, of cameras. I only needed a mask on one of them, think I will try and throw one on the 2nd one and see f I have the same problem
 
No, I learned a long time ago to check all the obvious stuff first LOL. It is saved as a png. I just thought about this. I have 2 of that model, of cameras. I only needed a mask on one of them, think I will try and throw one on the 2nd one and see f I have the same problem
Well that din't work? 2nd camera (same model) worked fine.
 
I'm just now trying to get this set up. I'm getting the following error...

[19.09.2020, 22:48:44]: OnCreatedAsync>> Error: Skipping image because queue is greater than '100'. (Adjust 'MaxImageQueueSize' in .JSON file if needed): S:\BI_New\Entry_Clone.20200919_224844007.jpg


This is still happening after editing the .json file as follows...

"MaxImageQueueSize": 9999,

Is there another Maximum Queue setting someplace? Thanks!
 
VorlonCD mod updates:
I'm just now trying to get this set up. I'm getting the following error...

[19.09.2020, 22:48:44]: OnCreatedAsync>> Error: Skipping image because queue is greater than '100'. (Adjust 'MaxImageQueueSize' in .JSON file if needed): S:\BI_New\Entry_Clone.20200919_224844007.jpg


This is still happening after editing the .json file as follows...

"MaxImageQueueSize": 9999,

Is there another Maximum Queue setting someplace? Thanks!

That is the correct setting. You probably just need to make sure all instances of AITOOL.EXE are closed before modifying the file. But the real answer is to find some way in BI to reduce the number of images created, and to add more deepstack servers to the list. Sure you have BI set to "only when triggered"? Reduce jpeg snapshot each mm:ss ?
 
  • Like
Reactions: 105437
Hello to all and thank you to GentlePumpkin and Chris Dodge for all your work on this project. This does a fantastic job of reducing false alerts from shadows of blowing leaves that have plagued me since starting with surveillance software.

I have been using two versions of Chris Dodge's project, and have come up with an issue sending alerts to Telegram. The first version that I used would send alerts to Telegram with no issues, however, I downloaded the latest version, and it is failing to send alerts.

If I use the test button in the Camera Actions page, it will give me a "Succeeded" popup, but the alerts will error out when trying to send to Telegram.

Errors seem to indicate an issue with the version of Newtonsoft.json.dll. I have checked, and I have ~15 instances of that file on my Windows 10 machine with versions ranging from 6.x to 12.x.

Any Ideas? Following are snips from log files from the 9/3/20 build that works, and the 9/17/20 build that does not work. Thanks in advance.

From Version 1.67.8.35473 built on 9/3/2020 6:42:26 PM
[19.09.2020, 12:27:21.695]: Trigger> Uploading image to Telegram...
[19.09.2020, 12:27:21.757]: TelegramUpload> uploading image to chat "##########"
[19.09.2020, 12:27:24.256]: TelegramUpload> ...Finished in 2499ms
[19.09.2020, 12:27:24.264]: Trigger> -> Sent image to Telegram.


From Version 1.67.8.22287 built on 9/17/2020 11:22:54 AM
[20.09.2020, 11:45:00.318]: TelegramUpload> uploading image to chat "-##########"
[20.09.2020, 11:45:00.370]: TelegramUpload> ERROR: Could not upload image C:\Users\ftsadmin\AppData\Local\Temp\Prater.20200920_114457814.jpg to Telegram: Could not load file or assembly 'Newtonsoft.Json, Version=11.0.0.0, Culture=neutral, PublicKeyToken=################' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) [FileLoadException] Mod: <TelegramUpload>d__30 Line:1342:33
[20.09.2020, 11:45:00.374]: TelegramUpload> ...'Default' 'Telegram_RetryAfterFailSeconds' value was set to '300' seconds, so not retrying until 9/20/2020 11:50:00 AM
[20.09.2020, 11:45:00.406]: Trigger> Error: Could not load file or assembly 'System.Buffers, Version=4.0.2.0, Culture=neutral, PublicKeyToken=################' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) [FileLoadException] Mod: <Trigger>d__32 Line:1570:29
 
Hello to all and thank you to GentlePumpkin and Chris Dodge for all your work on this project. This does a fantastic job of reducing false alerts from shadows of blowing leaves that have plagued me since starting with surveillance software.

I have been using two versions of Chris Dodge's project, and have come up with an issue sending alerts to Telegram. The first version that I used would send alerts to Telegram with no issues, however, I downloaded the latest version, and it is failing to send alerts.

If I use the test button in the Camera Actions page, it will give me a "Succeeded" popup, but the alerts will error out when trying to send to Telegram.

Errors seem to indicate an issue with the version of Newtonsoft.json.dll. I have checked, and I have ~15 instances of that file on my Windows 10 machine with versions ranging from 6.x to 12.x.

Any Ideas? Following are snips from log files from the 9/3/20 build that works, and the 9/17/20 build that does not work. Thanks in advance.

From Version 1.67.8.35473 built on 9/3/2020 6:42:26 PM
[19.09.2020, 12:27:21.695]: Trigger> Uploading image to Telegram...
[19.09.2020, 12:27:21.757]: TelegramUpload> uploading image to chat "##########"
[19.09.2020, 12:27:24.256]: TelegramUpload> ...Finished in 2499ms
[19.09.2020, 12:27:24.264]: Trigger> -> Sent image to Telegram.


From Version 1.67.8.22287 built on 9/17/2020 11:22:54 AM
[20.09.2020, 11:45:00.318]: TelegramUpload> uploading image to chat "-##########"
[20.09.2020, 11:45:00.370]: TelegramUpload> ERROR: Could not upload image C:\Users\ftsadmin\AppData\Local\Temp\Prater.20200920_114457814.jpg to Telegram: Could not load file or assembly 'Newtonsoft.Json, Version=11.0.0.0, Culture=neutral, PublicKeyToken=################' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) [FileLoadException] Mod: <TelegramUpload>d__30 Line:1342:33
[20.09.2020, 11:45:00.374]: TelegramUpload> ...'Default' 'Telegram_RetryAfterFailSeconds' value was set to '300' seconds, so not retrying until 9/20/2020 11:50:00 AM
[20.09.2020, 11:45:00.406]: Trigger> Error: Could not load file or assembly 'System.Buffers, Version=4.0.2.0, Culture=neutral, PublicKeyToken=################' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) [FileLoadException] Mod: <Trigger>d__32 Line:1570:29

Thank you Chris Dodge, this is resolved by Version 1.67.8.21724 built on 9/20/2020 11:04:09 AM

Working great!
 
  • Like
Reactions: Chris Dodge
Are there current installation/setup instructions anywhere? I would like to try this AI but everything I found points to this thread (1-1/2yr old, 87 pages) as the installationsetup instructions ! I looked through many pages of the thread and see that the software has undergone so many changes since it first came out that the early installation/setup instructions are probably not valid anymore. I see that most of this thread is very experienced users that have followed the project closely. I am a newcomer that would like very much to try the AI but very confused where to even start.
 
  • Like
Reactions: NicholasBoccio
Are there current installation/setup instructions anywhere? I would like to try this AI but everything I found points to this thread (1-1/2yr old, 87 pages) as the installationsetup instructions ! I looked through many pages of the thread and see that the software has undergone so many changes since it first came out that the early installation/setup instructions are probably not valid anymore. I see that most of this thread is very experienced users that have followed the project closely. I am a newcomer that would like very much to try the AI but very confused where to even start.
Watch this video from The Hookup to get you started.
 
  • Like
Reactions: gmouer
VorlonCD mod updates:


That is the correct setting. You probably just need to make sure all instances of AITOOL.EXE are closed before modifying the file. But the real answer is to find some way in BI to reduce the number of images created, and to add more deepstack servers to the list. Sure you have BI set to "only when triggered"? Reduce jpeg snapshot each mm:ss ?

Thanks for the reply @Chris Dodge! I created a different folder for the AI snapshots and limited the snapshots to every 3 seconds. Each clip is stored there for an hour and then deleted. I'm sure I'll have additional questions so I really appreciate your patience and help!
 
Drop the container docker and install SSH and WebTerminal addon in HA(remember to set a password before it can be started)
open the addon page and then run this command to start deepstack
Code:
docker run --restart=always -e VISION-DETECTION=true -v localstorage:/datastore -p 83:5000 --name deepstack deepquestai/deepstack

View attachment 70823

after you have setup the new docker it will show in portainer so you can manage it there if you want.

Note!
if you dont have deepstack already in docker you need to run this first

Code:
docker pull deepquestai/deepstack


UPDATE!
Was to happy way to early....still error in AI. But if i run it manually from a cmd and with curl all my attempt is success. Why?
Code:
[19.09.2020, 07:20:08.887]: Starting analysis of D:\aiinput\altansd.20200919_072008903.jpg
[19.09.2020, 07:20:08.900]: (1/6) Uploading image to DeepQuestAI Server
[19.09.2020, 07:20:08.950]: (2/6) Waiting for results
[19.09.2020, 07:20:09.029]: (3/6) Processing results:
[19.09.2020, 07:20:09.033]: System.NullReferenceException | Object reference not set to an instance of an object. (code: -2147467261 )
[19.09.2020, 07:20:09.036]: ERROR: Processing the following image 'D:\aiinput\altansd.20200919_072008903.jpg' failed. Failure in AI Tool processing the image.
[19.09.2020, 07:20:13.820]:
[19.09.2020, 07:20:13.823]: Starting analysis of D:\aiinput\altansd.20200919_072013908.jpg
[19.09.2020, 07:20:13.826]: (1/6) Uploading image to DeepQuestAI Server
[19.09.2020, 07:20:13.833]: (2/6) Waiting for results
[19.09.2020, 07:20:13.837]: (3/6) Processing results:
[19.09.2020, 07:20:13.840]: System.NullReferenceException | Object reference not set to an instance of an object. (code: -2147467261 )
[19.09.2020, 07:20:13.843]: ERROR: Processing the following image 'D:\aiinput\altansd.20200919_072013908.jpg' failed. Failure in AI Tool processing the image.
[19.09.2020, 07:26:24.238]:
[19.09.2020, 07:26:24.241]: Starting analysis of D:\aiinput\altansd.20200919_072624506.jpg
[19.09.2020, 07:26:24.244]: (1/6) Uploading image to DeepQuestAI Server
[19.09.2020, 07:26:24.252]: (2/6) Waiting for results
[19.09.2020, 07:26:24.255]: (3/6) Processing results:
[19.09.2020, 07:26:24.259]: System.NullReferenceException | Object reference not set to an instance of an object. (code: -2147467261 )
[19.09.2020, 07:26:24.262]: ERROR: Processing the following image 'D:\aiinput\altansd.20200919_072624506.jpg' failed. Failure in AI Tool processing the image.


This is going to sound silly, but the "true" needs to be "True" to work.

docker run --restart=always -e VISION-DETECTION=True -v localstorage:/datastore -p 83:5000 --name deepstack deepquestai/deepstack
 
  • Like
Reactions: jompa68
@Chris Dodge

Keep up the awesome work! We're lucky to have you improving this while our other buddy is busy. Excited to see the two merge into one and you two maybe combining forces hah!

Question for ya.
I changed to yours a bit back but haven't had time to fiddle with it too much.
One odd thing I noticed (may have missed a post and don't see on the repo mentioned) is thatyours seems to use a json file for the settings. How does this relate to the .txt ones?
What I am finding is in yours for example it's not showing any of the relevant things enabled yet they are working. Is it reading this from the txt file ones or if none are enabled it's just using all of them maybe lol?
Are the txt files used at all on yours?

Also a bug maybe on this latest one, unless someone else can confirm. Having it run at startup (via the setting in it, which just adds the registry entry) it is now throwing an error saying the start in folder is not the same after reboots. I didn't have this issue until after updating and am fairly sure I'd rebooted from a windows update the day before fine but thought would mention in case it's realted. I togged it off and on and same issue still. When the error happens I see the AI icon in the taskbar for a moment so it's like it runs and dies off instantly with that error. Thoughts?
 
@B-Murda - Upon first run (when it doesnt see the JSON file) it migrates all old settings from AITOOL.EXE.CONFIG (or WindowsFormsApp2.user.config) and all the \camera\*.txt files all into AITOOL.SETTINGS.JSON file. So nothing is read from those original camera txt files after the first import.

The error about the start in folder is fixed in the latest build. If you dont see it fixed with the latest file on the releases page (I updated the zip file a few days ago), watch for a new release in a day or two.
 
@Chris Dodge - Just wanted to comment and say your most recent release (1.77) is truly amazing! Having the ability to send MQTT messages with different variables in the topic and payload seriously reduces the amount of legwork I had to do in order to get person detection properly working with HASS. I am allowing a ton of motion events, even shadows triggering and such, and the dynamic masks just make this one of the BEST forks/tools I've seen for doing AI detection. Thank you so much for taking time to keep this tool updated with new and useful features! (I'm using a 6700 with BI and deepstack running on it)

It has seriously changed my entire setup and works pretty well.
 
Here is an update to my fork of @GentlePumpkin 's awesome tool.

Change log:
  • Camera option for 'Trigger Cancels' - Basically, rather than the URL triggering an event in BI, it will only be called when the detection is CANCELED. Note that you MUST change .22. As I understand this is how Sentry AI works - it just cancels a detection sent by BI. I haven't tested the 'Trigger cancels' camera checkbox yet since I dont have that configuration right now - let me know if it works.
(Along with previous updates like Deepstack (for windows) start/stop support, auto start, read BI camera settings, a LOG tab, save all cameras and settings to single JSON file, etc)

Hey @Chris Dodge and @B-Murda, could you please help me understand the use of &flagalert=1 and &flagalert=0 with AI Tool and BI?

I've seen two different methods discussed, and it's unclear to me if both are currently possible:
  1. With the first method, AI Tool would only send a "trigger cancel" command to BI if no relevant object is detected by AI Tool/Deepstack.
  2. With the second method, AI Tool would send a response to BI both when a relevant object is detected (&flagalert=1) and when no relevant object is detected (&flagalert=0).
Are both of these workflows currently possible with AI Tool and BI?
Do either of them rely on cloned cams, or can both methods work with only a single instance of a camera?
If I use the second method, do you fill both the Trigger URL and Cancel URL in AI Tool > Cameras > Actions > Settings?

Also, @B-Murda, you mentioned the Sentry appears to use the second method, but instead of setting the BI alert as Flagged, it instead has it's own custom value in BI, correct?
 
  • Like
Reactions: iGagy
@aesterling - I've never actually tried using this method with BI, but I've made it so both methods should work

Trigger cancel: In actions dialog, leave "Trigger URL(s)" empty and fill out "Cancel URL(s)" with the url+flagalert=0

Send for both: In actions fill out Trigger URL and set to flagalert=1, and set Cancel URLs to flagalert=0

You might be able to do this with the Blueiris MQTT interface too but I havent looked into it yet. See more details for all the mqqt and admin command here:
 
@Chris Dodge I'm still working slowly to get this implemented. Couple of things...
1) Sometimes AITool gets stuck processing a .jpg and doesn't return to "Running" status. What should I check to help with that?
2) When it analyses a snapshot, A lot of times, the moving object is either not in frame or partially off-frame.
3) When re-launching the AITool, I have to Stop/Start Deepstack for it to be running correctly. At first launch, it reports that it is not activated. I Stop/Start and it okay.
4) My trigger action will not confirm within AITool by using the Test button. It works perfectly in a web browser. What should I do?

Thanks!
 
Last edited:
@Chris Dodge I'm still working slowly to get this implemented. Couple of things...
1) Sometimes AITool gets stuck processing a .jpg and doesn't return to "Running" status. What should I check to help with that?
2) When it analyses a snapshot, A lot of times, the moving object is either not in frame or partially off-frame.
3) When re-launching the AITool, I have to Stop/Start Deepstack for it to be running correctly. At first launch, it reports that it is not activated. I Stop/Start and it okay.

Thanks!
1) The log should show some of the last things it did before it freezes but most likely the Deepstack server has stopped responding. It is kind of buggy sometimes. After 60 seconds I think it will timeout with an error in the log. You may want to set up more deepstack servers on other machines to take over if one freezes.
2) Where are you looking at the object? On History tab or in Mask Manager? In Mask Manager it sometimes doesnt show the exact picture that originally had the object in it... Have to work out a better way to handle that. If in HISTORY tab that its wrong it could be the bug referenced a few pages back several times.
3) It is probably running fine and you dont have to stop/start. It wont say 'activated' because it doesnt really know until you get your first object detection back from deepstack server. Stopping and starting it just allows AITOOL to have full control over the processes and report output from deepstack.
 
1) The log should show some of the last things it did before it freezes but most likely the Deepstack server has stopped responding. It is kind of buggy sometimes. After 60 seconds I think it will timeout with an error in the log. You may want to set up more deepstack servers on other machines to take over if one freezes.
2) Where are you looking at the object? On History tab or in Mask Manager? In Mask Manager it sometimes doesnt show the exact picture that originally had the object in it... Have to work out a better way to handle that. If in HISTORY tab that its wrong it could be the bug referenced a few pages back several times.
3) It is probably running fine and you dont have to stop/start. It wont say 'activated' because it doesnt really know until you get your first object detection back from deepstack server. Stopping and starting it just allows AITOOL to have full control over the processes and report output from deepstack.
1) Here's where the log picks up and shows the .jpg it froze on.

[21.09.2020, 15:19:55.143]: ImageQueueLoop> Adding task for file 'Driveway_AI.20200921_151954844.jpg' (Image QueueTime='0.0' mins, URL Queue wait='0ms', URLOrder=1 of 1, URLOriginalOrder=1) on URL ''
[21.09.2020, 15:19:55.145]: ImageQueueLoop> Done adding 1 total threads, ErrCnt=0, ImageProcessQueue.Count=0
[21.09.2020, 15:19:55.147]: DetectObjects> 127.0.0.1:81 - Starting analysis of S:\BI_AI\Driveway_AI.20200921_151954844.jpg...
[21.09.2020, 15:19:55.159]: DetectObjects> 127.0.0.1:81 - (1/6) Uploading a 312570 byte image to DeepQuestAI Server at [21.09.2020, 15:19:57.206]: DeepStack> Server.exe> [GIN] 2020/09/21 - 15:19:57 | 200 | 2.046302s | 127.0.0.1 | POST /v1/vision/detection
[21.09.2020, 15:19:57.207]: DetectObjects> 127.0.0.1:81 - (2/6) Posted in 2048ms, Received a 121 byte response.
[21.09.2020, 15:19:57.209]: DetectObjects> 127.0.0.1:81 - (3/6) Processing results...
[21.09.2020, 15:19:57.211]: DetectObjects> 127.0.0.1:81 - Detected objects: Person (99.68%),
[21.09.2020, 15:19:57.212]: DetectObjects> 127.0.0.1:81 - (4/6) Checking if detected object is relevant and within confidence limits:
[21.09.2020, 15:19:57.213]: Outsidemask> ->Camera has no mask, the object is OUTSIDE of the masked area.
[21.09.2020, 15:19:57.214]: DetectObjects> 127.0.0.1:81 - Person (99.68%) confirmed.
[21.09.2020, 15:19:57.216]: DetectObjects> 127.0.0.1:81 - The summary:Person (99.68%)
[21.09.2020, 15:19:57.217]: DetectObjects> 127.0.0.1:81 - (5/6) Performing alert actions:
[21.09.2020, 15:20:00.847]: OnCreatedAsync>
[21.09.2020, 15:20:00.849]: OnCreatedAsync> ====================== Adding new image to queue (Count=1): S:\BI_AI\Driveway_AI.20200921_152000844.jpg
[21.09.2020, 15:20:01.127]: WaitForNextURL> ---- All URL's are in use or disabled, waiting...

2) Looking at the History tab.
3) Good to know, thanks!
4) I added #4 after you replied.

Appreciate it!