AI is broken after switching from DeepStack to CodeProject

[spazstic]

I have 10 cameras running on my server with an i7 970 (hexacore), 32gb ram, SSD for OS and 4tb drive for Blue Iris storage. The server is also used for other purposes (mainly my Plex server).

It worked great for years using Deep Stack. I upgraded Blue Iris about a year ago and Deep Stack was no longer supported. Installed CodeProject and it's worked awfully since then. It completely misses alerts it should record (person in front of camera) and records garbage like cobwebs at night and trees blowing in the wind. Never had those problems with DS. Finally had some time to try to troubleshoot it, and I'm coming up empty.

I keep getting AI timeout errors and AI restart logs. CodeProject server appears to switch between online and offline constantly.

Thought maybe my system is getting overwhelmed (it is older), even though it worked fine with DeepStack. I tried switching to GPU on CP but I don't think the GTX 650 Ti is up to the task.

I have read through countless forums and configurations, watched Youtube videos, and I can't seem to get this working. DeepStack was working flawlessly. My next step is to revert to an earlier build of BI so that I can get DS back.

Any advice or suggestions appreciated.

 

[spazstic]

 

[spazstic]

Here's one of the analyses from CP. Terrible AI in my opinion. DS very rarely had anything false with over 40%. It's constant with CP.

 

[spazstic]

CP Server log. I'm guessing some of the missed images are due to the time it's taking to perform the object detection. Today is a windy day, so there are a lot of alerts occuring.

 

[MikeLud1]

What version CP.AI you have installed. Also the Object Detection (YOLOv5 .NET) model is an old version, the current version is 1.6.
You might need to uninstall CP.AI and delete folders C:\ProgramData\CodeProject and C:\Program Files\CodeProject then reinstall the current version of CP.AI 2.2.4.
Before reinstalling CP.AI disable AI in BI main AI setting. Do not enable AI until all the module are installed

 

[spazstic]

I had assumed I was up to date based on the BI status:

 

[MikeLud1]

I had assumed I was up to date based on the BI status:

View attachment 173945

If you go to the Install Modules tab does it show and update is available for the Object Detection (YOLOv5 .NET) module

 

[spazstic]

Thanks, good catch there.

I followed your original instructions - disabled AI in BI, completely uninstalled CP.AI and deleted the two folders, then reinstalled. It took a while then showed the latest version with Cuda running. Started BI, and enabled AI. CP.AI then threw a bunch of errors - GPU too old (v.3.0, v3.7 req'd).

Running on the YOLO v5 6.2 - 1.6.1 now in CPU mode.

However, the AI is still just terrible. In the example below, the trash cans, which are stationary, keep getting flagged as suitcase, or car, or truck. In this example it's flagged as a car at 83% confidence.

The ATV also alternates between motorcycle, truck, and boat.

I also like to have my dogs trigger, but the AI completely ignores them walking through the frame.

I never had these kind of failures with DeepStack. So far my experience with CP.AI is a significant downgrade, unless I'm doing something wrong.

 

[Futaba]

if you increase the confidence percentage to higher the yellow ones will go away. Try 85%.

 

[spazstic]

if you increase the confidence percentage to higher the yellow ones will go away. Try 85%.

Thanks, I'm aware of that. I've been playing with changing the percentage and creating blackout zones but my point is that CP.AI is much less accurate than Deep Stack was, so I don't see the push to change. Hopefully it will improve in the future.

 

[spazstic]

if you increase the confidence percentage to higher the yellow ones will go away. Try 85%.

Also, 85% is quite high. A person walking through the frame is in the 60-90% range, so you're going to miss a lot of positive IDs.

 

[MikeLud1]

The models CP.AI is a new version of the YOLOv5 models, you can download the original DeepStack models from the below link. After downloading the models just replace the models. I am not sure why you are getting poor results with CP.AI.

Post screenshots of your main IA settings and one of your cameras AI settings

Release v3.1 - Bug Fixes and Performance Improvements · ultralytics/yolov5

This release aggregates various minor bug fixes and performance improvements since the main v3.0 release and incorporates PyTorch 1.7.0 compatibility updates. v3.1 models share weights with v3.0 mo...

github.com

 

[spazstic]

Thanks, I'd like to get CP working if I can, since that's where BI is headed. Here are my configs:

 

[actran]

@spazstic AI performance looks really bad from your screenshot, 1-15secs+.

Mine is usually < 0.05 secs (<50 msecs), I have Nvidia 1650

Some things that will improve AI performance:
#1: Turn off default object detection and use 1 of Mike's custom model(s) instead - they're included with CP.AI.
#2: Uncheck "Use main stream if available". CP.AI uses Deepstack models also and they're trained on small images. Assuming you have BI5 configured with cam substreams at D1, that is usually sufficient.
#3: Uncheck "Static object analysis". When you have this enabled, you're telling BI5 to send images to CP.AI at regular intervals like every 120 secs to process. This will cause unnecessary load on your PC if this is enabled for every camera. Depending what you are trying to accomplish per cam, this is usually not necessary.

 

[Tinman]

It worked great for years using Deep Stack

Yet, you still have time overlay and title overlay enabled on your cam?

 

[spazstic]

@spazstic AI performance looks really bad from your screenshot, 1-15secs+.

Mine is usually < 0.05 secs (<50 msecs), I have Nvidia 1650

Some things that will improve AI performance:
#1: Turn off default object detection and use 1 of Mike's custom model(s) instead - they're included with CP.AI.
#2: Uncheck "Use main stream if available". CP.AI uses Deepstack models also and they're trained on small images. Assuming you have BI5 configured with cam substreams at D1, that is usually sufficient.
#3: Uncheck "Static object analysis". When you have this enabled, you're telling BI5 to send images to CP.AI at regular intervals like every 120 secs to process. This will cause unnecessary load on your PC if this is enabled for every camera. Depending what you are trying to accomplish per cam, this is usually not necessary.

Thanks for the suggestions. I'm using CPU and it's an older CPU (i7 970). I think a system upgrade in the future is needed.
#1 - I see Mike's custom models here but you say they're included with CP.AI. Where do I get CP.AI? I thought that was YOLOv5 .NET / 6.2
#2 - I do have substreams configured.
#3 - I'd like BI to not alert for items like parked cars. I assumed this setting would prevent that from happening. I can remove it for some of the cameras however.

 

[spazstic]

Yet, you still have time overlay and title overlay enabled on your cam?

Yes, why not? Let that overhead float on the camera side rather than the server.

 

[wittaj]

Yes, why not? Let that overhead float on the camera side rather than the server.

His point is you have time coming from both the camera and from BI. Turn off the BI overlay. It may only be a fraction of CPU, but every little bit helps on an underperforming machine.

Plus since they don't match, should something happen that it were to go to court, the defendant attorney will have a field day discrediting the video, even if it is only a 3 second time difference. Or years in your case.

And who knows maybe the big @$$ BI time overlay is being perceived as motion and why you keep getting the camera triggered LOL. I have seen that happen on one of my cams so I excluded the time area from the Zone and it stopped the triggers.

 

[actran]

@spazstic Look at the screenshot here. Make sure you have same settings. The list of custom models available to you is right there as a popup.


For each cam, set the custom model desired as shown. ipcam-general is good if you only need person/vehicle detection for both day and night. Look at other models for other objects. BTW, use higher confidence to eliminate false alarms. Again, CP.AI is using Deepstack models also so the quality of the predictions is driven by the model that you use.

 

[Tinman]

Plus since they don't match, should something happen that it were to go to court, the defendant attorney will have a field day discrediting the video, even if it is only a 3 second time difference.

LOL, I was looking at the 2000-06-06 date