CodeProject.AI Version 2.5

[tedrpi]

I recommend just use CUDA 11.8 because the cuDNN install script is for CUDA 11.x. If you install CUDA 12.x you will need to manually install cuDNN for CUDA 12.x

@MikeLud1 - thank you so much for this info, didn't see it anywhere in the documentation or CPAI forums. Not sure what fixed what but manually installing cuDNN for Windows 12.2, installing CPAI 2.5.1, switching to Yolo .NET, and disabling BI start stop control of CPAI has resulted in a much more stable and faster system. I'm seeing YOLO .NET to be considerably faster on a GTX 1660.

FWIW, for others, I used this guide to install cuDNN for 12.2 windows. Pay special attention when updating the cuDNN path to not overwrite what's in your existing path. I also found it best to remove old cuDNN entries from my PATH variable

Installation Guide :: NVIDIA cuDNN Documentation

 

[milosm]

I never changed the multi TPU so it should be default. Since that PC is blocked from the internet again I'll type what I think you are referring to.

** CPAI_CORAL_MULTI_TPU = false

I also just checked the memory and CodeProject..AI.Server is using 374 MB memory. It's fluctuating up and down 2-3 MB but when I started replying it was only at 295 MB. (I'm slow going back and forth between computers and typing LOL). The difference it is going up even if minimal. I'll keep an eye on it now since it is going up. It was only started 7 hours ago so the rate at leaking I can see that being an issue.

Is the memory leak a known issue? Is there a version I should revert to?

Unfortunately I didn't keep track of the CPAI versions since I usually use the logs and since part of the process is to uninstall and delete the folder I doubt there is a way to tell. However, looking at my downloads folder, I do see I have the following (created dates):

2.0.8 (5/13/2023)
2.4.7 (1/11/2024)
2.5.0 RC8 (1/20/2024)
2.5.1 (1/26/2024)

I'm pretty sure I used 2.0.8 with CPU/GPU until I got the TPU running on 1/8/2024. I also had issues with GPU until recently so was using CPU (I think it might have been drivers or Windows).

The AI times in BI are ranging between 87 and 172 ms (roughly as I skim through the log) with Coral TPU. If memory serves me correct, with the GPU they were in the 200-300 ms and CPU was 300-400 ms (both medium model size).

I didn't mind the slow response but when a few cameras trigger at once it pegs the CPU and I'm pretty sure that was the reason I lost some triggers (again that was before GPU was working which is why I ordered the TPU).

Again, if there is any recommendations, I'm open to them. What causes the AI: error 500? I had over a dozen in the last hour.

Almost debating getting another BI license and running on VM server so I can keep one machine stable (current version minus X) and then second machine to upgrade and test.

Thanks for the help.

Did you by any chance manage to resolve your AI: error 500 issue?

 

[nils_92]

Did you by any chance manage to resolve your AI: error 500 issue?

I'm still having this issue even though detections work

 

[sionroberts]

I'm still having this issue even though detections work

Same here. 500 errors then I see detects taking 0s which means they’re all failing. Rebooting is the only answer right now. Blueiris on Windows 11 on a very new NUC with Coral TPU on USB. CPU is tracking at < 15%. 9 cameras and TPU response time generally at 100ms or so. Model is standard Small. Works wonderfully apart from the 500 errors which creep in after a few hours operating.

 

[AlwaysSomething]

Did you by any chance manage to resolve your AI: error 500 issue?

I am still getting the errors.

It's strange because I used to think it was volume related (front 3 cameras triggering at the same time). But I looked in the logs and I had some in the middle of the night when only 1 camera triggered from my cat. It was also strange because there was at least one instance where the error was first and then immediately followed by 2 successful detects.

Looking at the logs, I also saw I lost a signal for a POE camera during the middle of the night as well (which never happened before). System also locked up again this morning and this PC is dedicated to BI. Hence, my screen name is Always Something.

I hate to do it but I think I need to rebuild the PC. Something definitely got fudged when I did the upgrade to BI and CPAI but I don't know what.

How does everyone else do upgrades on a PC that is blocked from the internet? The way I have been doing it is I have a firewall rule that I enable to allow it access. I only turn on the rule when I do the upgrade. Then I turn the rule off when the upgrade is complete. I also disabled Windows Updates but I never trust MS.

It's more time consuming but I may have to find out the IP BI and CPAI are trying to access and only allow those.

 

[AlwaysSomething]

On a positive note, I think BI and CPAI are great and amazed by them. I bought my first camera to replace my Ring doorbell last year and I've gotten more cameras since then. I'm constantly learning new things and always amazed. There is SOOO much stuff. If time and budget were unlimited I'd buy a lot more cameras. I have to say thanks to all the people who offer support on this forum. I'd do it on each thread I read but some are years old and I just got to reading them

 

[mailseth]

Yeah. There's something oddly addictive about building a camera system. I first had the doorbell camera, but then we had a string of thefts on the street where I couldn't see them. So I put two more cameras up on the street. But they were shitty Google cameras and the resolution wasn't nearly high enough to be useful. So then I added some PoE cameras zoomed in on faces and license plates. Haven't had thefts since then. Then a fugitive ran through the backyard. Had cops out in front with long guns out. So I put up a few cameras out there. Mostly for watching deer. And the occasional fugitive, as one does. But now they all trigger all the time on spiders and rain, so here I am re-writing the TPU implementation for CPAI now to efficiently alert on actual people instead of rain and wind. My partner just rolls her eyes.

 

[ChrisMaunder]

FWIW, for others, I used this guide to install cuDNN for 12.2 windows. Pay special attention when updating the cuDNN path to not overwrite what's in your existing path. I also found it best to remove old cuDNN entries from my PATH variable

Installation Guide :: NVIDIA cuDNN Documentation

We've updated the install_cuDNN script that comes with CodeProject.AI server to provide support for CUDA 10.2, 11 and 12. We're hoping to have a release in the next couple of days once we get some kinks out of the Coral multi-TPU setup

 

[ChrisMaunder]

I am still getting the errors.

It's strange because I used to think it was volume related (front 3 cameras triggering at the same time). But I looked in the logs and I had some in the middle of the night when only 1 camera triggered from my cat. It was also strange because there was at least one instance where the error was first and then immediately followed by 2 successful detects.

What setup are you running (and apologies if you posted this previously). USB or PCIe Coral? I assume this is a native Windows installer and not Docker?

 

[milosm]

For what it's worth and to give you more data points, my setup is USB Coral with a native Windows installer.

 

[ChrisMaunder]

For what it's worth and to give you more data points, my setup is USB Coral with a native Windows installer.

Great, thanks

 

[AlwaysSomething]

What setup are you running (and apologies if you posted this previously). USB or PCIe Coral? I assume this is a native Windows installer and not Docker?

It's actually the M.2 B+M Coral TPU but currently I have it using an M.2 to PCIe adapter. My plan is to eventually use the M.2 directly on the motherboard but I would need to move the OS from the NVMe to an SSD or move BI to a different PC that has two M.2 slots (which if I do I'm also debating if I want to virtualize it so I can have snapshots and potentially easier backups). Both BI and CPAI are on the same Windows 10 PC and it is a bare metal install (no VMs). The PC is also dedicated to BI and CPAI. I don't use it for anything else.

The Windows 10 install was a fresh install but it just occurred to me that it may have been using the Dell ISO and not the MS ISO (straight from Microsoft). It is Win 10 Pro so shouldn't have as much of the bloatware but may have some Dell junk. I also ran Chris Titus's Win Utility to disable all the unnecessary jobs, updates, etc.).

Let me know if there is any else you want to know and again, I appreciate the help.

 

[tigerwillow1]

Are any suggestions for fixing the failed detection of a cat? It's not the confidence setting in BI. All of the models in CPAI returned "Nothing found". The cat is near the right edge at the bottom of the house. The camera's IVS drew a box around it and tracked it no problem.


Here's an interesting analysis. For my purposes I don't care if it's the correct animal or not, just that it's an animal.

 

[wittaj]

Are any suggestions for fixing the failed detection of a cat? It's not the confidence setting in BI. All of the models in CPAI returned "Nothing found". The cat is near the right edge at the bottom of the house. The camera's IVS drew a box around it and tracked it no problem.
View attachment 184763

Here's an interesting analysis. For my purposes I don't care if it's the correct animal or not, just that it's an animal.
View attachment 184764

Glad to see you gave it a shot!!!

Was the initial setup easier or harder than you thought? Compared to an NVR?

The cat probably is having trouble due to the distance and it probably was walking around behind the watermark that probably interfered with the analysis.

Given the big IR blowout by that bush, some adjustment of brightness and contrast might help. In my experience the AI works best if there is a 7-10 difference between brightness and contrast (where contrast is the higher of the two).

 

[mailseth]

I’d look into tiling the image also, by chopping it up into smaller pieces and running inference on those. It’s something I’ve been working on for the Coral implementation, I don’t know if it’s enable anywhere else tho. Things run slower, however, because you’re doing 2x or 4x the analysis.

 

[tigerwillow1]

Was the initial setup easier or harder than you thought? Compared to an NVR?

Since you asked.... I was going to report my experience somewhere and I don't see a perfect place to do it. It's a bit of thread creep here because most of my feedback is about BI. For context, the only reason I took on BI+CPAI is because of my smart Dahua cameras that are too dumb to detect animals. So even with my glowing praises of BI, if CPAI doesn't cut it with the animal detection, it's a no-go. A lot of credit goes to you and fenderman for your encouragement and pushing back on my fear of CPAI which was mostly unfounded. I need more time to evaluate. On some days it almost looks like a zoo parading past the cameras. Right now it's stormy and they're holed up in some secure undisclosed location.

For CPAI, the install on win 10 was easy and went smoothly. The only rock I ran into was having to swap out yolo v5 6.2 for yolo v5.net. If this was in the instructions somewhere I missed it. One of my fears was having to find and load the ipcam models. It was a relief to finally find that the CPAI install had already loaded them, but it took me a while to figure it out. Getting BI to use the custom models took a bit of fiddling, and I don't remember what I did for that. It wasn't obvious to me that I had to add the object names (dog, cat, etc.) to BI's camera AI settings. Figuring out how to use the AI status window left me scratching my head for a long time. So even though installing CPAI was easy, it was a bit of work to get BI set up to use it.

With BI itself I have a bunch of polar opposite observations. On the "pro" side, the user interface is so much nicer that I'm at a loss for the right word to label it with. Scanning through triggers is significantly faster and easier. (Yes I remember you and others telling me that previously). BI is significantly more capable and flexible than the NVR, but with that comes complexity, a "con". The BI install is easy, the setup not so easy. Once you've learned what to do it's not too difficult, but the learning curve is pretty steep. IMO when starting from scratch, it's way easier to get to first use with a Dahua NVR than with BI. One of many examples is the storage setup, where you have to learn the different types of storage and configure where they go. I think I got it right on the 3rd try. With the NVR this is a nonexistent issue. Our technology-deprived church's on-and-off security camera system is on again. No way I'd recommend a BI system to them because nobody else would have a prayer (pun??) of maintaining it. Somebody else might be able to struggle enough to keep an NVR system running.

My initial gripe list for BI:
1. Camera IVS lines don't show. I realize that's at the edge of asking too much for a VMS not dedicated to Dahua.
2. With substreams enabled I wish the live camera windows could display from the main streams. I worked around this a bit by using substream 2 at 1080p, but I'd like the option to display the main stream.
3. BI still has the bug of enabling motion detection on the cameras. Wastes time figuring out what's going on and working around it.
4. Headless operation is not as easy as with the NVR. Can't get to any settings from UI3. Workaround is to use remote desktop, I guess.

Not too long of a list for something this complex, but I think a non-technical person would be hopelessly lost. I will take more time working with CPAI's animal detection, realizing there's probably not much I can do to make it work better. I'm not going as far as trying to build my own models.

 

[David L]

Since you asked.... I was going to report my experience somewhere and I don't see a perfect place to do it. It's a bit of thread creep here because most of my feedback is about BI. For context, the only reason I took on BI+CPAI is because of my smart Dahua cameras that are too dumb to detect animals. So even with my glowing praises of BI, if CPAI doesn't cut it with the animal detection, it's a no-go. A lot of credit goes to you and fenderman for your encouragement and pushing back on my fear of CPAI which was mostly unfounded. I need more time to evaluate. On some days it almost looks like a zoo parading past the cameras. Right now it's stormy and they're holed up in some secure undisclosed location.

For CPAI, the install on win 10 was easy and went smoothly. The only rock I ran into was having to swap out yolo v5 6.2 for yolo v5.net. If this was in the instructions somewhere I missed it. One of my fears was having to find and load the ipcam models. It was a relief to finally find that the CPAI install had already loaded them, but it took me a while to figure it out. Getting BI to use the custom models took a bit of fiddling, and I don't remember what I did for that. It wasn't obvious to me that I had to add the object names (dog, cat, etc.) to BI's camera AI settings. Figuring out how to use the AI status window left me scratching my head for a long time. So even though installing CPAI was easy, it was a bit of work to get BI set up to use it.

With BI itself I have a bunch of polar opposite observations. On the "pro" side, the user interface is so much nicer that I'm at a loss for the right word to label it with. Scanning through triggers is significantly faster and easier. (Yes I remember you and others telling me that previously). BI is significantly more capable and flexible than the NVR, but with that comes complexity, a "con". The BI install is easy, the setup not so easy. Once you've learned what to do it's not too difficult, but the learning curve is pretty steep. IMO when starting from scratch, it's way easier to get to first use with a Dahua NVR than with BI. One of many examples is the storage setup, where you have to learn the different types of storage and configure where they go. I think I got it right on the 3rd try. With the NVR this is a nonexistent issue. Our technology-deprived church's on-and-off security camera system is on again. No way I'd recommend a BI system to them because nobody else would have a prayer (pun??) of maintaining it. Somebody else might be able to struggle enough to keep an NVR system running.

My initial gripe list for BI:
1. Camera IVS lines don't show. I realize that's at the edge of asking too much for a VMS not dedicated to Dahua.
2. With substreams enabled I wish the live camera windows could display from the main streams. I worked around this a bit by using substream 2 at 1080p, but I'd like the option to display the main stream.
3. BI still has the bug of enabling motion detection on the cameras. Wastes time figuring out what's going on and working around it.
4. Headless operation is not as easy as with the NVR. Can't get to any settings from UI3. Workaround is to use remote desktop, I guess.

Not too long of a list for something this complex, but I think a non-technical person would be hopelessly lost. I will take more time working with CPAI's animal detection, realizing there's probably not much I can do to make it work better. I'm not going as far as trying to build my own models.

BI was like pfSense to me. They are both Awesome with many. many options. But overwhelming at first. I am still way behind in understanding Camera settings, not even sure I will get the full just of them. BI you will like, I found that cloning a camera helped me be able to play with the settings in CPAI. I separated my Delivery model from my Person model and once I get my Cameras set up at our new place I will have an animal clone. Found cloning does not take away from your CPU/GPU, that I can tell, plus it makes it easier to separate the different settings. But now with the Mesh CPAI, which I have not tried, I am sure it is like cloning.
I will say, once you go BI or pfSense you won't go back.

 

[tigerwillow1]

I've got something I can't figure out. A camera IVS trigger caused an image to be run through CAPI and in the BI status window it indicates Squirrel:60%[A}. (What does the [A} mean?). The min confidence in BI is set to the default of 50%. "squirrel" is on the "To confirm" list. BI log says "Alert cancelled nothing found".

In table form:
BI's AI status: squirrel:60%[A]
Min confidence setting: 50%
"To confirm" contains "squirrel"
BI log: Alert cancelled nothing found

Looks to me like something isn't working correctly. The "To confirm" list is pretty long, and squirrel is near the end. Could that have something to do with it?
Fun fact: The squirrel is actually a Honda Fit viewed from a high mounted camera.

 

[David L]

I've got something I can't figure out. A camera IVS trigger caused an image to be run through CAPI and in the BI status window it indicates Squirrel:60%[A}. (What does the [A} mean?). The min confidence in BI is set to the default of 50%. "squirrel" is on the "To confirm" list. BI log says "Alert cancelled nothing found".

In table form:
BI's AI status: squirrel:60%[A]
Min confidence setting: 50%
"To confirm" contains "squirrel"
BI log: Alert cancelled nothing found

Looks to me like something isn't working correctly. The "To confirm" list is pretty long, and squirrel is near the end. Could that have something to do with it?
Fun fact: The squirrel is actually a Honda Fit viewed from a high mounted camera.

Which custom-model are you using/loading?

 

[tigerwillow1]

Which custom-model are you using/loading?

For now I have the whole enchilada of what's loaded by default: license-plate, ipcam-animal, ipcam-combined, ipcam-general, ipcam-dark. But does that have anything to do with the AI status saying something was identified at 60%, and the BI log saying nothing found?