Hell Yeah! Direct Deepstack Integration - 5.4.0 - March 31, 2021

[CamCrazy]

OK, I have the 1660 card installed and running, Deepstack doing odd things though, seems like I have to jump through hoops to get it running, ie, quit Blue Iris, kill the service, start Deepstack via Powershell console, then fire up Blue Iris. I am thinking maybe the Cuda Toolkit updates 1 and 2 might be required? any thoughts. I actually installed both updates and I think it is better, seems to start with Blue Iris but the start/stop option doesn't function like when the CPU was in charge. Below is the post I added on the custom models thread:

"I finally got the GTX 1660 installed along with a WD Purple Pro, the latter is a bit noisier than I had hoped, that is off topic though! Anyway, after some teething troubles setting up Deepstack GPU and it not starting with Blue Iris, I've got some very initial impressions. I believe High mode in Deepstack seems not so good at night, Medium is where I am at now. Processing time is quicker, a lot quicker in the day. Night processing is quite long but I had '+ real-time images' set to 15-30 and accuracy seems significantly better than when using the CPU processing.

Also observed that Blue Iris doesn't seem to show the deepstack processing activity so much in the alert list now, they just appear, assume this is a speed thing. Also my alert list doesn't auto scroll to the top which is annoying, not sure if this is Deepstack GPU related or Blue Iris. Update, it seems you need to have the 'Now' option toggled on Blue Iris timeline for this to work, I am sure previously it defaulted to this even after playing an alert clip, unsure if this is by design or not - annoying since previously never gave it a second thought.

GPU spikes to 20-30% briefly when 3-4 cameras get triggered at once, CPU spikes to 50-60% briefly presumably processing the motion. Previously using CPU for Deepstack the processor would spike right to 100% and sit there causing processing issues with I assume Deepstack and motion in Blue Iris.

All in all some good results, I might play with pushing the '+ real-time images' up during the day and check the results, during the night when using custom model the processing times are up in the 1000's potentially, during the day with above settings they are always very low 100's. Will see how things progress."

Adding to that I am seeing massive improvements in my night model accuracy, getting close to daytime results with just IR and no artificial light. If I can get it running a bit more in sync with Blue Iris regarding situations like PC restarting etc that would be good. Also sorting the 'Confirm' options using percentages as detailed in post #482 below would be nice, I use that option on a few cameras.

A win for the GPU so far, I'd maybe avoid the WD Purple Pro drives if you want a quiet computer or NAS though. I am being picky, my PC has high grade fans and cooler which are incredibly quiet

 

[CamCrazy]

One other thing I have noticed today, since switching to GPU for deepstack the 'Confirm' option where I have things like Car:51, Person:60 etc, no longer applies and I get any percentage shown in the alerts list. Is this a GPU issue or Blue Iris, I am running latest release of Blue Iris, may switch to more stable one.

Update for those interested, I removed all Nvidia graphics drivers and re-installed, seems like that might have done it, the Start/Stop button within Blue Iris Ai now works as expected. Will know for sure in the next 24 hours.

 

[CamCrazy]

Since installing the drivers again for the GPU all has been well, night accuracy with custom model is in another league compared to CPU - which in fairness was choking I might add. Daytime results similarly are leagues ahead and processing times during the day are between 60-150ms even with real time images samples set up to 30. The GPU is relatively unstressed (sub 20% normally) but when several cameras get triggered at once with Deepstack events, GPU can spike to 40-50% briefly, keep that in mind depending on your usage and what card you plan to use, if any.

Rekor agent for ANPR is stressing my CPU more than anything now which is a nice problem to have Oddly enough it is worse when using GPU acceleration in Rekor, but that is for the other thread

As for the GPU, well, if you have the means, I highly recommend picking one up

 

[105437]

From one of the DeepStack developers regarding the new October release...

"This release is focused on resolving a number of bugs. We are moving to a montly release cadence for DeepStack and will be publishing release notes soon. A number of new features and performance improvements is slated for the October Release by end of this month."

 

[CamCrazy]

From one of the DeepStack developers regarding the new October release...

"This release is focused on resolving a number of bugs. We are moving to a montly release cadence for DeepStack and will be publishing release notes soon. A number of new features and performance improvements is slated for the October Release by end of this month."

That looks interesting, did wonder if a new version was on the horizon

 

[wittaj]

Since installing the drivers again for the GPU all has been well, night accuracy with custom model is in another league compared to CPU - which in fairness was choking I might add. Daytime results similarly are leagues ahead and processing times during the day are between 60-150ms even with real time images samples set up to 30. The GPU is relatively unstressed (sub 20% normally) but when several cameras get triggered at once with Deepstack events, GPU can spike to 40-50% briefly, keep that in mind depending on your usage and what card you plan to use, if any.

Rekor agent for ANPR is stressing my CPU more than anything now which is a nice problem to have Oddly enough it is worse when using GPU acceleration in Rekor, but that is for the other thread

As for the GPU, well, if you have the means, I highly recommend picking one up

I guess I am glad I am not the only one that has seen worse CPU % when using Rekor as GPU. Sunlight and shadows kills Rekor in the daytime LOL. Currently deciding on if I try a Jetson nano to completely take it off the computer.

But the GPU does great for DeepStack.

 

[CamCrazy]

I guess I am glad I am not the only one that has seen worse CPU % when using Rekor as GPU. Sunlight and shadows kills Rekor in the daytime LOL. Currently deciding on if I try a Jetson nano to completely take it off the computer.

But the GPU does great for DeepStack.

Likewise, it is pretty disappointing to be honest, I wonder if that might change going forward? Once I switch to dedicated ALPR camera zoomed in more hopefully it will improve. GPU is working wonders for DeepStack so far for me, very impressed in all respects.

 

[wittaj]

Likewise, it is pretty disappointing to be honest, I wonder if that might change going forward? Once I switch to dedicated ALPR camera zoomed in more hopefully it will improve. GPU is working wonders for DeepStack so far for me, very impressed in all respects.

Yeah I cannot figure out why moving ALPR to the GPU causes it to go up so much.

 

[CamCrazy]

Yeah I cannot figure out why moving ALPR to the GPU causes it to go up so much.

It is bizarre, I've not tested it long enough but I'm guessing it loads the CPU more consistently without the spikes?? couldn't say, either way it isn't right! I will contact Rekor/OpenALPR and see what gives.

 

[wittaj]

It is bizarre, I've not tested it long enough but I'm guessing it loads the CPU more consistently without the spikes?? couldn't say, either way it isn't right! I will contact Rekor/OpenALPR and see what gives.

Yeah, if it stayed consistent I could live with it, but at least on mine when I tried, the GPU version would keep buffering/timing out and I would miss plates, which isn't good either LOL. And I would watch the GPU and it never got close to maxing out, so I am stumped on why it isn't more consistent.

 

[icpilot]

Yeah, if it stayed consistent I could live with it, but at least on mine when I tried, the GPU version would keep buffering/timing out and I would miss plates, which isn't good either LOL. And I would watch the GPU and it never got close to maxing out, so I am stumped on why it isn't more consistent.

I have a hunch it may be due to you running both OpenALPR and DeepStack on the GPU. When you view the applications running on your GPU, do you see both alprd.exe and blueiris.exe, or just one of them?




My system currently shows only alprd.exe, even though I am also running the DS GPU app, and my DS times are consistently sub 50ms, whereas they were never less than 200ms running the CPU version. CPU percentage (i9-9900k) is rock solid at 20% +/- 3%. Everything hums along great in this configuration, even though my CPU usage is higher than expected. Oh, and the process consuming more than half of the CPU is alprd.exe. Currently have 12 full-time cameras plus 1 full-time ALPR cam.

However, when I stopped DS in BI and then opened it in Powershell, I noticed the GPU showed blueiris.exe in the app list, and Rekor Scout Agent kept faltering.

Maybe something to look at ....

 

[CamCrazy]

I have a hunch it may be due to you running both OpenALPR and DeepStack on the GPU. When you view the applications running on your GPU, do you see both alprd.exe and blueiris.exe, or just one of them?

View attachment 103800


My system currently shows only alprd.exe, even though I am also running the DS GPU app, and my DS times are consistently sub 50ms, whereas they were never less than 200ms running the CPU version. CPU percentage (i9-9900k) is rock solid at 20% +/- 3%. Everything hums along great in this configuration, even though my CPU usage is higher than expected. Oh, and the process consuming more than half of the CPU is alprd.exe. Currently have 12 full-time cameras plus 1 full-time ALPR cam.

However, when I stopped DS in BI and then opened it in Powershell, I noticed the GPU showed blueiris.exe in the app list, and Rekor Scout Agent kept faltering.

Maybe something to look at ....

I see them both running on mine regardless but I'm not convinced that is a great indication based on other programs listed in that Nvidia panel. Interesting though, I will try a few things later when time permits.

 

[wittaj]

Well I got the GPU solely to run OpenALPR, so when I was trying it Deepstack GPU wasn't even part of the equation.

I only went to DeepStack GPU after I gave up on using the GPU for OpenALPR.

I will probably trying uninstalling and trying again at some point lol.

 

[CamCrazy]

Well I got the GPU solely to run OpenALPR, so when I was trying it Deepstack GPU wasn't even part of the equation.

I only went to DeepStack GPU after I gave up on using the GPU for OpenALPR.

I will probably trying uninstalling and trying again at some point lol.

I just tried re-installing Rekor so will let you know how that goes...OK, I give up for now, made no difference

Just to add, I flagged this with OpenALPR/Rekor who have been back to me and I have sent logs, although to me they look clean.

Response from Rekor so far:

"We were not able to find any issues in the logs, however, upon reaching out to our engineers we discovered that this is expected behaviour because Nvidia keeps part of the CPU running to synchronize communication between the two devices. This results in the CPU spikes you are seeing when enabling GPU acceleration. Please let me know if you have any additional questions or concerns."

So basically if you use GPU acceleration you can expect to load up your CPU around 3-4 times more than using CPU only - it makes some sense, maybe keep back the 8-10% that Rekor normally uses but then GPU takes full load, but not let Rekor take 35% just to sync communications ah well, I replied but I don't see this going anywhere fast.

 

[icpilot]

I just tried re-installing Rekor so will let you know how that goes...OK, I give up for now, made no difference

Just to add, I flagged this with OpenALPR/Rekor who have been back to me and I have sent logs, although to me they look clean.

Response from Rekor so far:

"We were not able to find any issues in the logs, however, upon reaching out to our engineers we discovered that this is expected behaviour because Nvidia keeps part of the CPU running to synchronize communication between the two devices. This results in the CPU spikes you are seeing when enabling GPU acceleration. Please let me know if you have any additional questions or concerns."

So basically if you use GPU acceleration you can expect to load up your CPU around 3-4 times more than using CPU only - it makes some sense, maybe keep back the 8-10% that Rekor normally uses but then GPU takes full load, but not let Rekor take 35% just to sync communications ah well, I replied but I don't see this going anywhere fast.

That seems consistent with what I am seeing as well, except that my CPU isn't spiking much, or at all. The CPU is clearly running at a higher load with Rekor running on the GPU than when it ran on the CPU alone. Overall though, with OpenALPR and Deepstack running on the GPU, my system is far more stable than before, and the CPU percentages are rock solid (barely budging with plate or object detection).

 

[CamCrazy]

That seems consistent with what I am seeing as well, except that my CPU isn't spiking much, or at all. The CPU is clearly running at a higher load with Rekor running on the GPU than when it ran on the CPU alone. Overall though, with OpenALPR and Deepstack running on the GPU, my system is far more stable than before, and the CPU percentages are rock solid (barely budging with plate or object detection).

That is interesting, I only have a 4th gen i5 and it sits at around 55% with GPU running in Rekor but only 15-19% with Rekor in CPU mode. I'd rather have stable CPU and no spikes, more testing to be done I guess but not happy with CPU sitting at 50% or more all the time. What resolution and fps are you running through Rekor? probably the wrong thread for this

 

[icpilot]

That is interesting, I only have a 4th gen i5 and it sits at around 35% with GPU running in Rekor but only 15-19% with Rekor in CPU mode. I'd rather have stable CPU and no spikes, more testing to be done I guess.

Mine is an i9-9900k and it is running at 35% +/- 3%. Earlier I said it was 20%, but that is what Blue Iris shows as CPU usage. Task Manager shows the higher load percent. In terms of apps, the APLR background process is using a pretty constant 12% of the CPU and Blue Iris uses a similarly constant 7% of the CPU.

The GPU is an Nvidia 1060 w/3 GB RAM. On ALPR detection it pops up to 5% to 8% load (CUDA) on the GPU. Never above 10%. During ALPR detection, the CPU load does not move at all. Rock solid.

I am unable to see the impact Deepstack is having on the GPU. I am, however, confident DS is using the GPU as improvement in detection times is dramatically improved from the CPU version. Almost never does a detection take more than 50ms now, and the CPU version would never take less than 200ms - and sometimes was going over 1,000ms.

 

[icpilot]

What resolution and fps are you running through Rekor? probably the wrong thread for this

Forgot to answer this part.

The camera is a Loryta IPC-HFW5241E-Z12E 2MP camera. The FPS rate is set to 10.

 

[CamCrazy]

Mine is an i9-9900k and it is running at 35% +/- 3%. Earlier I said it was 20%, but that is what Blue Iris shows as CPU usage. Task Manager shows the higher load percent. In terms of apps, the APLR background process is using a pretty constant 12% of the CPU and Blue Iris uses a similarly constant 7% of the CPU.

The GPU is an Nvidia 1060 w/3 GB RAM. On ALPR detection it pops up to 5% to 8% load (CUDA) on the GPU. Never above 10%. During ALPR detection, the CPU load does not move at all. Rock solid.

I am unable to see the impact Deepstack is having on the GPU. I am, however, confident DS is using the GPU as improvement in detection times is dramatically improved from the CPU version. Almost never does a detection take more than 50ms now, and the CPU version would never take less than 200ms - and sometimes was going over 1,000ms.

Thanks for the information, that processor is a beast with 8 cores so you should be in good shape regardless. GPU does have a big impact on Deepstack processing, interesting to see that even your i9 was giving quite high processing times before GPU intervention. I will see if Rekor come back with any updates today.

Just out of interest this is how my GTX1660 GPU Cuda load looks when 3-4 cameras are experiencing Deepstack events simultaneously:



Several spikes above 30% and constant use above 20% for a short period, that is with 1408 Cuda cores available. I also have the Ai set to '+ real time image' of between 10-30, analyse every 250ms.

Deepstack processing times with the above settings during the day range from 90 to 250ms.

 

[slabbel]

That looks interesting, did wonder if a new version was on the horizon

I just downloaded and installed the September release: DeepStack September Update