Help with deepstack gpu for windows

SheriffWoody95 · May 24, 2022

PeteyPete said:
Interesting. I re-downloaded the 2021.09.1 version I linked and also the newer 2022.01.1 version from that site.
Tested on two different machines and I don't get an error (Chome reported no warning on the download and Windows Defender had no issue with a manual scan or during install).

Is your server patched? What versions are your protection signatures?
On my two test machines I have:

Code:

Get-MpComputerStatus |select *Version

AMEngineVersion : 1.1.19000.8 AMProductVersion : 4.18.2202.4 AMServiceVersion : 4.18.2202.4 AntispywareSignatureVersion : 1.361.1057.0 AntivirusSignatureVersion : 1.361.1057.0 FullScanSignatureVersion : NISEngineVersion : 1.1.19000.8 NISSignatureVersion : 1.361.1057.0 QuickScanSignatureVersion : 1.361.996.0

One of the machines is slightly older
QuickScanSignatureVersion : 1.361.701.0

Sorry for the really late reply to this... everything on my BI server checked out as you mentioned, but still the downloaded DeepStack GPU package refused to install with the same issue as before - spinning blue wheel for a couple of seconds, then nothing - no errors, no messages... just nothing.

However, I managed to figure out what was going on (and the reason for this reply is hopefully to help anyone else who has the same issue).

When downloading the file, Windows Defender throws a hissy fit about the "file is not downloaded often and could be malicious. Download anyway?". I clicked to download anyway and got the file in my downloads folder. However, despite agreeing to Windows Defender's demands it turns out that it still tags the file properties as "blocked"... but doesn't tell you it's done so, and there are no messages telling you when you attempt to open the file!

It is this "blocked" property that prevents the install from running.

Here's the URL of the article that explained it: Microsoft Defender SmartScreen Prevents Software Installation

Once the blocked property was removed, the DeepStack GPU installer ran and now my BI server CPU is back down running at 30-40% (rather that permanently pegged at 99%) with fast and accurate DeepStack person detection offloaded to my NVidia GT 1030.

Thanks for assistance!

PeteyPete · May 25, 2022

jleemhuis said:
I got deepstack GPU up and running on blue iris and it is identifying objects but it feels like the times are longer than they should be. I see anywhere from 800ms to 3500ms. There is no doubt I probably messed something up along the way but curious about what effects those times.

Those sounds like CPU response times rather than GPU response times, unless you're using huge models.
I'm have a lowly GT1030 and I'm seeing response times around ~150ms.
What models are you using?
Edit: Also check whether you're sending your cameras main stream, or sub stream for analysis. If you have high resolution cameras then it will be sending very large images from the main stream for analysis.
It's worth checking task manager to see if your CUDA cores are being utilised during deepstack operations.
You might need to change the view slightly per my screenshot:

jleemhuis · May 25, 2022

PeteyPete said:
Those sounds like CPU response times rather than GPU response times, unless you're using huge models.
I'm have a lowly GT1030 and I'm seeing response times around ~150ms.
What models are you using?
Edit: Also check whether you're sending your cameras main stream, or sub stream for analysis. If you have high resolution cameras then it will be sending very large images from the main stream for analysis.
It's worth checking task manager to see if your CUDA cores are being utilised during deepstack operations.
You might need to change the view slightly per my screenshot:
View attachment 128963

I actually went to using an older version of deepstack and it is down into the 2-400 range on High so I think that solved the problem.

eeeeesh · May 28, 2022

I just changed from DeepStack CPU to DeepStack GPU and I think it sounds more complicated than it is - there are several different versions and comments floating around...

I recently purchased a Quadro P400 v2 and I have it installed in my system with the motherboard bios set to use onboard graphics. So in Windows, the Task Manager will show two separate GPU's. Also, I used the latest version of CUDA from NVIDIA (11.3.1)

I wrote up the procedure I used - (attached text file). It 'seems' to be working fine, but if any one sees anything I missed or has some tips....

The main webpage with instructions and links can be found here: Using DeepStack with Windows 10 (CPU and GPU)

Jimmylavino · May 28, 2022

Thanks for the write up, but I am still having problems. It is really confusing when the cuDNN install instructions reference copying the downloaded files into the "C:\Program Files\NVIDIA\CUDNN" directory that doesn't exisit =(

eeeeesh said:
I just changed from DeepStack CPU to DeepStack GPU and I think it sounds more complicated than it is - there are several different versions and comments floating around...

I recently purchased a Quadro P400 v2 and I have it installed in my system with the motherboard bios set to use onboard graphics. So in Windows, the Task Manager will show two separate GPU's. Also, I used the latest version of CUDA from NVIDIA (11.3.1)

I wrote up the procedure I used - (attached text file). It 'seems' to be working fine, but if any one sees anything I missed or has some tips....

The main webpage with instructions and links can be found here: Using DeepStack with Windows 10 (CPU and GPU)

View attachment 129304

eeeeesh · May 28, 2022

Jimmylavino said:
Thanks for the write up, but I am still having problems. It is really confusing when the cuDNN install instructions reference copying the downloaded files into the "C:\Program Files\NVIDIA\CUDNN" directory that doesn't exisit =(

Where did you read that? (I think that is part of the problem - too many versions floating around. Keep it simple.) Use the two files on the main DeepStack Web page
(Keep in mind, I am not the expert on this but I did just go through this and it appeared to work for me. If I made an error somewhere, please let me know)

Short Version for Win10
1 - Install CUDA 11.3.1
2- Copy the files from cuDNN zip file into the matching folders
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

CUDA 11.3.1 creates all the folders
cuDNN is just a zip file. You open it up, or extract it somewhere (I always use WinRAR) and copy the files to the matching folders

files in the bin folder that get copied over (all .dll files)

files in the include folder that get copied over (all .h files)

files in the lib folder that get copied over (all .lib files)

Jimmylavino · May 28, 2022

Thanks for the details!

This is from the cuDNN Windows install guide showing the wrong folder locations:

eeeeesh said:
Where did you read that? (I think that is part of the problem - too many versions floating around. Keep it simple.) Use the two files on the main DeepStack Web page
(Keep in mind, I am not the expert on this but I did just go through this and it appeared to work for me. If I made an error somewhere, please let me know)

Short Version for Win10
1 - Install CUDA 11.3.1
2- Copy the files from cuDNN zip file into the matching folders
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

View attachment 129312

CUDA 11.3.1 creates all the folders
cuDNN is just a zip file. You open it up, or extract it somewhere (I always use WinRAR) and copy the files to the matching folders

View attachment 129311

files in the bin folder that get copied over (all .dll files)
View attachment 129313

files in the include folder that get copied over (all .h files)
View attachment 129314

files in the lib folder that get copied over (all .lib files)
View attachment 129315

eeeeesh · May 29, 2022

Jimmylavino said:
Thanks for the details!

This is from the cuDNN Windows install guide showing the wrong folder locations:

View attachment 129320

Ya, I got all twisted up trying to follow that also - install zlib, yada yada yada. (I thought that may have been were your read it. Here is the link.

Overview — NVIDIA cuDNN

docs.nvidia.com

)

That is why I made my post. It doesn't appear to be really that complicated...

There is so much info floating around - install CUDA tool kit, and then the two upgrades, etc etc. If you look, most of the posts were over a year ago. Things change. It looks to be really simple now. Like I posted. You download and install one program, and you download a zipped archive and copy some files over. That's the short version. The text file I attached has some more info

Everything points to my setup working. The log file shows that when Blue Iris starts, both of my GPU's are detected

I also have log entries showing DeepStack working

I watch as a car drives by and less than a second later, I have an alert with DeepStack boxes showing the vehicle. My P400 GPU shows a spike in Task Manager while the CPU percentage remains flat.

This intersection is 300' away (using a 25x Dahua) and it properly identified someone riding a bicycle!

These cars are going by at 30 mph+ and that is about 50' of roadway in view. Looks like it's working to me?

MikeLud1 · May 29, 2022

eeeeesh said:
Ya, I got all twisted up trying to follow that also - install zlib, yada yada yada. (I thought that may have been were your read it. Here is the link.

Overview — NVIDIA cuDNN

docs.nvidia.com

)

That is why I made my post. It doesn't appear to be really that complicated...

There is so much info floating around - install CUDA tool kit, and then the two upgrades, etc etc. If you look, most of the posts were over a year ago. Things change. It looks to be really simple now. Like I posted. You download and install one program, and you download a zipped archive and copy some files over. That's the short version. The text file I attached has some more info

Everything points to my setup working. The log file shows that when Blue Iris starts, both of my GPU's are detected
View attachment 129323

I also have log entries showing DeepStack working
View attachment 129324

I watch as a car drives by and less than a second later, I have an alert with DeepStack boxes showing the vehicle. My P400 GPU shows a spike in Task Manager while the CPU percentage remains flat.

This intersection is 300' away (using a 25x Dahua) and it properly identified someone riding a bicycle!
View attachment 129325

These cars are going by at 30 mph+ and that is about 50' of roadway in view. Looks like it's working to me?
View attachment 129326

You do not need to send Main Stream images to DeepStack because DeepStack will resize the image to 640 x 640 if set to High , 416 x 416 if set to Medium and 256 x 256 for Low (see DeepStack Code below). This resizing slows down DeepStack, you will get the better detection times with sending Sub Stream images with the same accuracy. Also to get the best accuracy set the mode to High not the default Blue Iris setting of Medium. The default DeepStack and my models are optimized for High (640 x 640)

Code:

"desktop_gpu": Settings(
            DETECTION_HIGH=640,
            DETECTION_MEDIUM=416,
            DETECTION_LOW=256,

Jimmylavino · May 29, 2022

eeeeesh said:
Where did you read that? (I think that is part of the problem - too many versions floating around. Keep it simple.) Use the two files on the main DeepStack Web page
(Keep in mind, I am not the expert on this but I did just go through this and it appeared to work for me. If I made an error somewhere, please let me know)

Short Version for Win10
1 - Install CUDA 11.3.1
2- Copy the files from cuDNN zip file into the matching folders
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

View attachment 129312

CUDA 11.3.1 creates all the folders
cuDNN is just a zip file. You open it up, or extract it somewhere (I always use WinRAR) and copy the files to the matching folders

View attachment 129311

files in the bin folder that get copied over (all .dll files)
View attachment 129313

files in the include folder that get copied over (all .h files)
View attachment 129314

files in the lib folder that get copied over (all .lib files)
View attachment 129315

Note that the lib folder on the target directory includes a Win32 and x64 folder. I assume you copied the cuDNN files to the x64 folder?

Jimmylavino · May 29, 2022

Well, despite the great install guide I am still faced with the dreaded Deepstack timeout error. CPU version of Deepstack works fine, just slow to respond (2000ms+) on my older Dell Xeon server. I am using sub-streams.

I now think the problem is my super old GPU (GeForce GT 710). I double checked the CUDA GPU support tables (CUDA GPUs - Compute Capability) and curiously they show support for the GeForce GT 430, 440, 520, 620, 630, 640, 705...but skip over 710 and go right to 720, 730. Maybe it is time to buy a better GPU....

eeeeesh · May 29, 2022

Jimmylavino said:
Note that the lib folder on the target directory includes a Win32 and x64 folder. I assume you copied the cuDNN files to the x64 folder?

Actually.... I just dumped them into the root of the lib folder. I thought if they wanted them in the Win32 or x64 folder they would have had those folders in the archive? Maybe someone else can weigh in????

PeteyPete · May 29, 2022

Jimmylavino said:
I now think the problem is my super old GPU (GeForce GT 710). I double checked the CUDA GPU support tables (CUDA GPUs - Compute Capability) and curiously they show support for the GeForce GT 430, 440, 520, 620, 630, 640, 705...but skip over 710 and go right to 720, 730. Maybe it is time to buy a better GPU....

The GT1030 also isn't listed, but works fine. I suggest checking with a tool such as Cuda-Z ( more info in my earlier post here: Help with deepstack gpu for windows ).
Edit: to be clearer, it's a combination of whether your card supports CUDA and the version of cuda it supports.

eeeeesh · May 29, 2022

I'm impressed!

Car driving by with the passenger window down and shows the passenger as a 'person' (credit to MikeLud1 for his custom models)

Jimmylavino · May 30, 2022

PeteyPete said:
The GT1030 also isn't listed, but works fine. I suggest checking with a tool such as Cuda-Z ( more info in my earlier post here: Help with deepstack gpu for windows ).
Edit: to be clearer, it's a combination of whether your card supports CUDA and the version of cuda it supports.

Thanks for the Cuda-Z tip. Apparently the GT 710 supports compute capability of 3.5. Is anyone running Deepstack GPU with this level of support?

Jimmylavino · May 30, 2022

Two other pieces of information... 1) I don't see any Deepstack process running under nvidia-smi.exe , 2) I don't see a drop down for Cuda in the windows performance monitor.

Jimmylavino · May 30, 2022

I came across this post with user _Peek and the same GT 710 GPU. Looks like the issue is lack of Pytorch support for compute capabilities of 3.5

. So either I learn how to compile or buy a newer GPU. I am quickly approaching the point where throwing new hardware at the problem might be the best solution.

Deepstack : timeout

Hi, Installed CUDA as per DeepStack for an Nvidia GPU on Windows. However, all triggered events tend to time out. Validating CUDA installation: nvcc -version nvidia-smi deviceQuery

ipcamtalk.com

GPU pytorch compile options for older card - i.e. GT710

Hello, I was curious if anyone has managed to compile pytorch to get Deepstack working with older cards where Compute Compatability is <= 3.5? I have successfully compiled pytorch as per: GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration and...

forum.deepstack.cc

eeeeesh · May 30, 2022

Jimmylavino said:
Two other pieces of information... 1) I don't see any Deepstack process running under nvidia-smi.exe , 2) I don't see a drop down for Cuda in the windows performance monitor.

View attachment 129399

View attachment 129400

I don't have a 'Cuda' dropdown either - I noticed that the other day, but everything seems to be working fine? (The CPU version of DeepStack was completely removed)

Quadro P400

Onboard Intel 630 graphics

Jimmylavino · Jun 4, 2022

In case this helps anyone else I purchased an ASUS GT 1030 2GB GDDR5 for around $100 and now Deepstack GPU works great! (Note: Beware of the GT 1030 cards with inferior GDDR4 RAM. Apparently some manufacturers quietly downgraded).

In the end it was my old GT 710 that didn't support the newer CUDA version required for Deepstack. The GT 710 supports compute capability 3.5 while the GT 1030 supports 6.1. I didn't feel adventurous enough to attempt forcing the GT 710 to work via a freshly compiled version of Pytorch, but someone else may want to try it: GPU pytorch compile options for older card - i.e. GT710 . Credit to loopy12 .

I am getting processing times in the 100-200ms range now with the GT 1030.

Thanks for the help eeeeesh and PeteyPete.

Cameraguy · Jun 4, 2022

Jimmylavino said:
In case this helps anyone else I purchased an ASUS GT 1030 2GB GDDR5 for around $100 and now Deepstack GPU works great! (Note: Beware of the GT 1030 cards with inferior GDDR4 RAM. Apparently some manufacturers quietly downgraded).

In the end it was my old GT 710 that didn't support the newer CUDA version required for Deepstack. The GT 710 supports compute capability 3.5 while the GT 1030 supports 6.1. I didn't feel adventurous enough to attempt forcing the GT 710 to work via a freshly compiled version of Pytorch, but someone else may want to try it: GPU pytorch compile options for older card - i.e. GT710 . Credit to loopy12 .

I am getting processing times in the 100-200ms range now with the GT 1030.

Thanks for the help eeeeesh and PeteyPete.

Yea my cheapo Quadro k620 is getting 100-200ms. Also check them out for like 35 bucks on ebay

Help with deepstack gpu for windows

n3wb

n3wb

n3wb

BIT Beta Team

Attachments

n3wb

BIT Beta Team

n3wb

BIT Beta Team

n3wb

n3wb

BIT Beta Team

n3wb

BIT Beta Team

n3wb

Attachments

n3wb

n3wb

BIT Beta Team

n3wb

Attachments

Known around here