Help with deepstack gpu for windows

[DigitalPackrat]

Thanks for this! That is definitely something I didn't try before switching back to the CPU version.

Can I ask what version of CUDA you're running? Is there an order of operations I might be missing? I followed the instructions for cuDNN and I just can't seem to get everything to "stick".

Also, do I need Visual Studio installed? I'm guessing, no?

EDIT: I got it working! Thank you so much! I think you were right, the Blue Iris starting the service seemed to be the issue. The desktop starting from shortcut didn't seem to fully do it though. What seemed to do the trick was shutting down BlueIris completely, using PowerShell to run:

deepstack --VISION-SCENE=True --VISION-DETECTION=True --VISION-FACE=True --PORT 80

verifying that it worked in the browser (127.0.0.1:80) and then firing up Blue Iris.

For others, I ended up completely uninstalling Deepstack CPU (via Control Panel AND deleting the leftover folders). I think installed Cuda 10.1 (no updates),downloaded cudnn-10.2, unpacked and added the files to the directories and made sure the environmental variables were present. I installed Deepstack GPU and then ran that code I referenced above in PowerShell, I then started up Blue Iris and made sure the AI/Deepstack settings matched ports/options that I selected and viola! Up and running.

I installed CUDA 10.1 with update 1 and 2. I did uncheck Visual Studio.

 

[CamCrazy]

I installed CUDA 10.1 with update 1 and 2. I did uncheck Visual Studio.

Have you managed to get this working OK yet? I've just been through the process and 'think' now it is working. I did have to install latest Nvidia drivers, Cuda 10.1 and also both updates 1 and 2 for what they were worth

Did have some issues with it starting along with Blue Iris but these seem fine now, some tweaking required with night models in deepstack compared to CPU version but results seem positive so far.

 

[IReallyLikePizza2]

I just installed cuda_10.1.243_426.00_win10 (The Update 2) , copied over the cudnn files, the environmental variable was already created. Stopped BI, uninstalled Deepstack and deleted folder, installed GPU version

Everything just worked, no problems

 

[IReallyLikePizza2]

By the way guys, no need to install the updates one at a time, I'm 99.9% sure they are all rollups, so you just need to install the latest

 

[PeteyPete]

TL;DR: If you're running a Windows Server Core installation, you're probably going to need VC_redist.x64.exe.


I just spent a good few hours chasing my tail trying to get DeepStack working properly with a GT 1030 on Windows Server 2019 Core.
Posting my findings here in case it helps someone else.

Versions and installation order I used based on suggestions from Page 1 of this thread:

1. Installed cuda_10.1.243_426.00_win10.exe (I deselected the GEForce Experience and Visual Studio Integration options)
2. Installed CUDNN cudnn-10.2-windows10-x64-v8.3.0.98.zip
3. Followed the NVIDIA instructions to set the environment variable and copy the various toolkit files, per Installation Guide :: NVIDIA Deep Learning cuDNN Documentation
4. Installed DeepStack-Installer-GPU-2021.09.1.exe

All my attemps to perform object detection would fail with a 500 timeout.

Failure on the deepstack server:
[GIN] 2021/11/16 - 21:56:54 |[97;41m 500 [0m| 1m0s | 192.168.6.1 |[97;46m POST [0m "/v1/vision/detection"

Failure reponse on the client:
~$ curl -X POST -F image=@person.jpg 'http://192.168.7.41:8999/v1/vision/detection'
{"success":false,"error":"failed to process request before timeout","duration":0}


I went in circles for ages, testing different models, cuda/cudnn versions, etc, and thinking it must have been CUDA memory related (I only have a 2GB GT1030).
In the end I stumbled upon a deepstack forum post which mentioned VC 2019 Redist.
After installing VC_redist.x64.exe and restarting deepstack everything started working.

Success on the deepstack server:
[GIN] 2021/11/16 - 22:08:06 |[97;42m 200 [0m| 113.706ms | 192.168.6.1 |[97;46m POST [0m "/v1/vision/detection"

Success response on the client:
~$ curl -X POST -F image=@person.jpg 'http://192.168.7.41:8999/v1/vision/detection'
{"success":true,"predictions":[{"confidence":0.85498047,"label":"person","y_min":80,"x_min":108,"y_max":352,"x_max":272}],"duration":0}



Hopefully this helps someone else out there.

 

[PeteyPete]

Stats for those curious about the memory usage - running on medium and no custom models - it's about 630MB

C:\>"C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe"
Tue Nov 16 22:36:27 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 426.00       Driver Version: 426.00       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 1030    WDDM  | 00000000:01:00.0 Off |                  N/A |
|  0%   30C    P8    N/A /  30W |    632MiB /  2048MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      3432      C   C:\DeepStack\interpreter\python.exe        N/A      |
+-----------------------------------------------------------------------------+

 

[MicahJames]

thank you all for feedback and trouble shooting in here. Unfortunately I’m still having a deep stack timeout error not sure if anyone else has seen or fixed this.
I installed new NVIDIA drivers. Installed CUDA and both updates. Installed cudnn in the correct directory. Even uninstalled deepstack and did a fresh install. It shows it’s activated if u navigate to the internal IP but BI isn’t getting a reply from it.
Any ideas ?

 

[PeteyPete]

I installed new NVIDIA drivers. Installed CUDA and both updates. Installed cudnn in the correct directory. Even uninstalled deepstack and did a fresh install. It shows it’s activated if u navigate to the internal IP but BI isn’t getting a reply from it.
Any ideas ?

That timeout behaviour is what I saw when Deepstack wasn't working properly.
Just because the web server is responding doesn't necessarily mean the models have loaded and detection works.

I suggest running the deepstack command manually and observing the output.
Something like c:\deepstack\deepstack.exe --VISION-DETECTION True --PORT <port number>

Then observe the servers response when detection is called
Eg:
[GIN] 2021/11/16 - 21:56:54 |[97;41m 500 [0m| 1m0s | 192.168.6.1 |[97;46m POST [0m "/v1/vision/detection"
(HTTP 500 is an error, HTTP 200 is a success, etc).

You can also use nvidia-smi.exe to see whether the deepstack/python process is running on the GPU. See my post above

 

[MicahJames]

That timeout behaviour is what I saw when Deepstack wasn't working properly.
Just because the web server is responding doesn't necessarily mean the models have loaded and detection works.

I suggest running the deepstack command manually and observing the output.
Something like c:\deepstack\deepstack.exe --VISION-DETECTION True --PORT <port number>

Then observe the servers response when detection is called
Eg:
[GIN] 2021/11/16 - 21:56:54 |[97;41m 500 [0m| 1m0s | 192.168.6.1 |[97;46m POST [0m "/v1/vision/detection"
(HTTP 500 is an error, HTTP 200 is a success, etc).

You can also use nvidia-smi.exe to see whether the deepstack/python process is running on the GPU. See my post above

Thank you I will try this. Also I wonder if it’s because my nvidia card is too old. It’s a GeForce GTX 460 =\

 

[PeteyPete]

Thank you I will try this. Also I wonder if it’s because my nvidia card is too old. It’s a GeForce GTX 460 =\

You might be at the limits of the card. It only supports compute 2.1 according to this document.
Interestingly my GT 1030 isn't listed on that site (others have mentioned this too), yet it supports compute 6.1.
A tool like CUDA-Z will provide further info. Eg:


My (limited) understanding is CUDNN and PyTorch need much higher compute levels than your 2.1 card offers.
There might be some hackarounds to use older versions of CUDNN & CUDA, along with much older YOLO (or other) custom models, but that's well beyond my level of knowledge!

Here in AUS a GT1030 can be found for <AUD$150, so you probably have even cheaper options in the States.

 

[MicahJames]

Thanks Pete! I really appreciate it

 

[OfficerD00fy]

Having issues guys getting my 1060 working. Deepstack just times out. Will try the 2 updates for Cuda 10.1

 

[autumnwalker]

TL;DR: If you're running a Windows Server Core installation, you're probably going to need VC_redist.x64.exe.

This is what I needed. Thank you!

 

[SheriffWoody95]

Quick question re. installing DeepStack GPU on Windows...

Running Blue Iris on Windows Server 2016, 6th Gen i5, 20GB RAM & Nvidia GT 1030.

All Nvidia prerequisites were installed successfully as per documentation (Using DeepStack with Windows 10 (CPU and GPU)). Downloaded the DeepStack GPU installer and Windows Defender took exception which I overrode and completed the download.

However, when I try and run the installer, the "blue donut" spins for a couple of seconds and then stops... but nothing happens. No installer, nothing.

Have tried setting compatibility with other versions of Windows, with the same outcome.

No matter what I do, I cannot get the DeepStack GPU installer to start the installation.

Any ideas anyone?

 

[IReallyLikePizza2]

Tried running as Admin?

But no issue here, I'm on Server 2019 and it just worked

 

[PeteyPete]

No matter what I do, I cannot get the DeepStack GPU installer to start the installation.

I haven't tested the 2022.01.1 build of DeepStack GPU yet so I don't know how it behaves under Windows server.
Maybe try an older version? 2021.09.1 is working for me

 

[SheriffWoody95]

Tried running as Admin?

But no issue here, I'm on Server 2019 and it just worked

Thanks for the reply, I appreciate it!

I've tried running as admin but still nothing. However, I think I've figured something out...

The CPU version has been running on my server since January when I downloaded the installer from the Github source. The versions I download now from "docs.deepstack.cc" don't come from come from Github anymore but "deepquest.sfo2.digitaloceanspaces.com" and no matter what version I download from here they all report potential malicious code (although more likely a false positive) and fail to start.

I'll jump on the Deepstack forums and see whether anyone else has noticed this.

Otherwise, I'm back on the old CPU version I downloaded in January...

Thanks again

 

[PeteyPete]

The versions I download now from "docs.deepstack.cc" don't come from come from Github anymore but "deepquest.sfo2.digitaloceanspaces.com" and no matter what version I download from here they all report potential malicious code (although more likely a false positive) and fail to start.

Interesting. I re-downloaded the 2021.09.1 version I linked and also the newer 2022.01.1 version from that site.
Tested on two different machines and I don't get an error (Chome reported no warning on the download and Windows Defender had no issue with a manual scan or during install).

Is your server patched? What versions are your protection signatures?
On my two test machines I have:

Get-MpComputerStatus |select *Version

AMEngineVersion : 1.1.19000.8
AMProductVersion : 4.18.2202.4
AMServiceVersion : 4.18.2202.4
AntispywareSignatureVersion : 1.361.1057.0
AntivirusSignatureVersion : 1.361.1057.0
FullScanSignatureVersion :
NISEngineVersion : 1.1.19000.8
NISSignatureVersion : 1.361.1057.0
QuickScanSignatureVersion : 1.361.996.0

One of the machines is slightly older
QuickScanSignatureVersion : 1.361.701.0

 

[jleemhuis]

I got deepstack GPU up and running on blue iris and it is identifying objects but it feels like the times are longer than they should be. I see anywhere from 800ms to 3500ms. There is no doubt I probably messed something up along the way but curious about what effects those times.

 

[Coldair]

does anyone have an easy setup for deepstack complete with models?