Deepstack : timeout

_Peek

Young grasshopper
Joined
Jun 1, 2020
Messages
38
Reaction score
11
Location
South
Hi,

Installed CUDA as per DeepStack for an Nvidia GPU on Windows.

However, all triggered events tend to time out.

1650169482263.png

Validating CUDA installation:

nvcc -version

1650169629687.png

nvidia-smi

1650169677633.png

deviceQuery <-
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vX.Y\extras\demo_suite

1650169859384.png


nbody.exe

1650170072439.png


oceanfft.exe

1650170132872.png

In BlueIris, DeepStack is set to start automatically and Tests all good in a web browser.

1650171239157.png

** Copy of deepstack.exe renamed to server.exe was done within the c:\DeepStack\server folder as to allow BlueIris to start Deepstack

What else is required for BlueIris to play nicely with Deepstack ?

-------------------------------------------------------

Windows 10 Pro - 21H2 ( 19044.1645 )
BlueIris 5.5.5.13

Nvidia GT 710
cuda_11.3.1_465.89_win10.exe
cudnn-windows-x86_64-8.4.0.27_cuda11.6-archive

DeepStack-Installer-GPU-2022.01.1.exe
 

Attachments

_Peek

Young grasshopper
Joined
Jun 1, 2020
Messages
38
Reaction score
11
Location
South
Got

RuntimeError: CUDA error: no kernel image is available for execution on the device

whilst "Get-Content .\stderr.txt" under c:\Users\User\AppData\Local\DeepStack\logs
 

MikeR33

Getting the hang of it
Joined
Jan 26, 2018
Messages
34
Reaction score
28
I'm running the same versions (working ok) and didn't need to rename the .exe file, maybe renaming it is actually stopping it working now?
 

_Peek

Young grasshopper
Joined
Jun 1, 2020
Messages
38
Reaction score
11
Location
South
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
 

MikeR33

Getting the hang of it
Joined
Jan 26, 2018
Messages
34
Reaction score
28
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
Understand but I don't think that's needed any longer (unless you are a lot of BI versions behind), I don't have a file called server.exe and it's all ok.

Did you extract all those folders\files that are required which go over the top of the Cuda install ok?

I would suggest clearing it all out and starting again, once it's in a bit of a mess through any error occurring it seems really tough to diagnose.

Hope you get it sorted.
 
Joined
Nov 2, 2021
Messages
21
Reaction score
12
Location
Canada
I've seen all these problems trying to get the latest version of deepstack working. Its obviously something specific to our setups causing these problems as others seem to be working.

I've seen the "no object detected" error as well as the "timeout" error.

With the latest version I noticed it did use a lot of memory on my GPU.. about double. And even more when you start to use custom models. I ended up removing all custom models to reduce the memory issues but still had problems.

Eventually I just downgraded back to the previous version and it works just fine.

It does look like they are working on some of these reported issues though. It may just be worth it for you to avoid having to fight with this any longer.

 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
@MikeR33 is correct. Copying and renaming the file is no longer necessary Blue Iris has been updated to work with the latest version of deep stack - as well as the older one.
 

_Peek

Young grasshopper
Joined
Jun 1, 2020
Messages
38
Reaction score
11
Location
South
Did you extract all those folders\files that are required which go over the top of the Cuda install ok?
Yip, followed deepstack's instructions to the letter in copying extracted CUDnn files into CUDA folder.

However, I didn't copy anything into the "windows_packages" subfolder of Deepstack as referenced at "GPU pytorch compile options for older card - i.e. GT710" and with the RuntimeError: CUDA error: no kernel image is available for execution on the device (c:\Users\User\AppData\Local\DeepStack\logs\stderr.txt) it has me currently fixated on compiling as an solution...

Now just to get an idea on the compiling process.
 

Cameraguy

Known around here
Joined
Feb 15, 2017
Messages
1,485
Reaction score
1,122
All of the sudden today my deepstack is returning timeout messages on every trigger.. I reverted back to bi 5.4.7.11 and its working but I was running a more current version and haven't updated in a while.. anyone have a clue why I'm now getting timeouts on newer bi versions.. deepstack is opening in web browser on all versions..
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
I've found DS and BI are very sensitive to each other. I was having timeouts and DS restarts for too often with some of the 5.5.6.x versions. Since 5.5.6.11 it's become very stable again. One of the things I've noticed is that when those timeouts occur there are a lot of instances of python.exe in memory, anywhere from 10 to 16. When it's stable all I ever see is 8. Exactly what it's related to I have no idea but at least the number of python.exe instances in memory can give a clue for stability.
 

Cameraguy

Known around here
Joined
Feb 15, 2017
Messages
1,485
Reaction score
1,122
I've found DS and BI are very sensitive to each other. I was having timeouts and DS restarts for too often with some of the 5.5.6.x versions. Since 5.5.6.11 it's become very stable again. One of the things I've noticed is that when those timeouts occur there are a lot of instances of python.exe in memory, anywhere from 10 to 16. When it's stable all I ever see is 8. Exactly what it's related to I have no idea but at least the number of python.exe instances in memory can give a clue for stability.
no clue where to find that
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
Bring up resource monitor in Win10. In the memory pane sort the applications alphabetically and scroll down to python.exe and count the from there.
 

Cameraguy

Known around here
Joined
Feb 15, 2017
Messages
1,485
Reaction score
1,122
Bring up resource monitor in Win10. In the memory pane sort the applications alphabetically and scroll down to python.exe and count the from there.
Well what the F.. I never install brand new updates.. but I was so irritated with these timeouts. Unreal seems like the newest update fixed it..
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
Glad it's working again. I updated to 5.5.6.13 yesterday and kept an eye on things. Memory utilization seems up a little, but not enough to worry about. DS has been stable as well. I also track what version of BI I'm running so I can revert back if things go sideways especially with DS.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,448
Reaction score
47,588
Location
USA
Many people have had issues with the newest version of Deepstack and going back to the previous one fixes the problem. So rollback either Deepstack or BI usually fixes it.
 

_Peek

Young grasshopper
Joined
Jun 1, 2020
Messages
38
Reaction score
11
Location
South
That's thus DS 2021-09-01 and BI 5.5.6.13 ?

My current working solution being DS in a docker container on XCP-NG host and BI 5.5.6.14 on physical Windows 10 and no issues.
 

Pentagano

Getting comfortable
Joined
Dec 11, 2020
Messages
576
Reaction score
269
Location
Uruguay
How do we get hold of the previous version of deepstack GPU 2021.. ?
I've been searching everywhere.

Thanks
 
Top