Deepstack : timeout

_Peek

_Peek

Young grasshopper
Jun 1, 2020
38
11
South
Hi,

Installed CUDA as per DeepStack for an Nvidia GPU on Windows.

However, all triggered events tend to time out.

1650169482263.png

Validating CUDA installation:

nvcc -version

1650169629687.png

nvidia-smi

1650169677633.png

deviceQuery <-
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vX.Y\extras\demo_suite

1650169859384.png


nbody.exe

1650170072439.png


oceanfft.exe

1650170132872.png

In BlueIris, DeepStack is set to start automatically and Tests all good in a web browser.

1650171239157.png

** Copy of deepstack.exe renamed to server.exe was done within the c:\DeepStack\server folder as to allow BlueIris to start Deepstack

What else is required for BlueIris to play nicely with Deepstack ?

-------------------------------------------------------

Windows 10 Pro - 21H2 ( 19044.1645 )
BlueIris 5.5.5.13

Nvidia GT 710
cuda_11.3.1_465.89_win10.exe
cudnn-windows-x86_64-8.4.0.27_cuda11.6-archive

DeepStack-Installer-GPU-2022.01.1.exe
 

Attachments

  • 1650170039234.png
    1650170039234.png
    29.4 KB · Views: 4
  • 1650170466426.png
    1650170466426.png
    49.6 KB · Views: 4
Got

RuntimeError: CUDA error: no kernel image is available for execution on the device

whilst "Get-Content .\stderr.txt" under c:\Users\User\AppData\Local\DeepStack\logs
 
I'm running the same versions (working ok) and didn't need to rename the .exe file, maybe renaming it is actually stopping it working now?
 
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
 
I think the GT710 may be too old, but could be wrong about that.
 
_Peek said:
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
Click to expand...

Understand but I don't think that's needed any longer (unless you are a lot of BI versions behind), I don't have a file called server.exe and it's all ok.

Did you extract all those folders\files that are required which go over the top of the Cuda install ok?

I would suggest clearing it all out and starting again, once it's in a bit of a mess through any error occurring it seems really tough to diagnose.

Hope you get it sorted.
 
I've seen all these problems trying to get the latest version of deepstack working. Its obviously something specific to our setups causing these problems as others seem to be working.

I've seen the "no object detected" error as well as the "timeout" error.

With the latest version I noticed it did use a lot of memory on my GPU.. about double. And even more when you start to use custom models. I ended up removing all custom models to reduce the memory issues but still had problems.

Eventually I just downgraded back to the previous version and it works just fine.

It does look like they are working on some of these reported issues though. It may just be worth it for you to avoid having to fight with this any longer.

1/2022 Deepstack release "no object detected" issue acknowledged by the developers. They are working on a fix. They are looking folks to help test.

There have been several posts indicating that some users have issues with the 1/2022 deepstack release. Most of these are related to specific newer video cards. Some users are having problems with the CPU version as well. The developers including @OlafenwaMoses (who are fantastic and doing this...
ipcamtalk.com ipcamtalk.com
 
_Peek said:
It's not renaming the file, it's making a copy of deepstack.exe and then renaming the copy to server.exe.

That the current worrkaround to have BI start/stop DS.
Click to expand...
@MikeR33 is correct. Copying and renaming the file is no longer necessary Blue Iris has been updated to work with the latest version of deep stack - as well as the older one.
 
  • Like
Reactions: sebastiantombs
MikeR33 said:
Did you extract all those folders\files that are required which go over the top of the Cuda install ok?
Click to expand...

Yip, followed deepstack's instructions to the letter in copying extracted CUDnn files into CUDA folder.

However, I didn't copy anything into the "windows_packages" subfolder of Deepstack as referenced at "GPU pytorch compile options for older card - i.e. GT710" and with the RuntimeError: CUDA error: no kernel image is available for execution on the device (c:\Users\User\AppData\Local\DeepStack\logs\stderr.txt) it has me currently fixated on compiling as an solution...

Now just to get an idea on the compiling process.
 
All of the sudden today my deepstack is returning timeout messages on every trigger.. I reverted back to bi 5.4.7.11 and its working but I was running a more current version and haven't updated in a while.. anyone have a clue why I'm now getting timeouts on newer bi versions.. deepstack is opening in web browser on all versions..
 
  • Wow
Reactions: sebastiantombs
I've found DS and BI are very sensitive to each other. I was having timeouts and DS restarts for too often with some of the 5.5.6.x versions. Since 5.5.6.11 it's become very stable again. One of the things I've noticed is that when those timeouts occur there are a lot of instances of python.exe in memory, anywhere from 10 to 16. When it's stable all I ever see is 8. Exactly what it's related to I have no idea but at least the number of python.exe instances in memory can give a clue for stability.
 
  • Like
Reactions: jrbeddow and Cameraguy
sebastiantombs said:
I've found DS and BI are very sensitive to each other. I was having timeouts and DS restarts for too often with some of the 5.5.6.x versions. Since 5.5.6.11 it's become very stable again. One of the things I've noticed is that when those timeouts occur there are a lot of instances of python.exe in memory, anywhere from 10 to 16. When it's stable all I ever see is 8. Exactly what it's related to I have no idea but at least the number of python.exe instances in memory can give a clue for stability.
Click to expand...
no clue where to find that
 
  • Like
Reactions: sebastiantombs
Bring up resource monitor in Win10. In the memory pane sort the applications alphabetically and scroll down to python.exe and count the from there.
 
  • Like
Reactions: Cameraguy
sebastiantombs said:
Bring up resource monitor in Win10. In the memory pane sort the applications alphabetically and scroll down to python.exe and count the from there.
Click to expand...
Well what the F.. I never install brand new updates.. but I was so irritated with these timeouts. Unreal seems like the newest update fixed it..
 
  • Like
Reactions: sebastiantombs
Glad it's working again. I updated to 5.5.6.13 yesterday and kept an eye on things. Memory utilization seems up a little, but not enough to worry about. DS has been stable as well. I also track what version of BI I'm running so I can revert back if things go sideways especially with DS.
 
  • Like
Reactions: JNDATHP and Cameraguy
Many people have had issues with the newest version of Deepstack and going back to the previous one fixes the problem. So rollback either Deepstack or BI usually fixes it.
 
  • Like
Reactions: Cameraguy
That's thus DS 2021-09-01 and BI 5.5.6.13 ?

My current working solution being DS in a docker container on XCP-NG host and BI 5.5.6.14 on physical Windows 10 and no issues.
 
You must log in or register to reply here.
Top Bottom