Deepstack Times to high?

[plainmad]

I’ve got BlueIris running on a Win 10 PC which is based on an i5 4440 with 8gb of ram. It has an SSD with various HDD for Plex media storage and a dedicated NVR drive for blue iris.

When I first installed deepstack I was getting response time of around 500ms and read that having a GPU can make a massive difference with people reporting times of under 100ms. A such I went and got a Nvidia P400 and uninstalled the CPU deepstack and installed the GPU version

however my response times still seem to be around 400ms and I haven’t seen the improvement I expected. Has anyone got any ideas on what might be causing this?

 

[wittaj]

Did you follow the steps to get the GPU to work with it. It is more than just installing the GPU version.

Have you looked at Task Manager to confirm the GPU is in operation?

Which version BI are you on. Some of them have been problematic.

 

[iwanttosee]

400ms sounds about right for a P400.
It only has 256 cuda cores

 

[jaydeel]

Not a solution, but perhaps a benchmark…
For over 90% of my triggers, I’m routinely seeing a 100-150 ms processing time for this hardware & configuration:

• I7-4770 processor
• PNY NVIDIA Quadro P400 V2
• 16 GB RAM
• 500 GB SSD
• 8 TB Purple hard drive

My DeepStack settings are:

• version 2021.09.01
• 1 instance
• ’High’ mode
• EDIT (3/21/22): ‘Medium’ mode yields a 70-110 ms processing time.

My camera settings are:

• using only the ‘Combined’ custom model, day & night
• 5 extra images each 500/750ms (500ms for street facing cams)
• ‘Use main stream…’ disabled
• ’Use motion-leading image’ enabled only for street-facing cams

My Blue Iris version is current (5.5.5.12).

More details here…

DeepStack Case Study: Performance from CPU to GPU version

I'm posting this thread for anyone considering making the transition from the DeepStack CPU version to the GPU version. Hopefully this will give you an impression of what you can expect in terms of enhanced performance. Please note some of my observations are system-specific. My system specs...

ipcamtalk.com

 

[plainmad]

Did you follow the steps to get the GPU to work with it. It is more than just installing the GPU version.

Have you looked at Task Manager to confirm the GPU is in operation?

Which version BI are you on. Some of them have been problematic.

I believe I have although the only step I might have an issue on was if I’ve got the correct packages for VisualStudio?
Howver if I look under what programs are running on the GPU it does show a Python program which only shows when BlueIris is running.
BlueIris is 5.5.5.9

 

[plainmad]

Not a solution, but perhaps a benchmark…
For over 90% of my triggers, I’m routinely seeing a 100-150 ms processing time for this hardware:

• I7-4770 processor
• 16 GB RAM
• 500 GB SSD
• 8 TB Purple hard drive
• PNY NVIDIA Quadro P400 V2

My DeepStack settings are:

• version 2021.09.01
• 1 instance
• ’High’ mode

My camera settings are:

• using only the ‘Combined’ custom model, day & night
• 5 extra images each 500/750ms (500ms for street facing cams)
• ‘Use main stream…’ disabled
• ’Use motion-leading image’ enabled only for street-facing cams

My Blue Iris version is current (5.5.5.12).

More details here…

DeepStack Case Study: Performance from CPU to GPU version

I'm posting this thread for anyone considering making the transition from the DeepStack CPU version to the GPU version. Hopefully this will give you an impression of what you can expect in terms of enhanced performance. Please note some of my observations are system-specific. My system specs...

ipcamtalk.com

This is the sort of time I’m looking for. I’ll have to try tweaking the settings for it and see what I get. Where is the combined custom model available?

 

[jaydeel]

Where is the combined custom model available?

See the attachments to this post

IP Cam Talk Custom Community DeepStack Model

I figured out how to download the images that DeepStack used to train the object model. With these images I can start to make the community custom DeepStack model, below are the steps to create the custom model. The first step would be to create a new DeepStack custom model using the same image...

ipcamtalk.com

 

[plainmad]

So I’ve just had a look on my BlueIris server and have ran a clip through with it doing deepstack live as you can do. My CPU spikes massively and when the clip carries on playing but with deepstack turned off it reduces. This along with the GPU not changing makes me think it’s not using the GPU for decoding.
Is there an idiots guide for installing GPU Deepstack. I thought I’d done it correctly but obviously not. I suspect I’m getting stuck with visual studio

 

[sebastiantombs]

Shut down BI. Uninstall the existing DS installation. Check to make sure the DS directory is entirely gone, if not delete whatever remains. Reboot the machine. Make sure BI is not running. Then install the GPU version and be sure to successfully complete every step in the order listed in the instructions. Visual Studio is a key component so it, too, needs to fully install. I'm running DS 2021.02.1 if that helps you.

DeepStack Installation

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

 

[Nunofya]

Shut down BI. Uninstall the existing DS installation. Check to make sure the DS directory is entirely gone, if not delete whatever remains. Reboot the machine. Make sure BI is not running. Then install the GPU version and be sure to successfully complete every step in the order listed in the instructions. Visual Studio is a key component so it, too, needs to fully install. I'm running DS 2021.02.1 if that helps you.

DeepStack Installation

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

Do you have to delete your DS custom folder that has dark, general in it? Also, to shut down BI, do you go through task manager, since it is running as a service? Don't have a lot experience with windows. How do you exactly check the DS directory to make sure its entirely gone? Would be nice to know the exact steps cause I might need to delete and reinstall BI to. Would the steps be the same?
Thanks for any help.

 

[plainmad]

Shut down BI. Uninstall the existing DS installation. Check to make sure the DS directory is entirely gone, if not delete whatever remains. Reboot the machine. Make sure BI is not running. Then install the GPU version and be sure to successfully complete every step in the order listed in the instructions. Visual Studio is a key component so it, too, needs to fully install. I'm running DS 2021.02.1 if that helps you.

DeepStack Installation

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

I will try that.
Stupid question but what visual studi do I need to download and do I need the additional packages like c++?

 

[plainmad]

Anybody able to advice on which Visual studio needs to be installed and any other packages with it?
I've got 2019 currently installed with no additional packages

 

[sebastiantombs]

The version of Visual Studio should be right in the instructions for installing DS on the DS forum site.

DeepStack Installation

Using DeepStack with Windows 10 (CPU and GPU)

docs.deepstack.cc

 

[Nunofya]

I can't get this DS night thing to work properly. Day time I get 100-500ms on detection times. Nighttime runs 4000's or higher while missing most detections. Run the lates DS, that's actually better. Was getting 5-10K times on the old DS.

Bought all new parts and nephew put this system together for me just to run BI:
This is what I got:

i7-12700k,

ASUS Prime Z690-A LGA 1700 motherboard

MSI MAG CoreLiquid P240-AIO CPU cooler,

16GB Crucial DDR5 RAM ( 2, 8GB sticks),

Samsung 980 Pro 500GB PCIe NVMe Gen4 SSD M.2, Thermalright M.2 Heatsink cooler,

Samsung 980 Pro 1TB PCIe 4.0 NVMe Gen4 SSD M.2, Sabrent M.2 2280 Heatsink(copper),

Segotep 750w fully modular gaming power supply 80 plus Gold Certified PSU w/silent 140mm fan

Windows 10 pro 64 bit

YuanLey 18 port PoE switch, 16 Poe+

Fractal Design Meshify Black ATX flexable dark tinted tempered glass window mid tower computer case

 

[kferrero]

Not a solution, but perhaps a benchmark…
For over 90% of my triggers, I’m routinely seeing a 100-150 ms processing time for this hardware & configuration:

• I7-4770 processor
• PNY NVIDIA Quadro P400 V2
• 16 GB RAM
• 500 GB SSD
• 8 TB Purple hard drive

My DeepStack settings are:

• version 2021.09.01
• 1 instance
• ’High’ mode
• EDIT (3/21/22): ‘Medium’ mode yields a 70-110 ms processing time.

My camera settings are:

• using only the ‘Combined’ custom model, day & night
• 5 extra images each 500/750ms (500ms for street facing cams)
• ‘Use main stream…’ disabled
• ’Use motion-leading image’ enabled only for street-facing cams

My Blue Iris version is current (5.5.5.12).

More details here…

DeepStack Case Study: Performance from CPU to GPU version

I'm posting this thread for anyone considering making the transition from the DeepStack CPU version to the GPU version. Hopefully this will give you an impression of what you can expect in terms of enhanced performance. Please note some of my observations are system-specific. My system specs...

ipcamtalk.com

Following this thread, longtime lurker that just made an account. I just switched from Deepstack CPU to GPU and appear to not be getting that noticeable of an improvement. I'm getting roughly 300ms processing time with a Quadro T400 which is better than this P400 and yet my processing times are nearly 3x yours.

Hardware Specs

• i7-6700
• 16gb RAM
• PNY nVidia Quadro T400
• 256gb SSD for OS and BlueIris
• 4TB WD Gold for recordings

My DeepStack settings are:

• version 2021.09.01
• CUDA 11.6.2
• cudaNN 8.4.0.27
• 1 instance
• ’Medium’ mode
• Mike's custom combined model
• Default object detection disabled
• Motion leading image enabled
• 5 additional images 500ms delay
• "Use Main Stream" disabled
• 280-350ms average processing time.

I've confirmed with performance monitor and task manager that the GPU is being used, there is a very clear GPU and CUDA usage spike exactly when motion happens and images are sent to the GPU for processing. Also nVidia desktop tool can show you which windows processes are running on the GPU and in my case only python.exe is running which is deepstack. Everything I can see confirms the GPU is being used, but processing times don't appear to be as good as they should be and I don't know why.

I feel like it would be beneficial to the community to have some sort of sticky thread or spreadsheet or something where as a community we can offer benchmarks of performance for Deepstack with GPU by video card so that people can more or less see where they stand and if they are getting the performance out of their card that others are, or if they may have something misconfigured. I've struggled scouring these forums, and google to find what type of processing time performance other people are getting, out of which specific video cards to see how my T400 should be working.

 

[jaydeel]

Everything I can see confirms the GPU is being used, but processing times don't appear to be as good as they should be and I don't know why.

First, let’s make sure we’re comparing the same data. The range I quoted was for MOST of the DeepStack events, not ALL events.
I filter out the long tail events. I think this is a better way to compare the effects of settings changes, hardware, etc. This is illustrated as follows.

Screenshot 1. ALL events for the past 10+ days.
Total events: 1747
Avg processing time: 270 msec


Screenshot 2. Histograms
These charts show the exact same data in screenshot 1. They nicely illustrate the long tail events. Note that the 2nd histogram has a log scale on the vertical axis - this better accentuates the tail events.



Screenshot 3. This last chart drills in on the proponderance of events in the leftmost bar on the above histogram.

Total events: 1604 … ~92% of all events (1604/1747)
Avg processing time: 94 msec

Note: I’m now using ‘Medium’ mode. I was seeing an average of ~125 msec in ‘High’ mode.

 

[jaydeel]

BTW, I am almost done developing a browser-based Javascript charting tool that anyone could use to create Google Charts like those shown in my previous post. It extracts data from Blue Iris logfiles or imports data from any CSV file. Now I just need to learn how to use GitHub to share it.

 

[kferrero]

BTW, I am almost done developing a browser-based Javascript charting tool that anyone could use to create Google Charts like those shown in my previous post. It extracts data from Blue Iris logfiles or imports data from any CSV file. Now I just need to learn how to use GitHub to share it.

That would be cool, because it's really frustrating me to hear your average processing time even on HIGH 2-3x better than mine on MEDIUM.

"Note: I’m now using ‘Medium’ mode. I was seeing an average of ~125 msec in ‘High’ mode."

It just tells me there's something not optimized on my end, either I'm not using the right version of CUDA or cudaNN or video driver or something. The 2022 version of DeepStack for example does not work for me at all, I had to roll back to the last 2021 version.

What version of CUDA and cudaNN are you using?

 

[jaydeel]

What version of CUDA and cudaNN are you using?

I had to google how to find the versions...

CUDA...
nvcc --version


cudaNN...
Navigate to: "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include"
Open cudnn_version.h



EDIT: Filenames of the installers I used

• cuda_10.1.105_418.96_win10
• cudnn-10.2-windows10-x64-v8.2.2.26.zip

 

[kferrero]

I had to google how to find the versions...

CUDA...
nvcc --version
View attachment 124703

cudaNN...
Navigate to: "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include"
Open cudnn_version.h

View attachment 124704

EDIT: Filenames of installers I used

• cuda_10.1.105_418.96_win10
• cudnn-10.2-windows10-x64-v8.2.2.26.zip

Update, thanks for the info! I uninstalled CUDA and cudaNN that I was running and matched your versions, that gave me an immediate, repeatable, measurable improvement of 100ms. I'm now averaging 180-200ms processing time. I'm going to further test by updating the graphics card driver to the latest driver available on nvidia's website to see if that offers any additional performance improvements too.

Crazy how wildly varied performance is with different CUDA and cudaNN versions alone and how little information I can find on why.


EDIT: Bonkers, so CUDA 10.2p2 with cudaNN 8.2.2 and the latest nVidia drivers now has dropped processing time to 80ms on MEDIUM.

Any stragglers that find this post at a later time, PNY Quadro T400 has optimal performance with CUDA 10.2patch2, cudaNN 8.2.2 and latest nVidia driver 512.15. CUDA 11 is worse from a performance perspective, and Deepstack 2022 does not work at all, stay with the last 2021 version.