DeepStack Case Study: Performance from CPU to GPU version

[AP-123]

Just one

 

[wittaj]

I run DeepStack on several cameras and run the custom model and default model and only use the 1 instances and it works just fine.

I guess I am trying to understand the benefit of the instances? Or is this something only for people that have Deepstack running on multiple computers all feeding to one?

 

[sebastiantombs]

I use two instances but am running about 12 cameras with DS. Some of those can trigger at the same time, basically, and the second instance makes detection time a lot faster, apparently, when that occurs.

 

[wittaj]

I use two instances but am running about 12 cameras with DS. Some of those can trigger at the same time, basically, and the second instance makes detection time a lot faster, apparently, when that occurs.

What was that you said about learning something new with BI everyday LOL.

Dang I guess I will give this a try and see if the cams that can trigger around the same time have improved timing with two instances.

 

[105437]

What was that you said about learning something new with BI everyday LOL.

Dang I guess I will give this a try and see if the cams that can trigger around the same time have improved timing with two instances.

@wittaj Please report back on your findings. I have a total of 6 cameras that trigger when someone comes up our driveway to our garage. I tried putting 3 cameras on one instance and the 3 on another and didn't see any appreciable gains.

 

[sebastiantombs]

@wittaj Don't forget to assign the instance for the cameras involved in the AI section of the camera trigger settings.

I think speed differences are also a function of what card is in use.

 

[105437]

Here's what it looks like for me coming up my driveway earlier with one instance of DS.

12/6/2021 12:40:54.844 PM Street_Cam MOTION_A
12/6/2021 12:41:01.097 PM Entry_Cam MOTION_A
12/6/2021 12:41:01.179 PM Street_Cam DeepStack: car:93% [183,193 373,351] 125ms
12/6/2021 12:41:05.181 PM Front_Cam MOTION_A
12/6/2021 12:41:05.555 PM Front_Porch_Cam MOTION_A
12/6/2021 12:41:06.864 PM Driveway_Cam MOTION_A
12/6/2021 12:41:07.600 PM Entry_Cam DeepStack: car:93% [426,202 856,475] 217ms
12/6/2021 12:41:08.927 PM Garage_Entry_Cam MOTION_A
12/6/2021 12:41:11.524 PM Front_Cam DeepStack: car:93% [52,47 239,126] 354ms
12/6/2021 12:41:11.925 PM Front_Porch_Cam DeepStack: car:90% [500,11 618,65] 264ms
12/6/2021 12:41:12.998 PM Garage_Cam MOTION_A
12/6/2021 12:41:13.216 PM Driveway_Cam DeepStack: car:93% [351,57 559,167] 168ms

 

[sebastiantombs]

Just a snippet -

0 12/6/2021 2:46:47.566 PM CE DeepStack: person:43% [382,23 400,66] 119ms
3 12/6/2021 2:46:49.399 PM FN_C MOTION_A
0 12/6/2021 2:46:49.755 PM FN_C DeepStack: person:41% [267,78 311,121] 102ms
3 12/6/2021 2:46:50.087 PM CW MOTION_A
0 12/6/2021 2:46:55.480 PM CW DeepStack: Alert cancelled [nothing found] 53ms
3 12/6/2021 2:47:10.800 PM CE MOTION_A
0 12/6/2021 2:47:11.189 PM CE DeepStack: car:86% [393,26 451,63] 101ms
3 12/6/2021 2:47:13.107 PM FN_C MOTION_A
0 12/6/2021 2:47:13.354 PM FN_C DeepStack: car:83% [913,84 1004,123] 60ms
3 12/6/2021 2:47:14.644 PM CW MOTION_A
0 12/6/2021 2:47:14.874 PM CW DeepStack: car:88% [686,170 860,250] 60ms

 

[105437]

Just a snippet -

0 12/6/2021 2:46:47.566 PM CE DeepStack: person:43% [382,23 400,66] 119ms
3 12/6/2021 2:46:49.399 PM FN_C MOTION_A
0 12/6/2021 2:46:49.755 PM FN_C DeepStack: person:41% [267,78 311,121] 102ms
3 12/6/2021 2:46:50.087 PM CW MOTION_A
0 12/6/2021 2:46:55.480 PM CW DeepStack: Alert cancelled [nothing found] 53ms
3 12/6/2021 2:47:10.800 PM CE MOTION_A
0 12/6/2021 2:47:11.189 PM CE DeepStack: car:86% [393,26 451,63] 101ms
3 12/6/2021 2:47:13.107 PM FN_C MOTION_A
0 12/6/2021 2:47:13.354 PM FN_C DeepStack: car:83% [913,84 1004,123] 60ms
3 12/6/2021 2:47:14.644 PM CW MOTION_A
0 12/6/2021 2:47:14.874 PM CW DeepStack: car:88% [686,170 860,250] 60ms

Good stuff, my activity represents a roughly 19 second time frame and yours around 27. That would put a load on things I suspect.

 

[wittaj]

First attempt didn't look good - CPU spiked big time.

Gonna reboot and let everything settle down and see if it is an improvement.

Like most things, I suspect as @sebastiantombs mentions, the hardware you have makes a big difference on if you see the gains.

 

[MikeLud1]

I will do some testing later tonight, triggering 5 cameras with one instance and then test with five instances

 

[wittaj]

OK, I thought the night testing would be the best option since I don't get shade triggers and other stuff.

Running two instances spiked my CPU (even though I am using the GPU version). Make times appeared to be lower, but not enough that I would justify the spike in CPU. About 40% bump over one instance.

 

[sebastiantombs]

I don't see much of a spike, maybe one or two percent. What CPU are you running?

 

[wittaj]

A 4th gen.

 

[MikeLud1]

I did some testing, test one running 5 instances of DeepStack set at high and triggering 5 cameras at the same time. Each camera is set to 10 1080P images at 500ms and is using its own instance of DeepStack. One note I am using a RTX 3060 GPU for DeepStack.

Test two running one instances of DeepStack set at high and triggering 5 cameras at the same time. Each camera is set to 10 1080P images at 500ms and all cameras are using the same instance of DeepStack.

Test three running one instances of DeepStack set at high and triggering one camera and waiting for DeepStack to finish before the next trigger. Each camera is set to 10 1080P images at 500ms and all cameras are using the same instance of DeepStack.

Conclusion using multiple instances of DeepStack does not speed up analyze times

Test 1: Average DeepStack times for 5 cameras having its own instance of DeepStack is 146ms


Test 2: Average DeepStack times for 5 cameras having all the same instance of DeepStack is 158ms


Test 3: Average DeepStack times for triggering one camera and waiting for DeepStack to finish before the next trigger is 100ms

 

[wittaj]

@MikeLud1 - I think between mine and your unscientific tests, it appears that the instances are really intended for the individual that has multiple computers laying around and put Deepstack on each one to run for a camera?

What did you CPU and GPU do doing the multiple instances?

 

[MikeLud1]

The CPU not much change the CUDA load jump to about 12% from 7% idle for both test 1 and 2.

 

[MikeLud1]

I think what would benefit if you are using more than one DeepStack model for a camera and each DeepStack model had its own instance running parallel.

 

[sebastiantombs]

At night I run both the stock and dark models. During the day just the stock model. No face detection. hen I added an instance it actually cut the detection times, basically, in half. Daytime went from ~125-175ms to ~50-100ms. Night time dropped from ~300-400ms to ~125-300ms. Obviously checking through both models will take longer, but there was a significant time reduction.

0 12/6/2021 9:58:14.510 PM CW DeepStack: Car:76% [702,159 933,281] 202ms
3 12/6/2021 9:58:16.312 PM CE MOTION_A
3 12/6/2021 9:58:16.596 PM BW_FR MOTION_A
3 12/6/2021 9:58:16.703 PM LY_E MOTION_A
0 12/6/2021 9:58:16.806 PM CE DeepStack: Car:90% [0,33 134,143] 225ms
0 12/6/2021 9:58:16.806 PM CE DeepStack: car:83% [0,44 131,135] 225ms
0 12/6/2021 9:58:17.562 PM BW_FR DeepStack: person:53% [750,90 855,417] 52ms
0 12/6/2021 9:58:19.819 PM LY_E DeepStack: Alert cancelled [nothing found] 136ms
0 12/6/2021 9:58:21.020 PM FN_C DeepStack: Alert cancelled [nothing found] 114ms
3 12/6/2021 9:58:31.700 PM BE MOTION_A
0 12/6/2021 9:58:32.063 PM BE DeepStack: person:54% [786,189 835,274] 109ms

 

[105437]

At night I run both the stock and dark models. During the day just the stock model. No face detection. hen I added an instance it actually cut the detection times, basically, in half. Daytime went from ~125-175ms to ~50-100ms. Night time dropped from ~300-400ms to ~125-300ms. Obviously checking through both models will take longer, but there was a significant time reduction.

0 12/6/2021 9:58:14.510 PM CW DeepStack: Car:76% [702,159 933,281] 202ms
3 12/6/2021 9:58:16.312 PM CE MOTION_A
3 12/6/2021 9:58:16.596 PM BW_FR MOTION_A
3 12/6/2021 9:58:16.703 PM LY_E MOTION_A
0 12/6/2021 9:58:16.806 PM CE DeepStack: Car:90% [0,33 134,143] 225ms
0 12/6/2021 9:58:16.806 PM CE DeepStack: car:83% [0,44 131,135] 225ms
0 12/6/2021 9:58:17.562 PM BW_FR DeepStack: person:53% [750,90 855,417] 52ms
0 12/6/2021 9:58:19.819 PM LY_E DeepStack: Alert cancelled [nothing found] 136ms
0 12/6/2021 9:58:21.020 PM FN_C DeepStack: Alert cancelled [nothing found] 114ms
3 12/6/2021 9:58:31.700 PM BE MOTION_A
0 12/6/2021 9:58:32.063 PM BE DeepStack: person:54% [786,189 835,274] 109ms

@sebastiantombs Any particular reason why you run both at night and not just the dark model?