5.4.9 - June 25, 2021

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,430
Reaction score
47,552
Location
USA
Man I was hoping upgrading cameras would fix this, but I'm still having an issue where my leading image of every DS analysis is at T-4500ms or so. And then there is no T-0, so the second image is my T+500ms. Could a setting cause this, or did is this a bug? Nobody else is seeing this? I even tried reinstalling DS, but I fear it's a bug where maybe I would have to reinstall BI.
What camera and what are your FPS and iframes?

They should match and your KEY in the BI camera tab should be 1.00. If they are like 0.10 or 0.25 that is probably why.
 

stev067

Getting the hang of it
Joined
Jul 9, 2021
Messages
34
Reaction score
36
Location
Kalamazoo
re you running the CPU version and if so what processor?
Yep CPU. I have a 5950x which is 16 cores, but I have only 4 cores dedicated to this VM I have BI running on. I've had it that way since before this started happening a couple updates ago, and my CPU utilization hangs around 20% so shouldn't that mean it's sufficient? I can add a few cores to double check.
Edit: Didn't see the second part right away. Yes the FPS are both 15 and the iframes are both 1.0
Edit2: Camera is IPC-T5442T-ZE
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
I think it's more processing time related than anything. I running bare metal on a 6700K and saw processing times getting long and detections dropping off. That made me switch to the GPU version. Are you running any additional models, like dark or logo? They add significant time, too, especially logo since it's so large.
 

stev067

Getting the hang of it
Joined
Jul 9, 2021
Messages
34
Reaction score
36
Location
Kalamazoo
I think it's more processing time related than anything. I running bare metal on a 6700K and saw processing times getting long and detections dropping off. That made me switch to the GPU version. Are you running any additional models, like dark or logo? They add significant time, too, especially logo since it's so large.
Ok stand by...I'm going to fire it up with 16 cores.
 

stev067

Getting the hang of it
Joined
Jul 9, 2021
Messages
34
Reaction score
36
Location
Kalamazoo
Even with all the extra CPU, with custom models and facial detection turned off, BI running on low, still same issue of super early T- frame and no T-0 frame.
Edit: And I have 4GB RAM allocated to this VM. If that might not be enough, I have total 64GB so I can allocate more.
Edit2: Reverting to 5.4.9.10 fixes the issue, which makes sense because it started at 5.4.9.11. Surprised nobody else has seen it. I'll open a ticket.
 
Last edited:

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
If you've got an NVidia, CUDA capable, video card hanging around give the GPU version a try. The added power load is under 20 watts running DS detection and, to me, well worth the extra power consumption.
 

beepsilver

Getting comfortable
Joined
Mar 9, 2014
Messages
863
Reaction score
982
Location
Nebraska
Even with all the extra CPU, with custom models and facial detection turned off, BI running on low, still same issue of super early T- frame and no T-0 frame.
Edit: And I have 4GB RAM allocated to this VM. If that might not be enough, I have total 64GB so I can allocate more.
Edit2: Reverting to 5.4.9.10 fixes the issue, which makes sense because it started at 5.4.9.11. Surprised nobody else has seen it. I'll open a ticket.
On .13 using CPU version. I'm seeing the same wrt to timing of DS analysis frames...changing leading image, pretrigger buffer, make time, DS analysis intervals, etc., affects DS analysis timing somewhat, but difficult to make sense of it enough to manage it properly. Also, there are some weird T+/- xxxxmsec times in the analysis. Sometimes the indicated T-time in the first image is later than the second image. And changing pretrigger buffer to 0 gave me T+745762154 msec [958]* in the first image and the second image (final) at T+750 msec nothing was found even though the object (me) was still in motion but had barely moved. That was with a make time of 1 sec, Begin analysis with leading image checked and 5 real time images at 750 ms. The silver lining is that even though I can't make heads or tails of this, detection rates are nearly flawless. I don't have an NVidia card to play with, but I suspect my results might be a bit different using the GPU version of DS.
 

stev067

Getting the hang of it
Joined
Jul 9, 2021
Messages
34
Reaction score
36
Location
Kalamazoo
On .13 using CPU version. I'm seeing the same wrt to timing of DS analysis frames...changing leading image, pretrigger buffer, make time, DS analysis intervals, etc., affects DS analysis timing somewhat, but difficult to make sense of it enough to manage it properly. Also, there are some weird T+/- xxxxmsec times in the analysis. Sometimes the indicated T-time in the first image is later than the second image. And changing pretrigger buffer to 0 gave me T+745762154 msec [958]* in the first image and the second image (final) at T+750 msec nothing was found even though the object (me) was still in motion but had barely moved. That was with a make time of 1 sec, Begin analysis with leading image checked and 5 real time images at 750 ms. The silver lining is that even though I can't make heads or tails of this, detection rates are nearly flawless. I don't have an NVidia card to play with, but I suspect my results might be a bit different using the GPU version of DS.
This is exactly what I have been seeing. I opened a ticket to Ken a little while ago. If you revert to .10, the issue will go away.
 

beepsilver

Getting comfortable
Joined
Mar 9, 2014
Messages
863
Reaction score
982
Location
Nebraska
So the default 15 seconds is likely more than enough time for most setups. If I understand this correctly, if 5 additional images are added to the analysis requirement (after the two automatic images) at 1 second intervals, a total of roughly 7 seconds should be needed to complete the analysis.
 

MnM

Young grasshopper
Joined
May 14, 2014
Messages
95
Reaction score
20
I have been using the AI Tool almost from the beginning when GentlePumpkin released it.
Now I am running the latest version by Vorlon. Using it for person detection only.

Running BI on a virtual Win10 PC (enough vCPUs assigned that usage never spikes beyond 40% and 12GB of RAM) and Deepstack as a docker on a separate virtual Ubuntu with GPU.
It works really well for me and in 98% of cases I get a detection push notification on my mobile within 1s of people arriving into the cameras' view.

I thought I will give the new DS integration a try and see how it goes. I expected good performance due to DS being directly integrated into BI and both running on the same virtual PC.
So I have installed the CPU version of DS on the same PC as BI and cloned a camera, adjusted the setting and tested it. Performance was good - under 50% spikes when detecting, around 25% normal usage.

However, no matter what options I try I always run into 2 issues ( I am using day time examples here and not night).

First - it just fails to detect. I have about 4 ppl moving for about 40 seconds in front of the camera. AI Tool works as usual and I got push notification with 1s since the 1st person arrive in the camera view.
40s seconds later I am still waiting for the BI DS to trigger something. It never did. This kept happening very often.

Second is the push notification - when detection actually detects persons (very rare!) sometimes I have to wait up to 10s or more to get that push notification on my mobile.
Strangely sometimes I get push notification from BI DS before the AI Tool. But this is very rare. Maybe 1 in every 50 times. The rest of the 49 times there is no person detection at all from BI DS and in very rare occasion I have a very late push notification from BI DS.

I have tried to use the Ubuntu docker DS in the BI settings. Similar experience as above.

Not sure if I am doing something wrong but for me is just not working well. Anyone else experience the same issues as me?
I would love to have DS and BI running on a single server.
 
Last edited:

stev067

Getting the hang of it
Joined
Jul 9, 2021
Messages
34
Reaction score
36
Location
Kalamazoo
I have been using the AI Tool almost from the beginning when GentlePumpkin released it.
Now I am running the latest version by Vorlon. Using it for person detection only.

Running BI on a virtual Win10 PC (enough vCPUs assigned that usage never spikes beyond 40% and 12GB of RAM) and Deepstack as a docker on a separate virtual Ubuntu with GPU.
It works really well for me and in 98% of cases I get a detection push notification on my mobile within 1s of people arriving into the cameras' view.

I thought I will give the new DS integration a try and see how it goes. I expected good performance due to DS being directly integrated into BI and both running on the same virtual PC.
So I have installed the CPU version of DS on the same PC as BI and cloned a camera, adjusted the setting and tested it. Performance was good - under 50% spikes when detecting, around 25% normal usage.

However, no matter what options I try I always run into 2 issues ( I am using day time examples here and not night).

First - it just fails to detect. I have about 4 ppl moving for about 40 seconds in front of the camera. AI Tool works as usual and I got push notification with 1s since the 1st person arrive in the camera view.
40s seconds later I am still waiting for the BI DS to trigger something. It never did. This kept happening very often.

Second is the push notification - when detection actually detects persons (very rare!) sometimes I have to wait up to 10s or more to get that push notification on my mobile.
Strangely sometimes I get push notification from BI DS before the AI Tool. But this is very rare. Maybe 1 in every 50 times. The rest of the 49 times there is no person detection at all from BI DS and in very rare occasion I have a very late push notification from BI DS.

I have tried to use the Ubuntu docker DS in the BI settings. Similar experience as above.

Not sure if I am doing something wrong but for me is just not working well. Anyone else experience the same issues as me?
I would love to have DS and BI running on a single server.
I guess I was having kind of the opposite experience as you. Running BI and DS (cpu) in a dedicated W10 VM, everything is working as expected. I tried switching the DS server to instead run as a docker on my Unraid machine which runs this VM. I changed the BI AI settings to interface with this new DS, but I was finding that the processing times were like 10x as high, taking multiple seconds and sometimes hitting 15s and timing out. So I went back to the way things were and didn't get any responses from others who run DS as a docker container, so I haven't revisited it yet.
Anyway, do you think there could be a conflict? Sounded kinda like you were interfacing BI with 2 DS instances simultaneously. Wish I had anything more valuable to add other than I'm running BI and DS in a VM without issues. Regarding your 'failure to detect' issue, are you getting triggers? I've frequently used the saved DS analysis files saved in the Alerts folder to watch triggers that were or weren't confirmed. It's an important distinction to know whether you had a trigger without a confirmed alert, vs no trigger at all.
 

MnM

Young grasshopper
Joined
May 14, 2014
Messages
95
Reaction score
20
I guess I was having kind of the opposite experience as you. Running BI and DS (cpu) in a dedicated W10 VM, everything is working as expected. I tried switching the DS server to instead run as a docker on my Unraid machine which runs this VM. I changed the BI AI settings to interface with this new DS, but I was finding that the processing times were like 10x as high, taking multiple seconds and sometimes hitting 15s and timing out. So I went back to the way things were and didn't get any responses from others who run DS as a docker container, so I haven't revisited it yet.
Anyway, do you think there could be a conflict? Sounded kinda like you were interfacing BI with 2 DS instances simultaneously. Wish I had anything more valuable to add other than I'm running BI and DS in a VM without issues. Regarding your 'failure to detect' issue, are you getting triggers? I've frequently used the saved DS analysis files saved in the Alerts folder to watch triggers that were or weren't confirmed. It's an important distinction to know whether you had a trigger without a confirmed alert, vs no trigger at all.
Thanks - that is really interesting you having the opposite!
In regards to DS - I was only running one cloned camera through the integrated BI and DS. All other cameras were running through Ubuntu DS and AITool. I will try and disable Ubuntu DS and see.
In regards to triggering - it was all over the place. I had triggers that DS said didnt find persons (and at that time I had 4 ppl in the camera view for about 40s). I also had triggers but no alerting at all.

I am gonna take a snapshot and then configure all cameras to use only the integrated DS and see.
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
I installed it but haven't checked to see what's new, busy day. I will say it's behaving like .14 in terms of memory usage so far. Memory went from ~2.15GB to over 4GB in about two hours in both. In .14 it dropped back to ~3GB overnight. I'll see what happens tomorrow and post how it's behaving as well as look to see what's new, if anything.
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
Based on how low memory utilization was with .10 I'd say yes. It did spike initially but dropped back and stayed pretty constant over a day and a half. .14 spiked and dropped back, but nowhere nearly as far. I'm not too worried about memory but do like to keep an eye on it, 32GB in the machine.
 
Top