All, ok I promised I would update when I had finished testing against some theories I’ve been working on against new FW bugs I've come across so here we go. I’ve found a potential of 4 new bugs doing this testing and am detailing them below in no particular order.
This is a longer post but wanted to make sure you had the detail. I’ve already fed all of these details plus files, logs, my codebase change recommendation notes back to Dahua development engineers and notified
@EMPIRETECANDY for his awareness and to help continue to drive these with me as a priority.
Lets go:
1 - Settings reverting/adjusting when logging out of Web GUI
This affects exposure and other under the hood adjustments which culminates in you ending up with a very different (in some cases flatter) image than you just set and saved.
Affects: Primarily 7/09 FW, have not seen this issue on 6/11 or 6/24 yet. Only mitigation is to downgrade currently
2 - Features/Functions Affecting Performance
After thorough testing I can see that certain functions (such as ROI I mentioned before but others too) and intensive tasks start impacting the cameras overall performance. This can be seen in a number of ways in my testing over the past few days but one key one is to look at network (NW) latency. You’ll see a performance degradation in the cameras latency under certain situations. This can take camera response times even through simple ICMP (ping) requests from 1-2ms to in excess of 150+ms and in many cases sustained while the camera is under a performance hit. If you monitor traffic on a managed switch between ports (cam and NVR etc) you’ll also notice this when it happens. Reducing the tasks the camera is doing assists but in some cases requires a reboot and cleaning of old files out. After that, performance returns to normal until the next highly intensive task.
Therefore this points to resource handling of the camera and potentially a memory leak type of effect. The impact of this can be localized such as Live View / Play Back View and other Web GUI functions BUT also can impact the overall stream (local or remote) as I mention in Bug 3 below.
I want to be clear this doesn’t happen all the time and does appear to be a perfect storm type of issue but wanted to mention it as I was able to track it down.
Affects: I’ve seen this across all FW’s now I’ve been able to pinpoint the indicators. While this is somewhat addressed in 7/9 (peak network latency seen around 64ms vs 150ms+ on 6/11 and 6/24) due to the ROI bug I raised there, as I mentioned this still needs more work regardless of FW version to resolve the issue completely.
3 - Frames Per Second Anomalies
I’ve now been able to trap (took me a long while and a lot of eyestrain) a very strange anomaly with FPS. Specifically the camera (under certain circumstances) is not meeting the FPS you set for recording. While some of this can be seen as part of the performance hit (it certainly shows up more when the camera is under strain) it also comes down to the underlying codec and encoding algorithms function.
Now I have to stress to anyone that wants to recreate this, you ABSOLUTELY WON’T see this all the time. I have though been able to replicate and trap this a couple of times and I see in some cases just 1 frame dropped, i.e. 29 FPS on a 30 FPS record BUT weirdly I also see this impact the other way, wait for it………..31 FPS on a 30 FPS record. I’ve put this through an editor, thrown timecode and frame counters at it and even manually counted frames and can 100% state this is occurring.
I even troubleshot this across storage mediums (to rule out write speed / transfer issues) and can prove that this is impacting the stream itself rather than just local (SD) storage. I took the same clip captured to both a set of SD cards and a Dahua 5216 NVR and sure enough saw the same issue. Again to be clear, this appears very randomly and points to both resource constraint + codec encoding algorithm needing tweaking.
Affects: Again I’ve seen this across all FW but it is rare. Glad I was able to trap it.
4 - Auto Iris and Auto Gain Controls Don’t Disengage
This is another new and interesting one that I’ve finally been able to pinpoint. I noticed that when I was dialing in this cam through this testing of the new FW (past few days) there were times where a setting would almost rubber band and trigger Auto Iris and AGC to go crazy. Then even if you reversed your change (for example moving gamma to 48 from 50 but then back to 50) it would not affect the image.
In pulling apart the configs and looking at the codebase I can now see whats happening. Auto Iris and AGC have been tweaked to such a small tolerance (to preserve color at night) that if you breach those settings through decreasing too much light (not ‘real’ light but exposure, gamma etc) they will kick in and quick. This is not unique to Dahua at all BUT it is something I noted in 6/24 (more overly aggressive AGC & Auto Iris) and why you keep hearing me state adjustments should be from 6/11.
You’ll notice this with 2 key characteristics, 1) watch your sky at night go from close to black or inky blue to brighter blue that changes the color balance of the entire scene with more noise and 2) watch a hard surface like a driveway or sidewalk go soft, desaturated and appear overall washed out, lacking sharpness. These are all indicative (as I mentioned in my videos during this review) of too aggressive AGC + Auto Iris algorithms.
Luckily I’ve found a little workaround for this one. If you suddenly see your image get impacted this way, go to the Exposure tab, slide Exposure Compensation all the way down to something like 3 and then move back to (with 1 or 2 clicks of the slider bar) to your chosen (and correct for FOV) comp setting. You’ll see the sky (in the example above) go black or inky blue again, the image return to its correct look and this will stay until you adjust light impacting setting again on the camera that breaches the threshold.
Affects: All FW currently BUT you see it more in less well lit FOV’s as the AGC and Auto Iris will of course kick in more quickly with darker scenes. Using the illuminators, a bulb, additional light that many have on scene will mitigate this happening. I do want to stress again though, as I did in my review that these 2 controls (AGC and Auto Iris) are under the hood and in the codebase, not user facing.
Well, after that long post hopefully you’re still with me. If there is a silver lining to ALL of the issues above, its that these are primarily FW related (at least from how I see the cam reacting today and what I see in the codebase) therefore easy to fix by Dahua.
I’m pushing hard to get a new test FW ASAP with bug fixes for these and the other bugs I’ve raised and again am ensuring that all of these are built on 6/11 (which has way less aggressive AGC/Auto Iris for a start).
Will keep you updated.
HTH
i go to bed now...
