Dahua IPC-HDW5231R-ZE's Going Off-Line

[Dramus]

For a couple months or more I've had a problem with my Dahua IPC-HDW5231R-ZE cams going briefly off-line for a minute to a minute-and-a-half on an irregular, but fairly frequent basis. Having eliminated the NAS and Surveillance Station software running upon it as the culprit, I assumed it was the other common denominator: The PoE network switch that powered them. So I replaced it with an EnGenius EWS2910P. (Nice switch, btw.)

Nope. That wasn't it. One camera has since gone offline and come back twice, and another once, in the last ten hours

Only now I can better see what's happening. From the new switch's log:

Oct  9 09:47:29 192.168.0.10 %Port-5: GigabitEthernet3 link down
Oct  9 09:47:33 192.168.0.10 %Port-5: GigabitEthernet3 link up
Oct  9 09:47:47 192.168.0.10 %Port-5: GigabitEthernet3 link down
Oct  9 09:47:49 192.168.0.10 %Port-5: GigabitEthernet3 link up
Oct  9 09:47:50 192.168.0.10 %Port-5: GigabitEthernet3 link down
Oct  9 09:48:03 192.168.0.10 %Port-5: GigabitEthernet3 link up

I'm not 100% certain, but I believe it's only the 5231's that are doing this. I also have an IPC-HDB4231C-AS and a re-branded IPC-HDW2231RP-ZS-2713 that I do not recall having done it.

The firmware on the 5231's is: V2.800.0000005.0.R, Build Date: 2019-03-25. I haven't updated to V2.800.0000016.0.R.200605, yet, because I saw no compelling reason. Maybe now there is one?

Funny thing about this is these cameras have all been running flawlessly since installation. The only recent change in the cameras is I started using IVS.

 

[Dramus]

I guess nobody's got any ideas, so I guess I'll just update one of the guilty parties to 16, and see what happens. Wish me luck.

 

[sebastiantombs]

It's kind of an open ended question. You should check the switch or wall wart supplying power to them. Next up is the cable. Take one down, use a known, good short, cable and plug it in to a switch port that hasn't been used and see if it drops out. I don't think updating firmware will solve the problem, but one never knows.

 

[Dramus]

Thanks for the followup.

Wasn't looking for troubleshooting help, but more wondering if anybody else had experienced anything like this.

The network switch has been replaced with an entirely different model from an entirely different manufacturer--power supply and all. In the process, of course all the ports are different than they were before. The "short cable" test is impractical, without taking a camera down and out of service for the length of the test, which, given the unpredictability of this happening, could be days or weeks.

The new switch even has built-in cable diagnostics, and claims all three cables are fine. (Never mind the fact I tested them at GigE speeds at the time of installation.) So I think it unlikely it's the cables. Plus I believe it's happened only with the 5231's, not with the other two Dahua cameras. Were it all five cameras I might suspect the cabling, but it seems to me unlikely that, of the five cable runs I did, only the three going to 5231's flaked-out. (And, no, they're not the longest or shortest, either. Two of them are the longest, the third the shortest--24 and 10 meters, respectively.)

Lastly: The symptoms are unusual, to say the least. Every time it's a set of three link down/up events.

No way for you to have known: But I have somewhere in the vicinity of fifty years of working with electronics. Everything from vacuum tubes through modern tech of all sorts, having retired as an IT professional three years ago, after twenty-five years in that role. So I'm somewhat familiar with troubleshooting methodology

I'll update the firmware on one of them soon, the one I believe is doing it most often, and report back.

 

[looney2ns]

But more times than not, issue's like this turn out to be a cable or water intrusion in the RJ45, due to improper waterproofing or the use of CCA cable.

 

[Dramus]

Thanks for the follow-up, @looney2ns.

Water intrusion is so unlikely in this case that I'm confident that is surely not the case. All three are mounted directly under deep soffits, well back from the edge. New roof three summers ago, with high-tech Mylar "felt", plus snow & ice shield going up three feet from the wall lines (which would be 5-6 feet from the soffit edges), and the camera bases were sealed with clear silicone caulk (mainly to keep wasps out).

Plus the communications disruption pattern seems to me too regular to suggest intermittent connectivity for mechanical reasons. Every single time it's three drops lasting a couple to a dozen seconds. And when there's a longer drop, it's always the third of the three. Never the first or second.

But, if the firmware update doesn't do the trick, I guess beginning to physically tear stuff apart will be next. Oh yay.

 

[sebastiantombs]

I would still suggest testing one of those cams by taking it down and connecting with a known good cable. I, too, have a long history in IT, dating back to the VIC20 and thin net/thick net. Mr. Murphy is always a partner in every technical endeavor. Don't discount the most likely causes, they have a tendency to bite you in the a$$ in the end and Mr. Murphy laughs every time.

 

[Dramus]

BTDT, @sebastiantombs What I'd more likely do is swap a camera that isn't exhibiting the problem with one that is. They are physically identical, it's just that the re-labeled Dahua doesn't have sound and a couple other features (I don't use) that the 5231's do have.

 

[Dramus]

sigh So the firmware upgrade went w/o issue, but the verkakte thing won't reload the configuration backup I saved, even after a second, manual reboot. So now I have to re-configure everything from scratch

This is special: It restored part of the configuration, but not all of it

In fact: Looks like everything except the Smart Plan and IVS configs. So, on the other cams, I guess it will behoove me to screen shot my IVS configs before doing this.

 

[looney2ns]

Thanks for the follow-up, @looney2ns.

Water intrusion is so unlikely in this case that I'm confident that is surely not the case. All three are mounted directly under deep soffits, well back from the edge. New roof three summers ago, with high-tech Mylar "felt", plus snow & ice shield going up three feet from the wall lines (which would be 5-6 feet from the soffit edges), and the camera bases were sealed with clear silicone caulk (mainly to keep wasps out).

Plus the communications disruption pattern seems to me too regular to suggest intermittent connectivity for mechanical reasons. Every single time it's three drops lasting a couple to a dozen seconds. And when there's a longer drop, it's always the third of the three. Never the first or second.

But, if the firmware update doesn't do the trick, I guess beginning to physically tear stuff apart will be next. Oh yay.

As you probably know, it doesn't actually have to be water intrusion, it can simply be from moisture due to condensation from temp changes.

 

[looney2ns]

sigh So the firmware upgrade went w/o issue, but the verkakte thing won't reload the configuration backup I saved, even after a second, manual reboot. So now I have to re-configure everything from scratch

This is special: It restored part of the configuration, but not all of it

In fact: Looks like everything except the Smart Plan and IVS configs. So, on the other cams, I guess it will behoove me to screen shot my IVS configs before doing this.

Best to always do a factory default after firmware upgrade, then do the setup manually, don't import any settings file.

 

[Dramus]

Yes, I know, @looney2ns.

I'm hopeful it was, as I suspected, a firmware bug. As I noted in my OP: The only recent change was beginning to use IVS. Coincidentally, it was only the IVS config that failed to restore when I reloaded the config after the update. If this update stops the problem on that one cam, the next thing I'll try is deleting the IVS config on one of the other two, to see if it stops on that one, too.

Best to always do a factory default after firmware upgrade, then do the setup manually, don't import any settings file.

I'm aware that's the common wisdom, here-abouts, but I've always restored the previous configs w/o issue.

 

[Dramus]

Well, so far it's looking like my suspicion, that's it's a firmware bug, may have been correct. The camera I upgraded last Sunday has not gone off-line since, and it had been the camera that had been going off-line most frequently. Not certain, yet, because there were times of up to a week or so that no cameras would go off-line. And I don't recall this problem manifesting itself immediately after configuring tripwires or intrusion zones, otherwise I think I would have made the connection more quickly. (And probably not gone out and spent $125 on a new switch.)

Meanwhile, the re-labelled 2231 I have, which had not been one of the cameras that had been going off-line, has now done so once. I configured a tripwire into it about the time I upgraded the firmware on the one 5231.

ETA: A couple days later I added a tripwire to another of the 5231's. It has now spontaneously rebooted itself once. I don't recall that one having done so before. The experiment continues.