Challenging Network Issue - Cameras going offline randomly

[plomaris]

I have 12 IP cameras and have had no issues for over 5 years with my current network configuration. In the last two weeks, several of my cameras will go offline briefly (for several seconds, as shown by "NO SIGNAL" in Blue Iris) at random intervals. Sometimes it will be 9 cameras, other times less. I do have an unusual network configuration (see attached diagram), but I have had no major issues for several years. I use a mix of POE cameras connected via ethernet, some via POE/Powerline adapters, and some connected to Google Wifi nodes. How would I go about diagnosing where the problem is? Before I start spending $$$ replacing the admittedly crappy Google Wifi or buying a new POE Switch, I want to figure out where the actual problem is.

It seems that the cameras that are impacted (shown in red text in the diagram) all flow through my basement POE switch, either via a Logitech POE/Powerline adapter or direct POE connection. Cameras connected via Google Wifi that connect wirelessly to my office main Google Wifi puck don't seem to be impacted. So, I'm thinking it is somehow related to either my POE Switch or the CAT6 that goes from the basement to the office. Since the problem occurs randomly (several hours apart, mostly), it is very difficult to diagnose. I thought perhaps the Google Wifi, which has a lot of issues with the latest firmware, or the TP-Link POE switch is going bad. But I'd prefer to know what the issue is prior to just buying new equipment.

Note that I haven't see any issues with my home Desktop PC, nor the 2 work laptops for my wife and I that are directly connected to the Office Ethernet switch. I also haven't seen any issues with the myriad of other devices on my network, such as wifi switches, Echos, phones, tablets, etc.

Any ideas on how I can determine where the problem is?

Thanks.

Tom

 

[hikky_b]

Hi Tom,

Have you checked any logs on that TP-Link switch in the basement? Might help you narrow down the issue.

Only time I've experienced similar is where the switches power supply begins to fail. Off hand that switch uses an external power supply so you could try another power supply and see if it solves the cameras dropping.

 

[plomaris]

Well, I looked but could not find any logs in either the web interface or the Easy Smart App for the TL-SG108PE switch. I may end up purchasing a new POE switch anyway, as this is an old V2 switch and doesn't have all the features of the latest V5 switch, such as self healing auto restart of inactive POE ports.

I'll see if I have a replacement power supply to see if that is the issue.

Thanks for the tip.

 

[TonyR]

Could moisture in the basement be affecting the connections on the POE switch down there?
Or thermal cycling (heating up/cooling down) be causing mechanical issues with the same connections?

An inspection of all male/female RJ-45's involved and/or temp substitution with some known "good" patch cables might tell a productive tale...

 

[looney2ns]

I would try spraying a little Deoxit on each connector.

Search results for "deoxit%20d5"

 

[plomaris]

TonyR and looney2ns - I didn't think of that, so I'll look into it. It's just odd that is it not always the same cameras. The basement is normally not too hot or cold, but I guess the temperature does vary. Humidity doesn't seem to be an issue either. Thanks for the suggestion.

 

[Swampledge]

It’s a long shot, but make sure there’s only one DHCP server on your network, and that your cameras have static IP addresses outside the DHCP pool. This sounds like a problem I had when my son moved in with us while building his new house, and he simply plugged his Google mesh system into my LAN.

 

[plomaris]

Swampledge - Yes, I only have one DCHP server on my network and each camera has a static IP. Nothing has changed in my network in years, so I'm leaning more towards an equipment issue or other external factor.

 

[kd5mdk]

The only way to identify common causes is to be systematic in documenting the issue when it happens.
I'd suggest labelling all of the cameras on your diagram with the IP addresses of them. (Could be only the last part to save space if you want so 192.168.0.122 becomes "122").
When any camera go offline, look at your diagram and point to all of the affected cameras. That will make it easier to see what types of connection they have in common.

Without any other information, the switch becoming flaky seems like a reasonable potential cause. It might also be something like electrical interference from a new load like an appliance or other source interfering with either that Cat 6 cable or (more likely) the Logitech powerline equipment. Maybe your neighbor became a ham radio operator.
But documenting exactly which cameras are impacted and how those specific cameras are connected should give you a lot more to go on than you had before.

 

[plomaris]

kd5mdk - I recently replaced the POE switch in the basement. The problem still occurs, but it does not "seem" to be as frequent. From what I've read, the latest Google Wifi firmware is garbage and Google doesn't care. The other day, both my laptop and my wife's went offline for about a minute and we are both directly connected via Ethernet to the Google Wifi Hub. Others have seen this type of behavior with both wifi and wired connections with Google Wifi. So, I'm thinking an upgrade to my mesh system is in the near future.