POE Cameras going offline everyday at the same time

ministry

ministry

n3wb
Feb 7, 2025
11
8
Italy
Hello!
I'm now in the community and I have a weird problem that is driving me nuts!! I hope this is the right forum section...

So, the explanation will be a bit long. A few months ago, I installed an Intellinet POE switch to power 3 outdoor cameras from different brands.
The switch have a budget power of 140W, 30w max for each port and each camera requires less than 12w.
I use Frigate as nvr. For several months, everything worked normally, but for the past few days, the cameras regularly go offline at exactly 12:00 PM or 2:00 PM (both from Frigate and from their native apps), while the Wi-Fi cameras continue to work.

This is my network configuration.
network.jpg

The cameras have static IPs and belong to the same subnet as the other devices (192.168.0.*). Both switches are unmanaged.

The issue only affects the POE cameras connected to switch B, while all the devices connected to switch A or via Wi-Fi work normally and are accessible from the outside.

I disabled Frigate thinking the issue might be there, but that didn’t solve it.

I set up a script that runs a traceroute from my PC (behind switch A) to one of the cameras (on switch B), and until just before 12:00 PM, I get responses in a few ms, but right after that, there’s no reply.

Now, the strange part: if I turn off and on switch B or the router, nothing changes, but if I restart switch A, the cameras come back online immediately! So I don’t understand where the issue is.
I thought it could be a hardware failure, but the fact that it happens exactly at 12:00 PM suggests a software cause.

What can I do to debug the situation??
 
  • Like
Reactions: mat200
Are the cameras still working and just not accessible via app? In other words, it may sound silly but are the devices still receiving power at 12:00 PM? I've saw a crazy problem with a UPS once.

Are they recording to local storage like an SD card that you can verify at 12:00 PM?

I'd start there if you can.

After you verified the power and possibly SD card and it's just application access then you can go to next step:

Try moving the PC you are using to access the cameras to Switch B. Since they are all unmanaged it should be quick and easy. Test it at 12:00 PM.

My first guess would be its probably a problem with Switch B. But then you said the following which is weird:

Now, the strange part: if I turn off and on switch B or the router, nothing changes, but if I restart switch A, the cameras come back online immediately! So I don’t understand where the issue is.
Click to expand...

If you are familiar with Wireshark or a similar tool you can look at the traffic at 12:00 PM to see if something is causing a problem. Almost seems like a Denial Of Service but strange that it only affects the cameras.

If you are not familiar with Wireshark or similar tool you go back to old school troubleshooting (although timely and a painful)...

Since it sounds like something on Switch A is causing a network problem start there. Right before 12:00 PM remove Switch A (and thereby it's devices) and see if the problem still occurs. If that doesn't solve the problem then it's it NOT Switch A or those devices. So it can be Switch B, the router, or other Wifi Device.

If it DOES solve the problem then you narrowed down the problem a bit to Switch A or a device connected to it. The next day keep Switch A connected but remove the devices to see if it is the switch itself (doubtful but best to confirm). Then add one device at a time per day narrow down the root cause.

Good luck.
 
  • Like
Reactions: looney2ns
Move one camera to a separate POE injector and plug it in to switch A and see if it stays up.
 
All cameras go offline everyday at the same hour and don't come up until I manually power cycle the switch.

AI answer:

If all your PoE cameras consistently go offline at the same time each day and only come back online after manually power cycling the switch, it most likely indicates a problem with the PoE switch itself, potentially related to its configuration, power limitations, or a scheduling feature that might be unintentionally set to temporarily disable power on specific ports at that time.

Possible reasons and troubleshooting steps:
Insufficient PoE power:

Check if the PoE switch can provide enough power for all your cameras, especially if you've recently added new cameras or if the switch is nearing its capacity.
Solution: Upgrade to a higher power PoE switch or distribute cameras across multiple switch ports to balance power usage.

Switch configuration issue:
Check PoE port settings: Ensure all the PoE ports on the switch are set to "Auto" or the correct power level needed by your cameras.
Check scheduling features: Review any scheduling settings on the switch to see if there's an unintended power cycle programmed for the specific time your cameras go offline.
Port overload: If multiple devices are plugged into a single switch port, consider distributing them across different ports to avoid overloading.

Faulty switch components:
Check switch health: Monitor the switch's status lights to see if any indicate potential issues.
Try a different switch: If possible, temporarily connect your cameras to a different PoE switch to see if the problem persists.

Network issues:
Check network connectivity: Ensure your network is stable and not experiencing any disruptions around the time your cameras go offline.
Router settings: Verify that your router is properly configured and not causing any network conflicts.

What to do next:
Consult your switch manual:
Refer to your PoE switch user guide for specific troubleshooting steps and configuration options.
Contact your network administrator:
If you're not comfortable managing network settings, consult your network administrator for assistance.
Consider a PoE injector:
If the issue is related to insufficient power, a PoE injector might be a solution to boost power to specific cameras.

Personally what helped me with my POE WAP issue was using a PoE injector and running a separate cable to AP.
 
AlwaysSomething said:
Are the cameras still working and just not accessible via app? In other words, it may sound silly but are the devices still receiving power at 12:00 PM? I've saw a crazy problem with a UPS once.

Are they recording to local storage like an SD card that you can verify at 12:00 PM?

I'd start there if you can.

After you verified the power and possibly SD card and it's just application access then you can go to next step:

Try moving the PC you are using to access the cameras to Switch B. Since they are all unmanaged it should be quick and easy. Test it at 12:00 PM.

My first guess would be its probably a problem with Switch B. But then you said the following which is weird:



If you are familiar with Wireshark or a similar tool you can look at the traffic at 12:00 PM to see if something is causing a problem. Almost seems like a Denial Of Service but strange that it only affects the cameras.

If you are not familiar with Wireshark or similar tool you go back to old school troubleshooting (although timely and a painful)...

Since it sounds like something on Switch A is causing a network problem start there. Right before 12:00 PM remove Switch A (and thereby it's devices) and see if the problem still occurs. If that doesn't solve the problem then it's it NOT Switch A or those devices. So it can be Switch B, the router, or other Wifi Device.

If it DOES solve the problem then you narrowed down the problem a bit to Switch A or a device connected to it. The next day keep Switch A connected but remove the devices to see if it is the switch itself (doubtful but best to confirm). Then add one device at a time per day narrow down the root cause.

Good luck.
Click to expand...

The cameras are offline but they are powered on; I can see the led blinking and, moreover, my Reolink lights up a flash when a motion is triggered.
On the switch ports, the lights are on but not blinking, meaning that there is no data traffic.

Eventually, tonight I come back and connected my laptop to switch B, and immediatly all lights started blinking and cameras came up, as they were in a sort of "sleep mode"!
What's going on??

Anyway, I'll proceed as suggested disconnecting switch A
 
Last edited:
  • Like
Reactions: pete_c
I have had good luck here with TP-Link Easy Managed switches.
 
ministry said:
The cameras are offline but they are powered on; I can see the led blinking and, moreover, my Reolink lights up a flash when a motion is triggered.
On the switch ports, the lights are on but not blinking, meaning that there is no data traffic.

Eventually, tonight I come back and connected my laptop to switch B, and immediatly all lights started blinking and cameras came up, as they were in a sort of "sleep mode"!
What's going on??

Anyway, I'll proceed as suggested disconnecting switch A
Click to expand...

So you can rule out power issue.

Keep in mind that if no app (Frigate or web app) is actively accessing the cameras then there will be no network traffic and lights on the switch ports won't be blinking or blinking slowly.

Keep the PC on Switch B till 12:00 PM the next day and see if the problem comes back. If the problem comes back it's probably a problem with the PC. You can also try accessing the cameras from another device (mobile phone or another PC) and see if that works. If the PC does NOT work but the mobile phone does works then chances are 99.9% that the PC is the problem.
 
First thing that comes to mind: do you really need a topology that forces all the camera feeds through the router before being distributed back out to the other devices? It seems like a direct cable link between the two switches might potentially solve this problem (unless you are using the router for VLAN isolation).
 
  • Like
Reactions: AlwaysSomething and bigredfish
AlwaysSomething said:
So you can rule out power issue.

Keep in mind that if no app (Frigate or web app) is actively accessing the cameras then there will be no network traffic and lights on the switch ports won't be blinking or blinking slowly.

Keep the PC on Switch B till 12:00 PM the next day and see if the problem comes back. If the problem comes back it's probably a problem with the PC. You can also try accessing the cameras from another device (mobile phone or another PC) and see if that works. If the PC does NOT work but the mobile phone does works then chances are 99.9% that the PC is the problem.
Click to expand...

What PC are you referring to? The notebook? This is a completely "new" machine that I had in a wardrobe, so it's not involved in the issue
 
Ok! I kept the notebook connected to switch B and cameras and all other devices are still perfectly working!
Then I tried disconnecting all cameras and I noticed that one of them was still connected via wifi (probabily due to initial setup) even if I can't find it in router devices list, so I don't know its ip address. Can this be the problem? I'll try to figure it out
 
  • Like
Reactions: mat200 and pete_c
It happened again twice, with the notebook connected!
The first time at exactly 4:00 AM:
  • The cameras were powered but offline.
  • The notebook was also offline and couldn't ping the cameras.
    Unplugging and reconnecting the notebook to switch B didn't work; neither did restarting switch A. To "restart the network," I had to bypass the router by directly connecting switch B to switch A (why?).
At this point, I blocked all ports on the router to rule out external attacks.

​

At exactly 7:00 AM, it happened again with the same symptoms! I tried turning everything off (router, switch A, and switch B), but all devices on switch B were still offline. I reconnected switch B directly to the router, and everything came back online (why?).

What I noticed:
When offline, the notebook (Ubuntu 20.04) loses its network address and does not appear anymore among the connected devices on the router.

Meanwhile, I was monitoring the network from another PC using tcpdump but I don’t know how to interpret the results
 
Might be stressing the POE switch. Wire up the network as shown below. Disconnect 2 of the 3 POE cameras and try again or use a POE injector and connect to switch A. Also what is the MFG and model of your router (router = switch, AP and firewall).

network2.jpg
 
  • Like
Reactions: jrbeddow
Again from Google AI:

A failing PoE switch will typically exhibit symptoms like powered devices (like IP cameras or access points) not powering on, experiencing intermittent power loss, network connectivity issues, and may show indicator lights on the switch indicating a power problem, all stemming from the switch's inability to deliver sufficient power over the Ethernet cable to the connected devices.
Key symptoms of a failing PoE switch:
  • Powered devices not working:
    This is the most obvious sign, where connected devices like IP cameras or access points simply won't turn on or will frequently lose power due to insufficient PoE power from the switch.

    • Faulty power supply:
      Issues with the switch's power adapter could prevent it from delivering adequate power.

    • Damaged internal components:
      Internal components within the switch may be damaged, affecting power delivery.

    • Overloaded power budget:
      If too many PoE devices are connected drawing more power than the switch can handle, it may lead to issues.

    • Faulty cabling:
      Damaged or incompatible Ethernet cables can cause power loss and network problems.

    • Incorrect PoE configuration:
      Mismatched PoE standards or improper switch settings can lead to power delivery problems.
 
Hello again
I already tried to reconfigure my devices as you suggested, but it didn't work.
I think I've tried all possible combinations (changing network cables, replacing the switch's power cable, connecting one camera at a time), but nothing… sooner or later, the cameras go offline.

In fact, the problem seems to have worsened because now it's unpredictable: sometimes they disconnect within a couple of hours, sometimes after half a day. Also, restarting the switch is not always enough (for example, last night I turned it off and on again, but the cameras remained offline… only to come back online in the middle of the night).

The only test I haven't done yet is completely disconnecting switch A and leaving only switch B, but that’s impossible because I need my devices to be online during the day.

In short, I've given up and requested a replacement for switch B, hoping the problem isn’t elsewhere. But could a faulty PoE camera have permanently damaged the device?
 
I would assume that it is a faulty POE switch rather than a faulty POE camera. My guess is that the replacement POE switch will work and that the cameras did not get damaged.

BTW I worked on the Milan Airport (MXP) when it was getting rebuilt in the late 1990's / early 2000's. It is a work of art.
 
  • Like
Reactions: AlwaysSomething
As many on this site have recommended it's worth it to pay a little more for a better switch.

I have 2 x TP-Link TL-SG1005P, 5 Port Gigabit PoE Switches that I've been using for a few years now with no problems.

I then bought a STEAMEMO 8 Port Gigabit PoE which was recommended by some on this site. I liked it at first but then I had problems similar to you where some or all cameras were still getting power but not network connectivity. However, I was not having the problem as frequently as you. Eventually after 6 months some of the ports just stopped working even after a power reset. Since others had liked it I thought I got a bad one so I replaced it. After replacing them 2 more times I gave up on them.

I finally bought a TP-Link TL-SG1218MP 16 Port Gigabit PoE Switch which has been running well for a few months now.

All the those switches plug into a TP-Link TL-SG108 8 Port Gigabit (non-POE) which is where my dedicated Blue Iris PC and a VM PC are plugged into.

This LAN is dedicated for cameras only. The Blue Iris PC has a 2nd NIC which connects to my main LAN.
 
Just an FYI. I have not had any issues with my 24 port managed TP-Link switches, 4, 8 and POE port managed switches in years.

Here also utilizing a PFSense firewall with multiple LAN / WAN ports connected to a broadband cable modem.

I came from the commercial Cisco world for work and initially for home. (Aerospace).
 
Thoughts:
Whats the Make and Model of all the switches and the router?
Are they managed or unmanaged switches?
Have you tried factory defaulting switch B?
Have you tried factory defaulting switch A?
If you have not try doing B then wait a day and see if it helps. Then do A.
Is the firmware updated on all the devices?

Lets start there.
 
You must log in or register to reply here.
Top Bottom