I went on vacation for 3 weeks

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
On July 19, 2024 at approximately 10:42 PM my BI NVR with 9 cameras whited out (the images recorded were white in a non-uniform fashion as if a phenomenon was being recorded) as if overwhelmed by a lighting bolt. 3 of the cameras did not come back online and went "dead" while the rest continued to function. The 3 cameras are on the same BV-Tech 8 Port PoE Switch with 1 Gigabit Uplink while the other 6 are on another POE switch of the same brand.

5 days later the three cameras come back online at 8:04 AM (I think simultaneously).


During those black out days I figured maybe the electrical panel tripped a circuit, or that the POE switch burned out, or ...

Well, when the cameras came back on line I felt hacked was the best explanation and when I came home that felt even more true since the circuit board [panel] wasn't tripped and the little freezer on that circuit was still frozen with no signs of melting.


ANYBODY HAVE SOMETHING LIKE THIS HAPPEN TO THEM? Do you have an explanation?

P.S. No lighting strike signs on the roof or building.
 
Last edited:

Teken

Known around here
Joined
Aug 11, 2020
Messages
1,832
Reaction score
3,250
Location
Canada
On July 19, 2024 at approximately 10:42 PM my BI NVR with 9 cameras whited out (the images recorded were white in a non-uniform fashion as if a phenomenon was being recorded) as if overwhelmed by a lighting bolt. 3 of the cameras did not come back online and went "dead" while the rest continued to function. The 3 cameras are on the same BV-Tech 8 Port PoE Switch with 1 Gigabit Uplink while the other 6 are on another POE switch of the same brand.

5 days later the three cameras come back online at 8:04 AM (I think simultaneously).


During those black out days I figured maybe the electrical panel tripped a circuit, or that the POE switch burned out, or ...

Well, when the cameras came back on line I felt hacked was the best explanation and when I came home that felt even more true since the circuit board wasn't tripped and the little freezer on that circuit was still frozen with no signs of melting.


ANYBODY HAVE SOMETHING LIKE THIS HAPPEN TO THEM? Do you have an explanation?

P.S. No lighting strike signs on the roof or building.
Having read your initial post I still have no clue what you’re trying to say or inferring to. :facepalm:

So I’m going to break this down because there’s a lot to unpack here.

There’s a problem with the BI NVR because all the recorded video is white. If you play back that video what period does this white out cover?!? Is there anything audible/ visible regardless of how small on the recording ?!?

Three cameras went dead?!? You know this how?? Because you can’t access the internal webpage of the camera? You can’t ping the cameras? If the system has the most basic monitoring present you would be able to review the logs for reboot, cold start, illegal login, NTP, etc.

Im going to assume you don’t have any kind of NMS in place and thus no SNMP traps enabled or Syslog server logging.

A basic ping monitoring system would validate if the camera / NVR / Network Appliances were not reachable. Which only provides some insight and isn’t a silver bullet as to everything.

It goes without saying if email alerts are setup this too would offer another layer of insight.

So the initial belief is lightning was present. Your only cursory review is checking the roof for signs of lightning???

99.99999% of all lightning damage comes via wiring. It doesn’t matter if it’s cable, telephone, satellite, Ethernet, etc.

A large percentage of lightning damage comes from induced (EMF) voltage through the air and injected into the wiring / electrical system.

As a lay person your only method to know if lightning was root cause is to go through the entire house. Review everything that was connected to an outlet / circuit and validate how they operate.

This cursory review (still) doesn’t tell you with 100% certainty that a device isn’t damaged! If you see something obviously damaged / burned that is the extreme case.

Almost in every case 99.999999% the components in the various devices are damaged and simply have a shorter life span or begin having odd issues to behaviour.

As it relates to breakers tripping during a lightning event. :lmao: By the time a individual breaker never mind the main breaker trips. You would know immediately because everything on that branch circuit would either be on fire or clearly BBQ!

Than, your reference to the freezer board?!? I’m going to assume you mean the pilot light that indicates there’s 120 VAC present?? As this doesn’t tell you the freezer is frozen / at temperature.

Lastly, we come to being hacked???

You have what in place to determine this root cause?? Do you have a firewall monitoring all the network traffic in your LAN / WAN??

Are you port forwarding?? Using any of a dozen P2P services? Subscribed and using any of those cloud services without any regards to network security, segmentation, isolation, and separation???

This is where you are right now: 0 Facts
This is where you need to start: 1 Gather Facts
 

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
Having read your initial post I still have no clue what you’re trying to say or inferring to. :facepalm:
Sorry I was glib.

So I’m going to break this down because there’s a lot to unpack here.
Wow, I didn't expect this much ;0)

There’s a problem with the BI NVR because all the recorded video is white. If you play back that video what period does this white out cover?!? Is there anything audible/ visible regardless of how small on the recording ?!?
No, the BI NVR kept working. I could WOL to try and do a reset of sorts and it went down and came back up. What does bother me is that the recording space was enough for a little more than 3 weeks. Given three cameras were offline for about 5 days I'm suprised I only have 2 weeks of recording to review as of yesterday. Losing the days in question before I backed them up.

Three cameras went dead?!? You know this how?? Because you can’t access the internal webpage of the camera? You can’t ping the cameras? If the system has the most basic monitoring present you would be able to review the logs for reboot, cold start, illegal login, NTP, etc.
The cameras went offline. I tried to http to them directly but got nada. As if they had been unplugged from the POE switch.

Im going to assume you don’t have any kind of NMS in place and thus no SNMP traps enabled or Syslog server logging.
Not using SNMP on my home network but now might do so for the future. I don't think that matters here.

A basic ping monitoring system would validate if the camera / NVR / Network Appliances were not reachable. Which only provides some insight and isn’t a silver bullet as to everything.
The cameras were clearly unreachable but the NVR was as was the rest of the network. I will be putting the cameras on a vlan and not just behind a second NIC on the NVR. These Chinese cameras make me nervous and before I was using zoneminder on freebsd with PF firewall to isolate the cameras.

It goes without saying if email alerts are setup this too would offer another layer of insight.
No email. Just mms texts. Which kinda sucks... often be delivered hours and even a day after an alert was triggered. That's a gmail problem.

So the initial belief is lightning was present. Your only cursory review is checking the roof for signs of lightning???
Well, all the cameras got affected by what I assumed was a bright light from a bolt of lightning inundating the house (on a 360 basis around the house).

99.99999% of all lightning damage comes via wiring. It doesn’t matter if it’s cable, telephone, satellite, Ethernet, etc.

A large percentage of lightning damage comes from induced (EMF) voltage through the air and injected into the wiring / electrical system.
I agree.

As a lay person your only method to know if lightning was root cause is to go through the entire house. Review everything that was connected to an outlet / circuit and validate how they operate.
Again, the cameras all seemed to be flooded by a white light at the same time.

This cursory review (still) doesn’t tell you with 100% certainty that a device isn’t damaged! If you see something obviously damaged / burned that is the extreme case.
It's all working fine now and the lightining hypotheses has now been a long disabused notion by me.

Almost in every case 99.999999% the components in the various devices are damaged and simply have a shorter life span or begin having odd issues to behaviour.
I continue to agree. And this is why a [cr|h]ack seems increasingly the only option by a process of elimination

As it relates to breakers tripping during a lightning event. :lmao: By the time a individual breaker never mind the main breaker trips. You would know immediately because everything on that branch circuit would either be on fire or clearly BBQ!
I didn't say the lightning tripped the breaker. It was an option that could have happened without any kind of strike. It was a line item trying to explain what had transpired.

Than, your reference to the freezer board?!? I’m going to assume you mean the pilot light that indicates there’s 120 VAC present?? As this doesn’t tell you the freezer is frozen / at temperature.
A freezer is on the same circuit. If the breaker tripped I was worried about all the stored steaks. Hmmm... think I'm going to pull one out.

Lastly, we come to being hacked???

You have what in place to determine this root cause?? Do you have a firewall monitoring all the network traffic in your LAN / WAN??

Are you port forwarding?? Using any of a dozen P2P services? Subscribed and using any of those cloud services without any regards to network security, segmentation, isolation, and separation???

This is where you are right now: 0 Facts
This is where you need to start: 1 Gather Facts
I've been a long time pfsense user (still an ISA Server fan). Way before it became as popular as it is now. Put my own hardware together to do so kinda reverse engineering the netgate specs. I used a VPN to see my cameras both on my laptop and smart phone with great ease.

I ignored your hostile tone and mocking of sorts because I can.

Peace out dude.

P.S. All Chinese made cameras are suspect by default for me since in the early 2000's I had a camera full of spyware pushing ads and God knows what else (foscom?).
 
Last edited:

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
26,328
Reaction score
51,480
Location
USA
Regarding delayed text messages, it isn't a Gmail problem - you can search your Gmail sent box and see it was sent - it is mobile providers are tightening up their spam algorithms. It is well documented here.

It is why most of us went to the Pushover app instead of relying on text messages.
 

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
Regarding delayed text messages, it isn't a Gmail problem - you can search your Gmail sent box and see it was sent - it is mobile providers are tightening up their spam algorithms. It is well documented here.

It is why most of us went to the Pushover app instead of relying on text messages.
thanks
 

Teken

Known around here
Joined
Aug 11, 2020
Messages
1,832
Reaction score
3,250
Location
Canada
Sorry I was glib.


Wow, I didn't expect this much ;0)


No, the BI NVR kept working. I could WOL to try and do a reset of sorts and it went down and came back up. What does bother me is that the recording space was enough for a little more than 3 weeks. Given three cameras were offline for about 5 days I'm suprised I only have 2 weeks of recording to review as of yesterday. Losing the days in question before I backed them up.


The cameras went offline. I tried to http to them directly but got nada. As if they had been unplugged from the POE switch.


Not using SNMP on my home network but now might do so for the future. I don't think that matters here.


The cameras were clearly unreachable but the NVR was as was the rest of the network. I will be putting the cameras on a vlan and not just behind a second NIC on the NVR. These Chinese cameras make me nervous and before I was using zoneminder on freebsd with PF firewall to isolate the cameras.


No email. Just mms texts. Which kinda sucks... often be delivered hours and even a day after an alert was triggered. That's a gmail problem.


Well, all the cameras got affected by what I assumed was a bright light from a bolt of lightning inundating the house (on a 360 basis around the house).


I agree.



Again, the cameras all seemed to be flooded by a white light at the same time.


It's all working fine now and the lightining hypotheses has now been a long disabused notion by me.


I continue to agree. And this is why a [cr|h]ack seems increasingly the only option by a process of elimination


I didn't say the lightning tripped the breaker. It was an option that could have happened without any kind of strike. It was a line item trying to explain what had transpired.



A freezer is on the same circuit. If the breaker tripped I was worried about all the stored steaks. Hmmm... think I'm going to pull one out.



I've been a long time pfsense user (still an ISA Server fan). Way before it became as popular as it is now. Put my own hardware together to do so kinda reverse engineering the netgate specs. I used a VPN to see my cameras both on my laptop and smart phone with great ease.

I ignored your hostile tone and mocking of sorts because I can.

Peace out dude.

P.S. All Chinese made cameras are suspect by default for me since in the early 2000's I had a camera full of spyware pushing ads and God knows what else (foscom?).
You’re right I was being hostile toward you because I was dealing with an asshole at that moment.

I should not have berated you as I did when you came to the forum looking for help.

For that I do apologize . . . :thumb:

As to mocking you - No

It would be very clear if I was doing so. Regardless, I have not read anything from you as it relates to logs.

You have two pieces of hardware that incorporates logging. Why haven’t you reviewed any of the Windows / camera logs?!?

You have a pfSense firewall in place ?!?!

Logs . . .

I would hope you have a UPS in place??? Again what would that UPS provide you in terms of a lights out / power quality issue???

Logs . . .

So my final comment still stands: 0 Facts
 

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
So my final comment still stands: 0 Facts
This part right here (quoted above) is the uncool part of you.


I once built a physical tap into my network that was invisible to the network and just sucked up network traffic. It became a bit useless with all the default https traffic. Those logs sit around until I get an instance of Zeek/BroIDS going (which is a steep learning curve right now). Remember that online logs can be adjusted.

Tap looks a little bit like this. Mine isn't labeled so well. Mine has sharpie markings. ha ha
1723232519100.jpeg
 

Teken

Known around here
Joined
Aug 11, 2020
Messages
1,832
Reaction score
3,250
Location
Canada
This part right here (quoted above) is the uncool part of you.


I once built a physical tap into my network that was invisible to the network and just sucked up network traffic. It became a bit useless with all the default https traffic. Those logs sit around until I get an instance of Zeek/BroIDS going (which is a steep learning curve right now). Remember that online logs can be adjusted.

Tap looks a little bit like this. Mine isn't labeled so well. Mine has sharpie markings. ha ha
View attachment 200686
Im not here to be cool or to be your friend. I’m here to provide information, knowledge. If I learn something along the way in this forum that’s great.

Regardless, you’ve taken the time to upload two photos of a tap?!?

In the same time you just wasted you could have reviewed the logs from: BI, Windows, Camera, Switch, UPS.

So my last statement remains: 0 Facts
 

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
Im not here to be cool or to be your friend. I’m here to provide information, knowledge. If I learn something along the way in this forum that’s great.

Regardless, you’ve taken the time to upload two photos of a tap?!?

In the same time you just wasted you could have reviewed the logs from: BI, Windows, Camera, Switch, UPS.

So my last statement remains: 0 Facts
buh-bye
 

CCTVCam

Known around here
Joined
Sep 25, 2017
Messages
2,744
Reaction score
3,616
I've just had a 4kt white out progressively over several days. No lightening and no apparent cause. Everything else is working fine on the network inlcuding other cameras in BI, bothi n the same switch. Mine's behind 2 firewalls with no remote access - technically it's VPN but the VPN isn't active atm in the router. Resetting it left it contactable via the web interface although I did eventually manage to re-establish contact. I did have a power cut but why a power cycle would destroy a camera is beyond me. If that's the case, Dahua need to look at fitting a soft start circuit.
 

spycke

n3wb
Joined
Apr 16, 2024
Messages
19
Reaction score
8
Location
florida
I've just had a 4kt white out progressively over several days. No lightening and no apparent cause. Everything else is working fine on the network inlcuding other cameras in BI, bothi n the same switch. Mine's behind 2 firewalls with no remote access - technically it's VPN but the VPN isn't active atm in the router. Resetting it left it contactable via the web interface although I did eventually manage to re-establish contact. I did have a power cut but why a power cycle would destroy a camera is beyond me. If that's the case, Dahua need to look at fitting a soft start circuit.
Similar but different experiences. My 3 down cameras came back up 5 days later without any input on my part. This is something I'm going to remember. Funny that the six cameras that didn't go down were of the same brand and model while the three that went down were different brands and models from the six and each other. Kinda want it to happen again while I'm on site. But not really... maybe
 
Joined
Aug 8, 2018
Messages
7,804
Reaction score
27,587
Location
Spring, Texas
On July 19, 2024 at approximately 10:42 PM my BI NVR with 9 cameras whited out
So are you saying that you have a BlueIris server, or an NVR plus a BI server? What cam brand and models?

5 days later the three cameras come back online at 8:04 AM (I think simultaneously).
Is there anything significant about the time 8:04am? Like does your NVR reboot at 8:00am once a week?

If you are using BlueIris, is there any entries in the BI logs? If it is an NVR, does it have logs that show anything at those times?

Summary...all 9 cams whited out on 7/19/2024 at 10:42pm. Then 6 cams came back (not whited out) correctly but 3 cams were offline. How long were the 6 cams whited out? Do you have saved video of those whited out cams?

Was there a storm when those cams whited out? Did the POE switch that hte 3 cams that were offline for 5 days have any other devices on it? If so, did they go offline or have any issues?

Could you have had a momentary power loss that caused your system to reboot?

I have seen all my outdoor cams white out due to lightning, but it was not electrical, rather the flash of light overwhelming the CMOS. So it only happened for a few seconds. At the same time, I have had cams go offline and reboot but not 5 days later. I have a Dahua B5442 Z4E that tends to do this fairly often during lightning storms.
 
Top