Rainbow Corruption

truglo

Pulling my weight
Joined
Jun 28, 2017
Messages
275
Reaction score
103
Update... from rainbows, to gaps in continuous footage.

My system has been up and running since my previous post without the type of rainbows we've discussed so far. However now I have noticed 3 other problems which are just as vaporous to troubleshoot. The first one is more of an annoyance, the second one is what I consider a big deal. The 3rd problem is I think another manifestation of the same HA rainbow issue.

1) Access through the mobile app has been a bit buggy. Half the time when I login I get a perpetual progress circle, and I have to click on another page to get the app working. For example when I login to cameras first and it happens to be frozen, I'll have to click alerts. Likewise if I login to alerts first, clicking cameras gets things running if it is frozen. I mention this behavior in this thread because it may be related to HA issues (then again for all I know it could be just a bug in the mobile app).

2) I also noticed my BI logs randomly have "camera signal restored" in the list. This is obviously a concern, so I investigated recorded video around the time of the last 'cam restored' log. Sure enough, there was about a 30second gap in the recording (note no rainbows, just a pause)! Nothing happened during this particular brief outage, but that opens up the slight chance that a burglar could happen to sneak by right as it's happening (no bueno).

3) As I was dragging the timeline cursor back and forth across this problem area, BI started showing rainbows on the BVR playback screen. When I clicked out of playback back to the main screen the cameras all showed fine. Then reviewing the footage in question again the rainbows were not there. I am counting that as another rainbow fail, and thus have moved to testing with all the "also BVR" boxes unchecked.

Hopefully the BVR box is the g-spot I'm looking for and it will at least fix 2 & 3... we'll see. I'll report back if anything else comes up.

Kevin
 

bp2008

Staff member
Joined
Mar 10, 2014
Messages
12,666
Reaction score
14,007
Location
USA
1) Sounds like connectivity problems. If the app failed to load the camera or alert list, that might result in the symptoms you describe. Usually this can be blamed on having overloaded the available internet upload bandwidth at the location of the Blue Iris server.

2) Your BI server is wired to the network, right? I have a tool that can help diagnose network connectivity issues. GitHub - bp2008/pingtracer: Ping Tracer continuously pings each network host between your computer and a given destination, helping identify the source of connectivity problems. If you point this at one of your problematic cameras, it will keep track of ping response time and give you a rough indication of your packet loss %. Any significant amount of packet loss (any at all) could result in dropped or corrupted frames or even dropped streams. Especially if the streaming is happening over UDP. Maybe corrupted frames are responsible for starting the rainbow effect, even.

3) That does sound about right. Hardware acceleration causes more problems than it is worth for reviewing clips.
 

truglo

Pulling my weight
Joined
Jun 28, 2017
Messages
275
Reaction score
103
1) Happened on lan as well (100mbps on an way underutilized enterprise switch... 3 cams 8k a pop... plenty of overhead). My vpn, which I use for remote, gets a minimum of 18mbps upload throughout the day.

2) I had the same thought that it could be just a dropped packet... but further inspection of the camera itself showed a login/logout at that moment. Hmm... wasn't me, and the way my switch+router are setup the only thing that can ping to the cams is the bi server and other cams. I also noted the bi recording got slightly dim on the frame just before dropout occured. A port mirror and wireshark may be in order this evening.

Very worthy of note: since unchecking "also BVR" the app has not stalled. If it does I will report back, but so far everything works as if ha was off (other than very low cpu usage).
 

truglo

Pulling my weight
Joined
Jun 28, 2017
Messages
275
Reaction score
103
A quick note on #2 from my previous post: "ONVIF login/logout" lines do normally appear on my camera logs when BI sends html get commands from an event schedule (ie, morning time BI sends a command to switch to day mode then refocus). However when the log is due to a get command, there is usually an indication of that in the logs as well (like "/cgibin/...ir off"). In the case of the myserious login/logout that resulted in 30sec of lost footage, there was no related "/cgibin" message in the log.

Is it possible BI is logging in to cameras to do something else besides events and stream recording?

BTW, no signal restored messages since unchecking also bvr. It's probably too early to say it's fixed, but my fingers are crossed.
 

k2ace

n3wb
Joined
Dec 29, 2016
Messages
18
Reaction score
1
I experienced the pixelated rainbow issue recently. It only effected a single foscam R2 camera. The other 6 cams linked are all dahua or hik cams, they weren’t effect. I’m running BI on a box with a MSI board, intel 4600 graphics, I7 4th gen, server 2016. It started about a week after I installed the latest intel drivers trying to fix what was believed to be a memory leak issue related to the intel 4600 drivers I was running. I was having to reboot the box about every week to keep it stable. The rainbows only occurred on the top of preview and alert images, all the recordings were fine. Initial I just disabled hardware decoding to solve the issue. This worked but I noticed cams were loosing connections randomly and other odd performance issues. In the end I uninstalled the intel drivers completely so its just using the Microsoft Basic Display Adapter drivers. Since then it’s been running for over a month, no drop connections or rainbows with hardware decode enable on all cams. No more memory leak and overall CPU usage is now lower then I have ever seen running. It was runing arround 45% before intel driver updates, 60% after intel driver update, and now 30% after removing intel graphics drivers completely.
 

ksnax

Young grasshopper
Joined
Jul 28, 2017
Messages
35
Reaction score
4
I recently started having this issue with an older DVR feed from four cameras. It affects all of them, but no direct to disc, so the recordings get corrupted as well. This thread has given me a good idea what to look at, but I figured I'd toss out my temporary solution of changing camera settings to restart every 5 minutes. The feeds still corrupt, but at least they don't stay that way until the next time I check. My other IP cams have no issues at all. Windows however, being like a 2-year old that just learned the word 'no' with respect to updates, is where I am focusing my attention going forward . .
 
Last edited:
Joined
Mar 28, 2018
Messages
11
Reaction score
6
Just to add my experience, I was rainbow free (using direct to disk and hardware acceleration) until I ticked that box marked 'also bvr'. I got rainbows (on playback only) until I unticked the box. All good again now.
 

cam913

n3wb
Joined
Dec 5, 2019
Messages
7
Reaction score
1
Location
Canada
I have had the Rainbow Corruption too. I have updated the Intel Graphics HD4000 from the 2015 driver to the latest v10.18.10.5069 2018-12-27 driver and still seen the same issue. The one fix was changing the Power Plan from Power Saver to High Performance in Windows.
 

m_listed

Getting the hang of it
Joined
Jun 11, 2016
Messages
176
Reaction score
57
I'm getting this issue on a Jidetech cheap Chinese camera. About to rip my hair out.

It works with H265 with hardware acceleration (which is amazing, but all the playback BVRs play just a black screen with H265). It works with H264 without hardware acceleration. But it does NOT work with H264 with hardware acceleration. And I need hardware acceleration. BVR playback is fine in all cases.

If I set the I-frame interval to 10 and the FPS to 20, it works for a while, goes pride for like 30 seconds, and then comes back to normal for a few more minutes, and so on.

Already on high performance mode.
 
Last edited:

tai4de2

Young grasshopper
Joined
Aug 25, 2019
Messages
41
Reaction score
15
Location
Near Seattle
I too am quite familiar with the rainbow corruption. 15 mostly 4K Amcrest cameras using H264, 10fps, into a Core-i9, direct to disk in BI, latest Intel drivers.

The only solution I ever found was to turn off hardware acceleration completely. The box steady state is still less than 10% CPU.

If I turn on hw accel I sometimes get the rainbow corruption, but I reliably get smearing in the playback. I know at least the latter can be fixed by forcing an old driver, but I prefer to just let Windows keep the machine up to date.

I never observed the Also BVR option to have any effect.

I think this has something to do with contention for the decoding hardware resources. I also run Sighthound in the same machine, but using the cameras’ VGA secondary streams. The rainbow corruption would sometimes only be in the upper left corner of the BI camera feeds — a VGA-size stamp on the 4K frame.


Sent from my iPhone using Tapatalk
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
I too am quite familiar with the rainbow corruption. 15 mostly 4K Amcrest cameras using H264, 10fps, into a Core-i9, direct to disk in BI, latest Intel drivers.

The only solution I ever found was to turn off hardware acceleration completely. The box steady state is still less than 10% CPU.

If I turn on hw accel I sometimes get the rainbow corruption, but I reliably get smearing in the playback. I know at least the latter can be fixed by forcing an old driver, but I prefer to just let Windows keep the machine up to date.

I never observed the Also BVR option to have any effect.

I think this has something to do with contention for the decoding hardware resources. I also run Sighthound in the same machine, but using the cameras’ VGA secondary streams. The rainbow corruption would sometimes only be in the upper left corner of the BI camera feeds — a VGA-size stamp on the 4K frame.


Sent from my iPhone using Tapatalk
Incorrect. There have been a bunch of changes to both blue iris intel hardware acceleration code and intel drivers since this thread was started. update both to the latest. You will not get smearing with the latest driver and setting the intel graphics to maximum performance. This usually occurs when running headless. Using an hdmi emulator also helps.
 

tai4de2

Young grasshopper
Joined
Aug 25, 2019
Messages
41
Reaction score
15
Location
Near Seattle
Incorrect. There have been a bunch of changes to both blue iris intel hardware acceleration code and intel drivers since this thread was started. update both to the latest. You will not get smearing with the latest driver and setting the intel graphics to maximum performance. This usually occurs when running headless. Using an hdmi emulator also helps.
I tried it recently (within the last month or two — admittedly, an eternity in cybertime) and had both smearing and rainbows, but will try again soon. I let Windows keep my machine up to date but will also check on the version of my video drivers.

I don’t run BI headless and never have, so that’s not my issue.


Sent from my iPhone using Tapatalk
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
I tried it recently (within the last month or two — admittedly, an eternity in cybertime) and had both smearing and rainbows, but will try again soon. I let Windows keep my machine up to date but will also check on the version of my video drivers.

I don’t run BI headless and never have, so that’s not my issue.


Sent from my iPhone using Tapatalk
This smearing issue has been resolved long ago. See threads. Ha code had been modified in the last few weeks. If you used Your oem windows vs ms media creation tool you are not getting the latest drivers.
 

tai4de2

Young grasshopper
Joined
Aug 25, 2019
Messages
41
Reaction score
15
Location
Near Seattle
This smearing issue has been resolved long ago. See threads. Ha code had been modified in the last few weeks. If you used Your oem windows vs ms media creation tool you are not getting the latest drivers.
I discovered tonight that my Windows installation was stuck on an Intel driver from 2018 — probably Intel’s final pre-DCH driver. It was a little tricky to coax Windows into installing the very latest (a DCH driver from 3/25/2020) but I did it and now I’m running a driver from about two weeks ago.

I also made the change you suggested to bump up the Intel driver power plan to High Performance.

All my cameras were set to “default” for HA. I turned HA back on using the master setting, which worked, but then I rebooted and when the system came back up all the cameras had changed their individual setting to “No” for HA. I had to manually reset all of them back to “Default” but now HA is working. Hopefully that doesn’t happen again.

BI CPU is 2-3% and GPU is about 6%. I am running 10fps and “limit decoding” on. Plenty of head room. I turned “Also BVR” off for all cameras because playing back multiple cameras at once was maxing out the GPU and causing glitches.

Let’s see if this resolved the issues I was seeing previously with HA.

Sent from my iPad using Tapatalk
 

fenderman

Staff member
Joined
Mar 9, 2014
Messages
36,897
Reaction score
21,250
I discovered tonight that my Windows installation was stuck on an Intel driver from 2018 — probably Intel’s final pre-DCH driver. It was a little tricky to coax Windows into installing the very latest (a DCH driver from 3/25/2020) but I did it and now I’m running a driver from about two weeks ago.

I also made the change you suggested to bump up the Intel driver power plan to High Performance.

All my cameras were set to “default” for HA. I turned HA back on using the master setting, which worked, but then I rebooted and when the system came back up all the cameras had changed their individual setting to “No” for HA. I had to manually reset all of them back to “Default” but now HA is working. Hopefully that doesn’t happen again.

BI CPU is 2-3% and GPU is about 6%. I am running 10fps and “limit decoding” on. Plenty of head room. I turned “Also BVR” off for all cameras because playing back multiple cameras at once was maxing out the GPU and causing glitches.

Let’s see if this resolved the issues I was seeing previously with HA.

Sent from my iPad using Tapatalk
What exact version of blue iris are you running?
 

tai4de2

Young grasshopper
Joined
Aug 25, 2019
Messages
41
Reaction score
15
Location
Near Seattle
So far so good on both smearing and rainbow corruption. Not seeing either of ‘em.

I’m running whatever the latest version of BI5 was, released yesterday (not in front of my computer to check at the moment).


Sent from my iPhone using Tapatalk
 
Top