Blue Iris UI3

[erkme73]

I didn't change anything related to layouts in 201-203, but it is possible you were just remembering behavior in 197-199 and did not actually see it broken in 200.

I will adjust the "stall detection" again for version 204, and make it wait a little longer for signs of activity before declaring the video player dead.

720p @ 5fps and single camera views should stream perfectly. If you open "Stats for nerds" it will show you the network delay and the player delay, that is a clue for the source of the problem. Network delay could be anything from the network or Blue Iris or even a camera problem. Player delay is 100% internal to UI3 so if you get a lot of delay there it could be worth changing the H.264 player choice in UI Settings.

Thanks for your help.

Well, if it helps any, I've had a single playback camera screen up for most of the day on UI3 - paused. When I clicked the X to go back to live, I got the live camera immediately. I did nothing else, and within a few seconds I got the "player stall" error. It was 1:1 on the zoom, and no input. I suppose it's possible that after sitting for hours on a paused playback clip that it some how lost its bearings?

ETA: I'm not on 204 yet. This is still 203 from last night. I was just commenting that it didn't involve any zooming. I'll jump to 204 and see what happens.

 

[bp2008]

Well, if it helps any, I've had a single playback camera screen up for most of the day on UI3 - paused. When I clicked the X to go back to live, I got the live camera immediately. I did nothing else, and within a few seconds I got the "player stall" error. It was 1:1 on the zoom, and no input. I suppose it's possible that after sitting for hours on a paused playback clip that it some how lost its bearings?

ETA: I'm not on 204 yet. This is still 203 from last night. I was just commenting that it didn't involve any zooming. I'll jump to 204 and see what happens.

Being paused shouldn't have affected it. But yeah in UI3-203 you only needed to have 5 frames queued and for the video player to simultaneously say it was waiting for more data, and it would trigger the stall detection. UI3-204 added a requirement that the video remain stalled for 2 seconds, and that timer increases by 2 seconds every time it triggers (resets to 2 seconds again if you reload the page). So that should eliminate most of the false alarms.

 

[erkme73]

I do have a bit of a follow up question about the jerky playback/streaming. I am quick to say it's all with the BI machine or network since the playback is rough regardless of client browser or machine hardware. However, when looking at the stats for nerds, I see this:



The playback is mostly smooth, but every second or so, it glitches, pauses/skips and the resumes. As it does this the delayed frames goes from 1 or 2 to about 15. At the same time the player delay bumps up to around 400-500ms. The network delay is staying at or very near 0000ms.

Does the screen grab make you think of anything specific? You'd said something about using another player, but the only options I have are auto/html5/java.

Also, the little blue scrubber line that moves across the stats for nerds box stops moving with every glitch - as do all the variables on the box.

It almost seems like a local issue. Yet, it happens on my PC and phone (android).

BTW, the screenshot above was for html5. When I use java (and reload screen), I get this:



While the player delay is relatively smaller (max was <100ms) the same pausing/hiccuping is happening - where all variables freeze with each glitch.

 

[bp2008]

Wow @erkme73 your screenshots have some weirdness in them.

especially this part: and here where there's a cutout:

If that is only that way in your screenshots then I can chalk it up to buggy screen capture software. But if it draws that way in-person too then I suspect hardware problems on your machine.


Anyway...

Some brief network interruption is seen at a few points in your first screenshot. That is where the network delay steadily rises and the player delay falls at about the same rate (at least until it hits 0); that is because the network isn't receiving frames anymore, but the player is still consuming them. Then suddenly the delayed frames all arrive from the network so network delay shoots back to 0 and player delay jumps up because now the player needs to catch up.



The network delay could be caused by a lot of things, but this particularly makes me suspect two possibilities, one being a buggy proxy server being used (possibly integrated into security software you're running on the clients or on the BI server) and the other possibility, just a badly performing network.

One diagnostic tool you could use is PingTracer which is an app of my own design that can send pings to a network host up to 10 pings per second and graph the response times. I suggest you run that on a client machine, pinging the BI machine at 10 pings per second, and see how the graph looks compared to the Stats for nerds graphs. If the network is performing well, it will be a relatively flat green graph, maybe even just 1 pixel tall because on a LAN the response time should be 0 milliseconds typically. If it is not performing well, you will see lots of spikes, possibly colored yellow or red.

If the network is working well for pings, that points the finger at internet security software or a bad proxy server you might have set up.

 

[erkme73]

Wow, thanks for all the detail explanation. A lot to unpack here. I will dig into your suggestions a bit later this evening when I get some more me time. I quickly ran your ping tracer and got this:



I have no proxy set up, and no antivirus software on either machine (other than what Windows 10 comes with - and I've attempted to neuter that as much as possible). Interestingly, the firewall was up on the BI machine (presumably from an update). So I disabled it and still find the same glitching.

 

[tward392]

View attachment 122059

Don't remember what time that was, probably right before I posted it, here it is with the time. Not a big deal to me, just posted that it was doing it on one camera here.

This appears to be fixed now, not sure which version of the last couple I installed fixed it but it's no longer happening.

 

[bp2008]

This appears to be fixed now, not sure which version of the last couple I installed fixed it but it's no longer happening.

Good to know. Thanks.

Wow, thanks for all the detail explanation. A lot to unpack here. I will dig into your suggestions a bit later this evening when I get some more me time. I quickly ran your ping tracer and got this:

I have no proxy set up, and no antivirus software on either machine (other than what Windows 10 comes with - and I've attempted to neuter that as much as possible). Interestingly, the firewall was up on the BI machine (presumably from an update). So I disabled it and still find the same glitching.

That graph looks fine. If the glitching occurred during that graph run and no problem was evident on the ping graph at the same time, then I wouldn't blame it on the network. Sounds like it is probably BI having trouble encoding the streams for the web server then. If your web server streaming profiles are configured to use hardware acceleration, try without it. Not sure what else to suggest. You could screenshot your Streaming 0 configuration I suppose.

 

[erkme73]

Good to know. Thanks.



That graph looks fine. If the glitching occurred during that graph run and no problem was evident on the ping graph at the same time, then I wouldn't blame it on the network. Sounds like it is probably BI having trouble encoding the streams for the web server then. If your web server streaming profiles are configured to use hardware acceleration, try without it. Not sure what else to suggest. You could screenshot your Streaming 0 configuration I suppose.

Ok, I had a little more play time just come up. I went to playback of a clip from earlier this evening. At first it was relatively smooth at the camera frame rate of 20 FPS. While watching the thin blue scrubber line on the stats for nerds window, I could clearly see the artifacts you highlighted earlier. It's not the screen capture software.


It's a 1 min video grab, but around the 10 second mark, the jerkiness returns. It is showing a network delay (and I get the orange clock). At the same time I'm using your ping trace and shows smooth pings to the server (top part of the video).

Clearly there is something going on with how the stats for nerds is showing the parameters visually - but I don't think it's a local client issue since I get the same jerkiness and network delay errors on my Moto Edge android phone.

When I go to the same camera on UI3 on the phone and PC at the same time and I watch the video bit rate, they both mimic each other perfectly - the waveform looks identical, and there is huge variability in that waveform as the jerkiness occurs. The blue line on the phone is intact and does not fracture like the one on the PC - but it does speed up and slow down during the jerkiness, just like on the PC.

 

[bp2008]

So strange with the graph drawing. Maybe a buggy video driver on the PC.

You're saying the phone and the PC both stutter at the same time if they are playing the same thing? That combined with the clean ping graph strongly suggests that it is Blue Iris internally having problems. Is playback smooth in the local console or does it do the same thing?

Some thoughts:

• It could be failing hardware in the BI server causing unusual delays that BI's frame processing pipelines weren't designed to handle.
• Are you using sub streams? They add a lot of complexity to how BI handles frame timing. They're supposed to work best when the frame rate and i-frame interval of the main and sub streams are all the same. e.g. 20 fps, 20 iframe, for both streams. But if the camera is having trouble encoding the streams at full frame rate, or does something weird with the frame timing, that could throw Blue Iris off... would be worth testing with sub streams disabled.
• Streaming 0 configuration could cause problems. Some people go in there and change a bunch of stuff without understanding any of it.
• Try a "non-VBR" streaming profile like 2160p 4K. It is unusual but some setups can buffer network data for quite a while and cause delays if it is a low bit rate stream, which UI3's VBR profiles often are (low bit rate). the 2160p stream all but guarantees you will get 6-8 Mbps which would force such a buffer to clear out quickly.

 

[erkme73]

Yes, if I'm watching the same live-stream camera on both phone and PC, both on the same network, I get identical data on the stats window.

Is the variability of the bitrate waveform normal for an image which is showing no perceptible change in video content?

Every one of my cameras is using sub-streams. And all are set to use same frame and i-frame. Most are set to 15 / 15. Is there a global way to disable sub-streams or would i have to go into each camera to disable it one at at time?


Thing is, I didn't have nearly this amount of jerkiness when running on 5.5.0.15 - and I can revert back to it and I'm smooth. That's really what makes me think there is some kind of incompatibility with my hardware and the whatever changed since that version.

My server is running without a physical monitor (headless) and I'm using VNC to make changes on it. That has some inherent video latency, so I cannot really confirm whether the console has the same jerkiness - at least not without climbing into my network closet and put a monitor on the PC. I'll do it, but only after exhausting some of the easier items on the differential.


As for my stream 0 configuration under the web server options, here's what I currently have setup:





Wasn't quite sure where you meant for me to do the non-VBR. I assume in the UI3 interface? I just tried it for 4k on both phone and PC, and both continued the same behavior.

If I haven't said it enough, I really appreciate your interest in getting me squared away. I miss the days where reviewing an event on BI was fun. Now it's so frustrating to find things...

Fortunately when I export everything is smooth as butter.

Hey, BTW, do you know of a way to cancel a queued export? I posted separately on this but no takers. I accidentally started an export on my phone via UI3 - my fat finger hit export when I was trying to click the timelapse checkbox... and it was before used the red and green trim handles to get it down to what I wanted - and so it dutifully put it in the queue - all 8 hours worth.

 

[erkme73]

Here's another video clip capture showing the lack of fluidity. This is using the 4k (non-vbr) setting in UI3.



Neither player nor network delay appear to be excessive while it's happening. Though the video bit rate is all over the place. Notice also how the blue line on the stats box stops moving forward with each jerk. Does this on the phone as well.

 

[erkme73]

Have a look at these ping graphs. I'm getting this on every device now. Does that point to network issue you think?

 

[bp2008]

Is the variability of the bitrate waveform normal for an image which is showing no perceptible change in video content?

The bit rate is usually very steady in such a case, except when a new i-frame arrives in the stream that causes a massive upward spike (which is pretty infrequent normally in UI3 video streams). The fact it jumps up more frequently after a pause simply indicates there was a period of time when the video wasn't arriving, then whatever blockage was holding it back got cleared, and all that was buffered arrived suddenly which causes a spike in the apparent bit rate. UI3 doesn't calculate bit rate based on frame timestamps, rather on the real world time when the data was received.

Is there a global way to disable sub-streams or would i have to go into each camera to disable it one at at time?

No global way.

As for my stream 0 configuration under the web server options, here's what I currently have setup:

That all looks fine.

Wasn't quite sure where you meant for me to do the non-VBR. I assume in the UI3 interface? I just tried it for 4k on both phone and PC, and both continued the same behavior.

Yes, in UI3 there are the 2160p VBR and 1080p VBR profiles, which have a set quality of 20, and then everything below it is more like "CBR". Technically those are VBR too, but I didn't name them as such because in a default configuration their quality level is so high it will almost always make the stream hit the bit rate cap which makes them effectively "CBR". I actually reduced the default quality settings for most of the profiles in one of these last few UI3 patches, but this change to the defaults doesn't affect existing UI3 users who already have a streaming profile list saved in their browsers.

Hey, BTW, do you know of a way to cancel a queued export? I posted separately on this but no takers. I accidentally started an export on my phone via UI3 - my fat finger hit export when I was trying to click the timelapse checkbox... and it was before used the red and green trim handles to get it down to what I wanted - and so it dutifully put it in the queue - all 8 hours worth.

No. It was a while ago now, but if I didn't put a cancel button into the interface, then I'm pretty sure we didn't design a cancellation system. You might be able to cancel it from the local console by accessing the convert/export queue via this item above the clip list .

Neither player nor network delay appear to be excessive while it's happening. Though the video bit rate is all over the place. Notice also how the blue line on the stats box stops moving forward with each jerk. Does this on the phone as well.

In this case the bit rate fluctuations could be partly caused by all the motion in the video. The Stats for nerds panel mostly only updates when a video frame gets rendered, so when playback gets stalled, so do the graphs. That is definitely a place I could make improvements.

Have a look at these ping graphs. I'm getting this on every device now. Does that point to network issue you think?

No, that looks pretty normal actually. I have never figured out why a pattern appears like that but I've seen it lots of times and it isn't drastic enough to be of much concern.

 

[bp2008]

Something you can try, too, is running pingtracer on the BI machine and entering the IPs of all your cameras (comma separated) so you can monitor them in realtime and see if there's a problem on that end of the network.

 

[erkme73]

Something you can try, too, is running pingtracer on the BI machine and entering the IPs of all your cameras (comma separated) so you can monitor them in realtime and see if there's a problem on that end of the network.

That's awesome, i was going to ask if you'd considered allowing more than one at a time. Very cool. Will play with it tonight..

 

[erkme73]

This ping trace is powerful. I'm running 10/sec pings on 6 cameras for grins and am getting an occasional no response. It seems to happen to two cameras at once. Do you think this is just me saturating the ping command, or is it a hiccup that I should investigate further?

 

[bp2008]

This ping trace is powerful. I'm running 10/sec pings on 6 cameras for grins and am getting an occasional no response. It seems to happen to two cameras at once. Do you think this is just me saturating the ping command, or is it a hiccup that I should investigate further?

It could be a sign of a minor issue, but nothing I would worry about too much. I've never had great ping responses to most of my cameras. See what I mean, here's 8 of them:

 

[MrRobinHood]

Hey BP, any chance we could get arrow navigation between cams to skip forward and back between one cam and the next when one is select to be maximised please?

I tried all the arrow keys and n and p, but none seem to work.

Also, I'm not sure if this is a bug or a config issue on my end: Audio works in live view, but if I use the time line to go back in time and check something, the video loads great, but I see the muted audio icon, which doesn't seem to change when I adjust the volume.



Any ideas?

Thanks

 

[bp2008]

@MrRobinHood UI Settings > Hotkeys has "Next Camera" and "Previous Camera" bound to < and >. But those same keys are also used for Skip ahead and back 1 frame, when viewing clips. As the timeline view is sort of a hybrid, I chose to support the next and previous frame version of those hotkeys there, and just not have the camera switching hotkeys work. Its not great, but its the way it is for now.

Blue Iris doesn't support audio when using the timeline. Not even when using the local console. The BI developer is aware, but didn't have time to implement it yet.

 

[Ssayer]

Dumb question, but I'm known for those... I've searched the thread and don't seem to see anywhere anyone has even asked this... Does UI3 have a Solo Selected Camera setting like the console has?