Network buffer balance

[Brett_F]

Several threads talk of increasing this setting to as high as 20MB per camera. I am unclear to the exact reasoning behind this suggestion, is it simply to avoid dropped packets / frames ?
I have tried several settings & once I increase above the default minimum of 0.5MB I get significant "lag time" between cameras when viewing live. With 4 cameras, 3x D2D with time being streamed from the cameras themselves & one camera just MPG4 encoding (XVID) with BI time overlay enabled, there is 30 second to 1 minute lag between cameras. I can go outside, wave & smile, trigger the camera, walk back in, wait 15 seconds & watch myself. Ideally I want to minimize dropped frames yet I still require accurate live stream. It is unclear if this setting is specific to systems suffering performance or packet loss issues or other tweaks with this setting that can assist in performance, but "not at any cost".
I've set it @ 1MB & that seemed reasonable yet still had 15 second lag at a minimum, so it appears it buffers all the time, not just when stressed, which to me seems counterproductive. I want it to buffer when required (stressed) but only at that time, not all the time. How are others working with this setting & when are they adjusting it, simply for stress & performance?
I regularly run "netstat -s -p tcp" (to detect any packet loss issues) on the PC running BI & am always within that 1-2% range of packet re-transmit values which suggests the network is running as optimally as expected.

 

[Rockford622]

I have set my cameras video buffer anywhere from 1 MB to 20 MB (same model of camera) and have never seen any difference in Blue Iris's performance, so I'm not really sure what it does, but I wouldn't fixate too much on that setting.

As for your lag issue, that does sound quite severe. On any of my cameras, there is at most 1/2 a second delay between observed video on my PC and real life. Is this camera you are talking about wireless?

I'm not sure what netstat will show you in terms of dropped packets (generally netstat is used for established connections), but ping x.x.x.x -t would be a better tool to determine dropped packets. Ideally your ping times on your LAN should be in the 1 to 2 ms range, and fairly consistent with no timeouts.

 

[Q™]

I set my buffer at 20MB because that’s what @fenderman told me to do and I’m good at doing what I’m told by people whom I consider an authority. There are many issues which cause lag, including accessing the BI console using a remote access utility or particular settings in the BI console.

 

[Brett_F]

My cameras (4) are all LAN PoE & that netstat command run on the PC tells me I have little to no network congestion. With the PC "NOT" dropping packets & minimal segment retransmits, tells me the LAN portion is within specs. As for PING, yes, all connections meet the 1-2 ms range (one camera the farthest averages 4ms, not unreasonable), the 4 cameras ALL wired LAN, all on a switch "below" the router so basic Ethernet layer 2 "store & forwarding" taking place, no router / firewall congestion to consider. The LAG issue really didn't present itself until I went to a D2D environment, when I was simply streaming & letting B2B do the decoding I had high CPU but no lag. So naturally one assumes D2D the issue here but a dedicated WD Purple HD formatted in 64K clusters (albeit @ 5400 RPM), you wouldn't think there is that significant an impact. I have some performance bottleneck (no hwd decodings, XEON cpus with no QuckSYnc) that I am trying to resolve. It's a headless PC, all I do is RDP into it to make changes otherwise it is all via web for viewing. I solve the CPU issue, have a network lag issue, 2 steps forward, 1 step back. I'm actually getting pretty frustrated with this software.

 

[Q™]

RDP from the local LAN, or from the Cloud?

 

[Brett_F]

Local LAN, Intel NIC (on PC running BI), receive & transmit buffers 2048. I've tweaked everything I can think of. Switch is a DLINK DGS-10008P (reviews seem to suggest it is sufficient), Amcrest cameras IP2M-842's & IP4M-1026, firmware up to date on everything. PC-switch GB speed, cameras are all PoE 100MB 1280x720 15 fps & 15 frame interval.

 

[fenderman]

My cameras (4) are all LAN PoE & that netstat command run on the PC tells me I have little to no network congestion. With the PC "NOT" dropping packets & minimal segment retransmits, tells me the LAN portion is within specs. As for PING, yes, all connections meet the 1-2 ms range (one camera the farthest averages 4ms, not unreasonable), the 4 cameras ALL wired LAN, all on a switch "below" the router so basic Ethernet layer 2 "store & forwarding" taking place, no router / firewall congestion to consider. The LAG issue really didn't present itself until I went to a D2D environment, when I was simply streaming & letting B2B do the decoding I had high CPU but no lag. So naturally one assumes D2D the issue here but a dedicated WD Purple HD formatted in 64K clusters (albeit @ 5400 RPM), you wouldn't think there is that significant an impact. I have some performance bottleneck (no hwd decodings, XEON cpus with no QuckSYnc) that I am trying to resolve. It's a headless PC, all I do is RDP into it to make changes otherwise it is all via web for viewing. I solve the CPU issue, have a network lag issue, 2 steps forward, 1 step back. I'm actually getting pretty frustrated with this software.

It's not the software..it's user error.. otherwise we all would have that lag right?
Let's start with your network setup and then replacing your crappy old powehog server.. connect PC to the same switch as the cameraa not to the router.

 

[Brett_F]

Yes, I thought I made that clear PC (running GB) & all 4 cameras on same switch, router is "out of play" (ASUS RT-68U) up a level so one thinks it should be performing as anticipated.
Re: " crappy old powehog server " .. you mean the PC? It is 4 yrs old, 6 core XEON, 24GB RAM, it's no speed demon but realistically it should manage this (with exception it is one of the Intel CPUs that does not have Quick Sync), it is a Precision workstation. SSD Boot drive, BI runs off the WD Purple HD. I even have a RAM drive for BI Temp folder to reside.

 

[fenderman]

Yes, I thought I made that clear PC (running GB) & all 4 cameras on same switch, router is "out of play" (ASUS RT-68U) up a level so one thinks it should be performing as anticipated.
Re: " crappy old powehog server " .. you mean the PC? It is 4 yrs old, 6 core XEON, 24GB RAM, it's no speed demon but realistically it should manage this (with exception it is one of the Intel CPUs that does not have Quick Sync), it is a Precision workstation. SSD Boot drive, BI runs off the WD Purple HD. I even have a RAM drive for BI Temp folder to reside.

What switch?

 

[Brett_F]

Switch is a DLINK DGS-10008P (reviews seem to suggest it is sufficient), I was wondering that as well, but the lack of packet loss suggests it was holding up.

 

[SouthernYankee]

Could the delay be caused by the FPS and I frame setting ?
Are you looking at only a single camera or a screen with multiple cameras ?

I would suggest putting a head on the server and see if you still have the problem, by bypassing the RDP

 

[fenderman]

Switch is a DLINK DGS-10008P (reviews seem to suggest it is sufficient), I was wondering that as well, but the lack of packet loss suggests it was holding up.

Do you notice any ill affects leaving the buffer at the lowest? What is the blue iris temp folder you are storing on the "ram drive"? how did you setup the "Ram drive"..I would suggest simply writing to the storage disk and keeping BI and the database on the ssd. What bitrates are you using for the cams?

 

[Brett_F]

Ill effects of the buffer at it's lowest is choppy recording / playback. "Appears" less smooth, higher CPU but I have to run it for a few days ideally to really see the effects. The TEMP folder is just the BI baseline TEMP folder variable set up in "Other" in options section, I assume when it is building the larger files for archiving etc. & it is the AMD RAM Drive software, quite stable. I left BI off the SSD because there is a write & shelf life to SSD's & I was thinking to limit the BI use for that reason. The bitrate for all cameras is max @ 2048 & web play back same bitrate. I found that 1MB buffer for 264B cameras (3 ea 1280x720) & 2MB buffer for the one MPG2 (XVID encoding 4x3) is the sweet spot.

 

[Brett_F]

I could lower FPS & I frame rate but am not happy with the results of anything lower than 15. Trying to find the right trade off of quality versus performance. We have a lot of moving traffic & I want to capture that effectively. I did put a monitor on the unit & watch for a spell, didn't seem to make a significant difference. I will run this way for a few days & try not to reboot or make changes & keep it running latest 4.7.6.9 (July 15 / 2018) build.

 

[fenderman]

Ill effects of the buffer at it's lowest is choppy recording / playback. "Appears" less smooth, higher CPU but I have to run it for a few days ideally to really see the effects. The TEMP folder is just the BI baseline TEMP folder variable set up in "Other" in options section, I assume when it is building the larger files for archiving etc. & it is the AMD RAM Drive software, quite stable. I left BI off the SSD because there is a write & shelf life to SSD's & I was thinking to limit the BI use for that reason. The bitrate for all cameras is max @ 2048 & web play back same bitrate. I found that 1MB buffer for 264B cameras (3 ea 1280x720) & 2MB buffer for the one MPG2 (XVID encoding 4x3) is the sweet spot.

put the temp folder back on the ssd without and ram drive software and see what happens.

 

[Rockford622]

Did you try and connect directly to the cameras using VLC as a streaming player? Perhaps it's not Blue Iris introducing the lag, but the cameras themselves.

 

[Brett_F]

Though I doubted a significant difference, switching TEMP folder locations had no effect (I benchmarked both ways to give it a fair evaluation), it is back on the RAM drive which does assist in performance overall & will extend the life of the SSD. Too many small writes to the SSD via BI, these SSDs do have a write shelf life so I'd prefer to not use small writes to the SSD & leverage the RAM drive instead, better performance & no write count hit.
If I connect directly to the cameras, the live feeds are fine, the more I increase the buffer size in BI, the more lag. I haven't written code in a long time but it appears that BI only flushes the buffer periodically (full or some other value "if else") so when the CPU is low use, it isn't committing & just waiting for some trigger to flush & commit.
I've set it all to 0.5MB again (that seems to have the least lag so clearly it's not the network) & will look to tinker with QOS & the transmit / receive packet buffers on the NIC of the PC itself to tweak the network performance.
I'll assume if I went to 1080P / 264H stream & more FPS (higher than current 15 fps & 1280x720) then perhaps this buffer value comes into play & can assist as obviously the buffer would fill up that much more quickly & flush.
I've used this software in limited fashion for a few years & never really cared much for performance as I didn't have HD cameras & wasn't really looking to do more. Now with 3/4 cameras capable of HD I have begun to tinker with the software.I think there is a subset of us here that run headless, do not have monitors attached & the interface is always accessed remotely, & monitored via web interface. This software is more designed to use Intel QuickSync in a head (with monitor in front) state & so RAM, GPU & other potential variables cannot contribute to the overall success of this software.

 

[fenderman]

Though I doubted a significant difference, switching TEMP folder locations had no effect (I benchmarked both ways to give it a fair evaluation), it is back on the RAM drive which does assist in performance overall & will extend the life of the SSD. Too many small writes to the SSD via BI, these SSDs do have a write shelf life so I'd prefer to not use small writes to the SSD & leverage the RAM drive instead, better performance & no write count hit.
If I connect directly to the cameras, the live feeds are fine, the more I increase the buffer size in BI, the more lag. I haven't written code in a long time but it appears that BI only flushes the buffer periodically (full or some other value "if else") so when the CPU is low use, it isn't committing & just waiting for some trigger to flush & commit.
I've set it all to 0.5MB again (that seems to have the least lag so clearly it's not the network) & will look to tinker with QOS & the transmit / receive packet buffers on the NIC of the PC itself to tweak the network performance.
I'll assume if I went to 1080P / 264H stream & more FPS (higher than current 15 fps & 1280x720) then perhaps this buffer value comes into play & can assist as obviously the buffer would fill up that much more quickly & flush.
I've used this software in limited fashion for a few years & never really cared much for performance as I didn't have HD cameras & wasn't really looking to do more. Now with 3/4 cameras capable of HD I have begun to tinker with the software.I think there is a subset of us here that run headless, do not have monitors attached & the interface is always accessed remotely, & monitored via web interface. This software is more designed to use Intel QuickSync in a head (with monitor in front) state & so RAM, GPU & other potential variables cannot contribute to the overall success of this software.

It has nothing to do with the quicksync or that its headless...there is a bottleneck somewhere in your setup...
how are you viewing the headless server? for testing you should be viewing on a local monitor. If the delay is via your remote viewing software then that would explain it.

 

[Rockford622]

How about trying Blue Iris on a different PC? Just throw it on a laptop or whatever else you might have and see what happens.

I have used Blue Iris for years, with and without a monitor, and never seen the type of delay you are talking about. Seems like it's something with that PC and it's having a real hard time decoding video.

The point of a buffer is not to delay playing video until the buffer is full. No matter the size of the buffer, it should not introduce a delay in your video.

 

[SouthernYankee]

Put a head on the PC and view it there.