The D20XB is pushing data over RTSP rather its generic 34567 port - fairly sure the 800 series has the same port available. I can replicate the same issue when i force my 601-D20X onto the RTSP port.
This is the result when using RTSP... (it gets far worse than this)
Yeah, not pretty right? This is the settings from what you see above...
This is with the in-built profile that BI provides, not using RTSP... but port 34567.
The issue is not limited to
BlueIris, but the camera series its self. The sunba cameras have some "issues" with RTSP streams. If you load this up on VideoLan, parallel with BI, you can observe the camera having noticable /severe issues with the RTSP protocol: rtsp://YOUR.LAN.IP.HERE/user=admin_password=tlJwpbo6_channel=1_stream=0.sdp?real_stream
---
Edit: Also, set your cameras encode config on Primary/Main to CBR rather VBR.