@bp2008 I've got a problem that's officially got me stumped. I too am getting an odd 503 error at seemingly random intervals when accessing UI3. I can log in fine and the cameras visually appear fine, but every so often (fairly frequently) I get a toast message with a 503 json error. When I access UI3 through http://local_ip: port it doesn't seem to happen, but when accessing via dns name (internally or externally) it does. Now the only difference between these 2 'routes' internally is that the dns-name route goes through my Apache reverse proxy server. However, I don't have this issue on any other sites/services that go through the same reverse proxy server, of which I have quite a few...and my reverse proxy config is pretty simplistic.
The Chrome console reports the following 3 lines of errors when this happens:
POST
503 libs-ui3.js?v=105-5.2.1.3:5
Response: 503 error ui3.js?v=105-5.2.1.3:19150
Status: error
Error:
URL: json
error toast: An error occurred loading the camera list.<br>Response: 503 error<br>Status: error<br>Error: <br>URL: json ui3.js?v=105-5.2.1.3:19037
And the following is my Apache reverse proxy virtualhost config:
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName url.redacted.com
ServerAdmin webmaster@localhost
Header set X-Frame-Options "SAMEORIGIN"
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
Header set X-XSS-Protection "1; mode=block"
Header set X-Content-Type-Options "nosniff"
Header set Referrer-Policy "same-origin"
Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure;SameSite=Strict
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/url.redacted.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/url.redacted.com/privkey.pem
ProxyRequests Off
ProxyPreserveHost On
<Location />
ProxyPass http://internal_ip: port/
ProxyPassReverse http://internal_ip: port/
</Location>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
</IfModule>
So far I have tried rebooting everything, making sure BI is fully up to date, adjusting the BI VM's specs to make sure it has plenty of resources (CPU & RAM always below ~40/50%), adjusting the streaming/quality settings in BI > Web Server > Advanced, removing all of the 'header set' commands in my virtual host config, etc. I realize this may very well not be a UI3 issue, but I am hoping that maybe someone has dealt with a similar issue before and might have some thoughts on what I could do to working around it. Thank you!
EDIT: Ok, now I'm even more confused. I know just enough to be dangerous with Chrome's developer
tools, and in the network tab I'm seeing
MANY successful JSON POST requests (including probably ~90% of the 'camlist' post requests), but then randomly it'll fail. I've attached pictures of 2 different JSON POST requests, one right after another, both to the same URL and everything. However, the one that succeeds sees the server as "BlueServer" (presumably BI) whereas the one that fails sees "Apache" (my reverse proxy). I'm starting to think this isn't as "random" as I initially thought because when it does happen it's always this 'camera list' command that fails. whjereas everything else seems to succeed.
EDIT 2: Final edit I'll make for now until someone else has any ideas, but here's some more weirdness I'm seeing attached in screenshot #3. At the top, every other 'json' line is a successful 'camlist' POST request. Interestingly, the 'camlist' POST request that fails ALWAYS seems to be immediately before what the data:image request is doing. I'm not sure if there's anything to this or not, but I can reliably replicate this by refreshing the page.
