I did try auto fix with no luck.Yes, and I could totally this compatibility setting getting assigned if Blue Iris was to crash on startup and someone was to blindly let windows try to auto-fix it. Windows' auto-compatibility-fix thing has never helped me.
try uninstalling your nvidia drivers
see if BI will open with normal windows drivers
If it does, then try a different nvidia driver
Oh, also, you have to open the Nvidia control panel while your color depth is low (i.e. with Blue Iris running) or else of course it will show 32 bit color.
Maybe you should check the Compatibility tab of the Properties one more time, and also go directly to blueiris.exe and look at the compatibility tab there too. Though on may system the shortcut and the exe both share the same compatibility options.
Did you make any direct registry edits trying to fix this?