As an update to this, I was able to narrow down the problem to "Intel HD hardware acceleration" option, inOptions -> Cameras -> Intel HD hardware acceleration (restart)
Previously this was set to "Yes (H.264)" but some days ago I set it to "No" and now BlueIris process stays around 2GB memory...