Might be better then to purchase an NVR dedicated to those smart motion detect features
Right, I'm running all those functions directly on the camera rather than on a NVR. That probably doesn't help the camera load.
I just noticed I had left "Motion Detection" on in the camera and turned it off. Let's see how this plays out, but at least the ping replies went from ~300 ms every second ping to consistently 4-8 ms like it should be. When ping replies take so long that they did previously, it tells me that the camera is up to such heavy processing that the IP stack – and probably other applications – lags behind. At least if you know that your network connections are okay.