change the NIC Type on the camera from Auto to 100M Full-dup (or whatever is the highest full duplex you can select).
That does suggest that you have a cabling problem that is causing the auto-negotiation to periodically re-try.
There is no reason for the auto-negotiation to kick in again after initial connection unless there is a signal loss.
If your switch / router has the facility, check out the error counts on the port to see if they are zero.
If not, see how much variability in response times you see with a continuous ping. Though it has to be pretty bad before it shows up that way.