@jcummins - Close to the lens, bright IR.. anything will appear very bright and large enough to trip up your motion detection. Bugs, dust particles, water droplets like mist/fog. In the cold, tiny snowflakes as well.
Adjustments like alastair has suggested, secondary lighting, or using...