How to capture clear frames of fast moving objects?

bimbombum

n3wb
Nov 15, 2019
2
1
USA
I am working on a computer vision use case for which I need to capture images of boxes that are moving quite fast and then identify them. Boxes have text and images on them so for my computer vision model to identify them the frames that I capture from the video need to be crystal clear. Right now the moving object is quite blurry in each frame. I have set the resolution to 3840x2160, 20 fps, 1/250 shutter speed. If I take the shutter speed to 1/500 or more the feed becomes very dark. Can anyone please recommend what settings I should use?
I am using Hikvision DS-2CD2085G1-I for this.
 
  • Like
Reactions: xtropodx
Hi @bimbombum, no expert in HIK here, but there are scientific processes with IPC's (and photography in general) which are shareable across different brands.

You gave a lot of information in your first post, but there is a very important factor missing. And that's lightning. (and your iframe interval in less matter).

Lightning is (in my opinion) the most difficult parameter to mess with. Especially if it is not doing what you want it to do :-) I'm assuming you are working in low light situations.

Let me give an example: if your shots are blurry, either the object moves too fast for 20fps, or your shutter is not "fast" enough. Slow shutter means that more light is coming through the lens, so you're quick entering the "overlighted" scenery. Good to detect (for example) IR reflection on license plates, however the minor speed increases increase blurriness. So higher shutter speed is the way to go, but then your images become ... darker. Hence my assumption you are working in low light situations.

So what can you do about it? Install some additional lights (eg ir if you don't want to blast everybody blind). Starlight camera's (from dahua) work extremely well in low light, even in pitch dark (and only 1 ir) I can fetch license plates from passing cars.

Now I also mentioned iframes - this is something often forgotten: the higher (than your fps that is) the iFrame interval is, the more "optimizing" with delta frames happens between the different iFrames. Advice: put ftp = iFrame interval

Good luck!
CC
 
change it to 2 or 4 mp ultra low light, and set the shutter speed around 1000 for high speed. add very bright LED light too. the higher the shutter speed less light going into the sensor. the darker the picture.
 
Beg or borrow a camera capable of recording 2MP at 60fps to test. Lighting, shutter speed and other factors do come into play but for the sharpest individual frames you will need to shoot at 60fps - as demonstrated here (use the pause button to check).

 
  • Like
Reactions: djernie
Beg or borrow a camera capable of recording 2MP at 60fps to test. Lighting, shutter speed and other factors do come into play but for the sharpest individual frames you will need to shoot at 60fps - as demonstrated here (use the pause button to check).


What a good test result to see ipOsX! I would like to understand the technicalities better. Would you please take the time to help me understand why, if the shutter speed etc is exactly the same at 24fps as at 60fps, the clarity is different. Would the clarity be identical in each case if you viewed a single frame instead of pausing the video?
 
What a good test result to see ipOsX! I would like to understand the technicalities better. Would you please take the time to help me understand why, if the shutter speed etc is exactly the same at 24fps as at 60fps, the clarity is different. Would the clarity be identical in each case if you viewed a single frame instead of pausing the video?

There is more to it than simple fps (eg iFrames). But do the math how much storage you'd need at 60fps. I'd mess around with the "easy-to-fix" parameters first (eg adequate ambient lightning with shutter speed) and then push up the fps.
 
It's complicated. If taking a still image, shutter speed control would give you all the clarity you want. But the perception of moving images is also affected by frame rate, the number of key frames or I-frames and P-frames, the refresh rate of the screen you are viewing them on and the conditioning of the human brain. And with a stationary camera, as most CCTV cams are, the speed that the subject moves across the frame is a major complication.

Whatever frame rate and however many key frames you set for your video, your monitor will probably show you 60 images per second since most monitors have a 60Hz refresh rate (or 50 images per second if the refresh rate is 50Hz). If the video doesn't contain 60 (or 50) frames per second then the monitor will duplicate frames to make up the total. This creates flicker and a degree of blur depending on the maths involved. So the optimal video diet for a clean viewing experience is 60fps (or 50fps).

Next you have to compute the optimal frame rate for the subject matter. If it is a stationary subject, 1fps is plenty to capture clean images. If it is moving only slowly, maybe 5-10fps would deliver clean images. But if the subject is moving quickly (which in the movie industry means it crosses the frame in less than 7 seconds) then you will need to boost the frame rate to capture it clearly. That's because there are so many pixels changing from frame to frame. If you don't capture in the optimal 60fps (or faster) then your software will compensate for the missing data by interpolating new frames in between the existing frames resulting in a smeary or motion blur effect - which is often wanted by movie makers but is unhelpful for CCTV. Traditionally, 24fps has been the optimal rate for shooting movies because motion blur helps to deliver realism, ie that's what the naked eye sees. But it's the last thing you want in CCTV footage. It's apparently the last thing that video game players want too since their on-screen action comes fast and furious. So most games players demand 60fps for clarity and no motion blur to jolt them out of their virtual world. Some recent animation movies were shot in 48fps and higher for the crisp, clean imagery it made possible.

The final piece of the jigsaw is video compression. Whatever codec is used to compress video - typically H264 or H265 - it does its job by selectively losing data. It splits video up into frames which are very similar to the preceding frame and frames which contain a lot of pixel changes and it produces video which is made up of a number of accurate or pure frames, called I-frames plus a greater number of predicted or interpolated frames, called P-frames. When the video subject is moving quickly, the amount of interpolation is greater and the quality of the P-frames is lower. It's the only way the video can be forced to fit onto a memory card or to play smoothly over a network. But by increasing the frame rate to 60fps, the number of I-frames can be increased and the number of interpolated frames reduced, resulting in a cleaner, clearer video with much sharper individual frames.

You can test this out for yourself by playing some 30fps and 60fps videos on Youtube and pausing them to compare the clarity of the individual frames - assuming you have a computer which is powerful enough to play 60fps. Look for 1080p videos to compare as 4k videos won't play at 60fps on most PCs.

Here's something to read: 5 reasons for 60 fps - Paul Bakaus' blog

What a good test result to see ipOsX! I would like to understand the technicalities better. Would you please take the time to help me understand why, if the shutter speed etc is exactly the same at 24fps as at 60fps, the clarity is different. Would the clarity be identical in each case if you viewed a single frame instead of pausing the video?
 
I was playing around with this. My front yard camera (which happens to also catch the street in front of my house) is a 5321R-Z which is capable of 60 FPS and multiple 1080P streams. I set the main stream to 60 FPS, the 2nd sub-stream to 7 FPS and had Blue Iris to record both.

Side-by-side comparison. Super easy to tell the 60 FPS clip from the 7 FPS clip!




Raw 60FPS export (~19.8 MB):


Raw 7FPS export (~9.3 MB)





Below are two frames captured (via VLC) from the raw exports... one came from the raw 60 FPS export, the other from the raw7 FPS export. Can you spot enough difference in quality to tell which frame came from which export?

vlcsnap-2019-11-18-15h00m36s921.png


vlcsnap-2019-11-18-15h02m41s965.png
 
Last edited:
ipOsX, a big thank you for your detailed assessment - with so many variables it is a lot more complicated than I had realised. I need more time to absorb the details. However, given the need to limit the cpu load and storage space in a typical CCTV system, underlying question will be to determine which compromises deliver the best result. For example, picking up on what catcamstar said, if I were able to use 60fps and high iframe rate with one of my 2MP cameras, I imagine I would have to chuck my other 9 cameras away.

I‘ve also tried to look carefully at what arisotobrat has posted and yes, I can see a difference between the raw 60fps and 7fps videos. However, in this case, doesn’t the difference simply come down to the difference in the number of bytes of data rather than the frame rate? I wonder what the difference in video quality would have been if he had changed the bit rate in one of them to make the file size equal? This view is reenforced by the fact that even the static images in the videos are clearer in the 60fps video.

And there again, my reasoning is probably undermined by aristobrats two snapshots - they both look identical quality to me - I don’t think it’s limited by what I’m viewing them on. I don’t understand it!

Anyway, thanks again for the excellent food for thought - I’ll look out for more!
 
  • Like
Reactions: ipOsX
Good job, @aristobrat - and nice camera! It's not the best illustration though because the moving vehicles are a small part of the image and at a size where it isn't easy to distinguish the differences. Had they been passing close to the camera - as presumably the OP's boxes are - then the differences would have been more discernible.

Nonetheless, when magnified, the 7fps cars are significantly more noisy, grainy and pixellated than the 60fps cars - to the point where it might be possible to read words on the side of the 60fps cars but not on the 7fps cars. I have attached a couple of blow-ups to illustrate those differences. Of course, Youtube also introduces artefacts which compound the problem on all videos.

The extra artefacts on the 7fps vehicles are simply because the comparison frame doesn't actually exist - it has been invented or interpolated by the codec based on the preceding and following actual frames. At 60fps, the frame is either a real frame or it is based on spatially closer real frames and so requires less interpolation and therefore generates fewer artefacts.

If the area of movement had been much larger within the frame then the differences would have been more obvious since there would have been a large area full of artefacts. Whether it makes sufficient difference for the OP to be able to read text which he can't currently read, perhaps his tests will tell.

To get these screengrab images I opened your 1080p videos at full screen on my iMac which is currently 2560x1440, thus oversizing your 1920x1080 video.

(Btw, I haven't further compressed these images, eg into JPEGs, as that would have introduced yet more artefacts.)

blue 7vs60fps.pngblack 7vs60fps.png
 
Dave, yes, if you are doing continuous recording, 60fps is going to hurt. Youtube say that 1080p video at 60fps eats 3.04GB per hour. But recording 4K video at 30fps hurts a lot more. If recording continuously I would want to look at cameras which only switch into max fps when they detect movement.

aristobrat's snapshots are both pretty noisy on the car. Neither is as clean as the 60fps video grab of the same car that I posted above although they are less noisy than the 7fps shot I posted. His two snapshots are not of exactly the same frame which could make a difference; and the artefacts will be greater the further away the chosen frame is from a real frame, ie an I-frame. Again, to really see what's going on we would need to see the car filling more of the frame.
 
Again, to really see what's going on we would need to see the car filling more of the frame.
I should be able to do that in the next day or two.
 
  • Like
Reactions: ipOsX
Yes ipOsX, but assuming the result is better at 60fps, I will struggle to understand why. Presumably, the time taken by the camera to digitise the image is fixed regardless of the frame rate and it’s only captured once per frame. Aristobrat’s camera will have a ceiling on it’s encoder speed and bit rate. So, when there’s a vast change in the image from frame to frame, won’t it have say 10x more time to to a better job when using 7fps and therefore stream much more valid data?

Whilst accepting the fact that there needs to be a certain amount of change in a pframe image before it’s used, I am still not quite convinced that artefacts are the principal problem and it’s not instead primarily influenced by the difference in the streaming rate/file size from the camera at 7fps and 60fps in aristobrats’s video comparison (even though with a simple calculation it would appear the bits per frame are already a lot more at 7fps... my level of understanding has a long way to go).
 
Dave, even at basic level, 7fps could miss the critical moment when a moving box passes in front of the camera with its text legible. In the OP's case, that should be enough reason to test 60fps. There will still be some clean frames in the 7fps feed but they might not be the frames you want when the subject is moving fast. This applies to fast-moving thieves too of course. I'm assuming that @aristobrat optimised his bitrate for both illustrations but perhaps he will tell us. The screen grabs I posted above prove that there is more noise in the cars at 7fps than 60fps which wouldn't be explained by the bitrate setting; but had I chosen a different frame, there might have been no difference.
 
I'm assuming that @aristobrat optimised his bitrate for both illustrations but perhaps he will tell us.
This is what I used. Let me know any suggested changes and I'll use them on my next test.

Screen Shot 2019-11-19 at 10.11.04 AM.png
 
ipOsX, please don’t label me as being argumentative, compared with you I am at the bottom end of the scale and feeling my way. I accept your point that 7fps may miss the crucial instant when an intruder turns his head towards the camera. But this post is about capturing clear frames and motion blur. And why would there be more noise at 7fps? If the encoder didn’t bother to update small pframe image changes, wouldn’t the result show a smear - ie less noise?

Another thing, it’s just occurred to me that although aristobrat‘s camera is triple streaming, won’t it only have one processor that encodes each stream sequentially? If so, perhaps it doesn’t have more time to create a better result. What’s more, perhaps also the lion‘s share is biased towards the main stream. Looking at one of my Dahuas, I see that the maximum frame rate setting for sub stream 2 is only half that of the main stream. So, I’m already guessing the result of aristobrats’s next test... Unfortunately, maybe a fully definitive test would need two cameras side by side.