All
I've been asked about these 3 areas many times and even more so lately, so thought I would create a post (started small but you know me, I then decided I needed to add more information).
Therefore with that said, I wanted to share my input, perspective and recommendations here on these topics. I've broken this down into sections and if people want more technical then just DM me, don’t want to bore everyone. Apologies for the longer post.
I-Frames & Frames Within A Stream in General
Breaking down a stream, you have 3 types of frames (I, P, B). These become important when you’re talking about video compression. I’ve shared codec usage, make-up, how they work and my recommendations in the past so won’t rehash that here. The key though is to understand what frame plays what role. I-Frames (Intra Frames) are Key Frames (for those that have been around video for a long time), consist of macro blocks and are focused on intra prediction (basically focusing on matching blocks only within the the same frame). They do not offer temporal prediction which is limited to P-Frames (Predicted Frames referring to previously encoded images) that can also be used for spatial prediction too. B-Frames (Bi-directional frames can reference frames that take place before or after them). Important to point out that the H.264 codec does use macro block based compression (if people weren’t aware or hadn’t seen the previous posts)
I-Frames are used as reference points compared to other frames in the stream and generally mark the end of the GOP (group of pictures) or segmented video portion. Their primary impact (most peoples first thought) is to bandwidth, secondary is picture quality (this is of more importance to me with CCTV operations and deployments). The simplest way to look at these are, lower I-Frame interval (closer to 1 in your GUI for example), the more frequent the I-Frames and the higher the image quality (subjective to some BUT is a fact) but you also increase bandwidth requirements. Higher the I-Frame interval (further away from 1 for example), lower the frequency of I-Frames and image quality will be lower as will bandwidth. This however is also dependent on camera, codec and most importantly the subject or target. If you have fast moving objects you will almost always benefit from more I-Frames regardless of impact to storage / bandwidth (which as I mention below, you can plan for proactively).
The reason for matching your I-Frame interval to FPS (or if you see it as increasing I-Frames in general) is that you increase the opportunity for better quality frames within your capture as you are setting a ‘full image capture’ for every frame you are recording. Remember that I-Frames do not rely on previously encoded frames, only working within the same frame, this is what allows them to improve and maintain overall stream quality. The encoder you’re using (with associated codec) then utilizes I-Frames as reference (as I mentioned above) when encoding the Predicted (P-Frame) and Bi-drectional (B-Frame). As mentioned many times before, this DOES increase your bandwidth used BUT isn’t necessarily catastrophic (multi cam deployments of course do add up) for most well dialed in systems where you are focused on a crisp IVS rule (OR AI if you’re using that) critically only capturing real targets for a set period of time within the FOV. Also we have to remember as I and others have stated on many occasions, storage is ‘relatively’ cheap and certainly no one should be in a situation where they are looking to save on storage space at the expense of securing ID related captures. This is a critical part of CCTV deployment planning, live and historic retention of captured footage, and should be figured into your project plan whether it’s residential, commercial or even government related.
As I eluded to above, this is even more impactful in busy, high traffic FOV’s where artifacting can increase when using less I-Frames and can be incredibly important for special application work (ALPR/Face Recognition etc). In the case of ALPR/ANPR and facial recognition for example, matching (or exceeding in some cases) the number of I-frames (to FPS) on cameras in that environment can be highly beneficial to overall image detail in the few frames you have with critical recognition data in them.
Another area that people often don’t cover is the other added benefit of I-Frames and why having these more prevalent in your stream can assist if/when you hit a corruption event. Remember that I-Frames hold the key to corruption recovery too since they are based on image quality (full image) and not dependent on previous encoded images. Therefore, when a Predicted (P-Frame) or Bi-directional (B-Frame) frame is corrupted it benefits from the the next I-Frame to recover from the corruption by re-encoding or decoding and benefits greatly when frames beyond the I-Frame (used as the re-encode / decode point) exist.
Lastly in terms of scanning forward or backward in a video, I would be remiss if I didn’t mention that as well. Remember that I-Frames are in fact a Key Frame and therefore play a part in where a video will seek to when moving forward and backward. The longer the I-Frame interval (or Key Frame) placement the longer the seek point will be in the stream when moving forward and backward. Not the most important part of I-Frame usage but mentioning it anyway.
So, can you use less I-Frames than your FPS, YES, should you, my recommendation is NO. For most situations, rather than running the risk of less quality in each frame captured (again, subjective BUT a fact), especially in high traffic FOV’s or special applications, simply plan your storage correctly as part of your deployment and have a system that can adapt to the changing targets in FOV. Does that mean everyone should follow this rule, absolutely NOT. I advocate for what I feel works best in my experience across many large and small deployments but hey, many people run strange configs around FPS even based on a specific need or in some cases misunderstanding but thats down to individual needs, preference, understanding and deployment scenario. Also don’t forget that some NVR SW platforms also require FPS and I-Frame interval to match.
CBR vs VBR
Now, continuing on from ‘image quality’ of I-Frames vs bandwidth / storage impact, I’ll now cover CBR vs VBR. I’ll make this one super short. CBR or Constant Bit Rate is what I would and do advise 99.9% of the time. Variable Bit Rate (VBR) encoding, although ‘good’ in theory (and has improved over time) is NOT well implemented in most CCTV solutions (even pro level and cinema cameras have issues here). A big part of the problem is what I call ‘spin-up time’ to detect motion changes and adjust encoding algorithms for the motion seen within the FOV. With the right algorithm and processing power, you have options, without it, you unfortunately see issues. VBR takes longer to encode, can place more strain on the camera / NVR / NVR platform and absolutely can make or break a critical capture moment. Due to the above, I operate and advocate for the fact that it is never worth taking a chance that your required ID frame(s) is ruined by a slow processing algorithm on cam and associated codec, therefore set CBR (granted to your desired bitrate appropriate to your target requirements) and forget about it.
Shutter Speed
Now onto another critical area, that of Shutter Speed. On a well dialed in (and not cheap security or webcam style) camera, Shutter Speed ABSOLUTELY, 100% makes a difference and IS required and effective for freezing action. Your setting for shutter speed of course is determined by your target acquisition requirements or special project applications. For example, as I’ve shared for years, 1/30 is the lowest I would advise to go for human targets at slower speed, 1/60 to 1/100 minimum for more moderate targets (think humans walking quicker, light jog) and then 1/250 is the minimum for higher speed targets such as vehicles (NOT ALPR/ANPR). I recommend these to ensure motion blur free caps as much as possible.
Back to the special applications that I mentioned and ALPR is one of those where much high shutter speed IS needed to ensure crisp plate capture for manual or automated LPR work (outside of barrier based, entry chokepoints etc). In a number of cases based on height, angle, distance to target, plate within FOV, you may be able to get away with 1/500 BUT on faster moving targets and if the other dimensions at install location are not 100% on point as well (height, distance, angle) then 1/1000 or 1/2000 would be my recommended minimum. Staying on the special applications train for a second, this also depends on camera being used. While all of the settings I mention above are valid across the board, when you add into the mix special traffic cameras such as the ITC’s I’ve mentioned and reviewed, they up the ante on their abilities to capture in more harsh environments / trickier installs. This is due to not only the usual mix of traffic specific algorithms, optics and SOC pairings BUT critically in the way they process a target within FOV. I’ve spoken at length before about how Video Metadata / AI cameras are looking for a target that matches VEHICLE and only then will process a plate whereas traffic cameras do NOT need a full vehicle recognition to process for a plate directly and are amazingly accurate even at lower confidence levels. This gives you latitude in your install location and in your configuration of the camera itself. These traffic cameras also have front and rear light compensation for plates (as I’ve demonstrated before) and therefore also give further options in your configuration, improved over standard AG / light adjustment or SMART IR would on ‘standard’ (non-traffic) security cameras.
Another ‘special application’ is for face recognition, an increasingly popular deployment requirement that I and others are called in for as part of a project. This too benefits from shutter speed adjustments. You want as clear an image of facial features as possible and often times if this is NOT in a controllable environment (I.e. not an entrance with biometrics and facial ID right in front of you) then dialing in your camera, adjusting shutter speed higher, will assist here too (note: within reason and does have a point of diminishing returns for facial recognition). The goal being to reduce as much motion blur as you can (along with other factors like limiting noise reduction that crushes detail etc etc) to ensure you a) capture recognizable features clearly first time and/or b) allow for a high confidence match against your known vs unknown databases.
On the flip side of this, often times people are tempted to go with low shutter speed to improve low light situations, which it can BUT at the detriment to image quality due to horrendous blur/motion smear (remember, NEVER use AUTO). In this case ADD MORE LIGHT to the FOV, whether on-camera (not my first choice and wouldn’t advocate for this except when absolutely necessary even on newer cams) or off-camera. Continuing on the subject of lighting for a second. Lighting should also be constant wherever possible to remove the ‘bloom’ effect you can get of motion activated lights coming on and killing your image as SmartIR tries to adjust etc.
The key takeaway for people is that shutter speed DOES make a difference and does work as intended on these cameras. If you are not seeing any difference between a 1/30 and a 1/1000 shutter speed or 1/100 to 1/2000 then either something is wrong with your camera OR you need to double check your overall configuration. As I’ve mentioned here and before, critically dialing in your camera for your location, desired FOV, lighting on scene and most importantly the target requirements is an absolute must. Don’t ruin your captures by not dialing in correctly. On that note, this is why WDR and other backlight options should also be used VERY carefully as they will also introduce blur. Similarly as I mentioned above, keep NR under control, super low in the day (around 30-35 max) and no more than 40 at night. If using for LPR/NPR then you can even drop below 30 at night BUT I would never go less than 25 (this still allows the ALPR/ANPR traffic cams to work well) AND this will depend highly on your install location, lighting on scene etc. None of these cams are OR should be looked at as plug n play, they’re not as we’ve discussed many times. Dialing in your cams which includes shutter speed (remember to compensate for light impact) will take your useful captures to the next level.
Closing Thoughts
In summary, the important thing to remember is that you don’t / can’t control targets in your location BUT you do control how your cams are configured within that deployment and that is critical. You never know when someone will run up to an entrance way faster than anticipated or speed past an area at an angle that limits time in FOV. So, do you look to save bandwidth on the assumption that you may capture some of those events with critical frames OR do you set yourself up for success (as best you can) by dialing in your config to ensure your hit rate for ID frames improves and that short of a crazy event (like camera falling off of its install location), you’re covered. I’ll take and advocate for the latter and setting you up for success any day of the week in this business. So keep the above in mind, hope it helps
WC
I've been asked about these 3 areas many times and even more so lately, so thought I would create a post (started small but you know me, I then decided I needed to add more information).
Therefore with that said, I wanted to share my input, perspective and recommendations here on these topics. I've broken this down into sections and if people want more technical then just DM me, don’t want to bore everyone. Apologies for the longer post.
I-Frames & Frames Within A Stream in General
Breaking down a stream, you have 3 types of frames (I, P, B). These become important when you’re talking about video compression. I’ve shared codec usage, make-up, how they work and my recommendations in the past so won’t rehash that here. The key though is to understand what frame plays what role. I-Frames (Intra Frames) are Key Frames (for those that have been around video for a long time), consist of macro blocks and are focused on intra prediction (basically focusing on matching blocks only within the the same frame). They do not offer temporal prediction which is limited to P-Frames (Predicted Frames referring to previously encoded images) that can also be used for spatial prediction too. B-Frames (Bi-directional frames can reference frames that take place before or after them). Important to point out that the H.264 codec does use macro block based compression (if people weren’t aware or hadn’t seen the previous posts)
I-Frames are used as reference points compared to other frames in the stream and generally mark the end of the GOP (group of pictures) or segmented video portion. Their primary impact (most peoples first thought) is to bandwidth, secondary is picture quality (this is of more importance to me with CCTV operations and deployments). The simplest way to look at these are, lower I-Frame interval (closer to 1 in your GUI for example), the more frequent the I-Frames and the higher the image quality (subjective to some BUT is a fact) but you also increase bandwidth requirements. Higher the I-Frame interval (further away from 1 for example), lower the frequency of I-Frames and image quality will be lower as will bandwidth. This however is also dependent on camera, codec and most importantly the subject or target. If you have fast moving objects you will almost always benefit from more I-Frames regardless of impact to storage / bandwidth (which as I mention below, you can plan for proactively).
The reason for matching your I-Frame interval to FPS (or if you see it as increasing I-Frames in general) is that you increase the opportunity for better quality frames within your capture as you are setting a ‘full image capture’ for every frame you are recording. Remember that I-Frames do not rely on previously encoded frames, only working within the same frame, this is what allows them to improve and maintain overall stream quality. The encoder you’re using (with associated codec) then utilizes I-Frames as reference (as I mentioned above) when encoding the Predicted (P-Frame) and Bi-drectional (B-Frame). As mentioned many times before, this DOES increase your bandwidth used BUT isn’t necessarily catastrophic (multi cam deployments of course do add up) for most well dialed in systems where you are focused on a crisp IVS rule (OR AI if you’re using that) critically only capturing real targets for a set period of time within the FOV. Also we have to remember as I and others have stated on many occasions, storage is ‘relatively’ cheap and certainly no one should be in a situation where they are looking to save on storage space at the expense of securing ID related captures. This is a critical part of CCTV deployment planning, live and historic retention of captured footage, and should be figured into your project plan whether it’s residential, commercial or even government related.
As I eluded to above, this is even more impactful in busy, high traffic FOV’s where artifacting can increase when using less I-Frames and can be incredibly important for special application work (ALPR/Face Recognition etc). In the case of ALPR/ANPR and facial recognition for example, matching (or exceeding in some cases) the number of I-frames (to FPS) on cameras in that environment can be highly beneficial to overall image detail in the few frames you have with critical recognition data in them.
Another area that people often don’t cover is the other added benefit of I-Frames and why having these more prevalent in your stream can assist if/when you hit a corruption event. Remember that I-Frames hold the key to corruption recovery too since they are based on image quality (full image) and not dependent on previous encoded images. Therefore, when a Predicted (P-Frame) or Bi-directional (B-Frame) frame is corrupted it benefits from the the next I-Frame to recover from the corruption by re-encoding or decoding and benefits greatly when frames beyond the I-Frame (used as the re-encode / decode point) exist.
Lastly in terms of scanning forward or backward in a video, I would be remiss if I didn’t mention that as well. Remember that I-Frames are in fact a Key Frame and therefore play a part in where a video will seek to when moving forward and backward. The longer the I-Frame interval (or Key Frame) placement the longer the seek point will be in the stream when moving forward and backward. Not the most important part of I-Frame usage but mentioning it anyway.
So, can you use less I-Frames than your FPS, YES, should you, my recommendation is NO. For most situations, rather than running the risk of less quality in each frame captured (again, subjective BUT a fact), especially in high traffic FOV’s or special applications, simply plan your storage correctly as part of your deployment and have a system that can adapt to the changing targets in FOV. Does that mean everyone should follow this rule, absolutely NOT. I advocate for what I feel works best in my experience across many large and small deployments but hey, many people run strange configs around FPS even based on a specific need or in some cases misunderstanding but thats down to individual needs, preference, understanding and deployment scenario. Also don’t forget that some NVR SW platforms also require FPS and I-Frame interval to match.
CBR vs VBR
Now, continuing on from ‘image quality’ of I-Frames vs bandwidth / storage impact, I’ll now cover CBR vs VBR. I’ll make this one super short. CBR or Constant Bit Rate is what I would and do advise 99.9% of the time. Variable Bit Rate (VBR) encoding, although ‘good’ in theory (and has improved over time) is NOT well implemented in most CCTV solutions (even pro level and cinema cameras have issues here). A big part of the problem is what I call ‘spin-up time’ to detect motion changes and adjust encoding algorithms for the motion seen within the FOV. With the right algorithm and processing power, you have options, without it, you unfortunately see issues. VBR takes longer to encode, can place more strain on the camera / NVR / NVR platform and absolutely can make or break a critical capture moment. Due to the above, I operate and advocate for the fact that it is never worth taking a chance that your required ID frame(s) is ruined by a slow processing algorithm on cam and associated codec, therefore set CBR (granted to your desired bitrate appropriate to your target requirements) and forget about it.
Shutter Speed
Now onto another critical area, that of Shutter Speed. On a well dialed in (and not cheap security or webcam style) camera, Shutter Speed ABSOLUTELY, 100% makes a difference and IS required and effective for freezing action. Your setting for shutter speed of course is determined by your target acquisition requirements or special project applications. For example, as I’ve shared for years, 1/30 is the lowest I would advise to go for human targets at slower speed, 1/60 to 1/100 minimum for more moderate targets (think humans walking quicker, light jog) and then 1/250 is the minimum for higher speed targets such as vehicles (NOT ALPR/ANPR). I recommend these to ensure motion blur free caps as much as possible.
Back to the special applications that I mentioned and ALPR is one of those where much high shutter speed IS needed to ensure crisp plate capture for manual or automated LPR work (outside of barrier based, entry chokepoints etc). In a number of cases based on height, angle, distance to target, plate within FOV, you may be able to get away with 1/500 BUT on faster moving targets and if the other dimensions at install location are not 100% on point as well (height, distance, angle) then 1/1000 or 1/2000 would be my recommended minimum. Staying on the special applications train for a second, this also depends on camera being used. While all of the settings I mention above are valid across the board, when you add into the mix special traffic cameras such as the ITC’s I’ve mentioned and reviewed, they up the ante on their abilities to capture in more harsh environments / trickier installs. This is due to not only the usual mix of traffic specific algorithms, optics and SOC pairings BUT critically in the way they process a target within FOV. I’ve spoken at length before about how Video Metadata / AI cameras are looking for a target that matches VEHICLE and only then will process a plate whereas traffic cameras do NOT need a full vehicle recognition to process for a plate directly and are amazingly accurate even at lower confidence levels. This gives you latitude in your install location and in your configuration of the camera itself. These traffic cameras also have front and rear light compensation for plates (as I’ve demonstrated before) and therefore also give further options in your configuration, improved over standard AG / light adjustment or SMART IR would on ‘standard’ (non-traffic) security cameras.
Another ‘special application’ is for face recognition, an increasingly popular deployment requirement that I and others are called in for as part of a project. This too benefits from shutter speed adjustments. You want as clear an image of facial features as possible and often times if this is NOT in a controllable environment (I.e. not an entrance with biometrics and facial ID right in front of you) then dialing in your camera, adjusting shutter speed higher, will assist here too (note: within reason and does have a point of diminishing returns for facial recognition). The goal being to reduce as much motion blur as you can (along with other factors like limiting noise reduction that crushes detail etc etc) to ensure you a) capture recognizable features clearly first time and/or b) allow for a high confidence match against your known vs unknown databases.
On the flip side of this, often times people are tempted to go with low shutter speed to improve low light situations, which it can BUT at the detriment to image quality due to horrendous blur/motion smear (remember, NEVER use AUTO). In this case ADD MORE LIGHT to the FOV, whether on-camera (not my first choice and wouldn’t advocate for this except when absolutely necessary even on newer cams) or off-camera. Continuing on the subject of lighting for a second. Lighting should also be constant wherever possible to remove the ‘bloom’ effect you can get of motion activated lights coming on and killing your image as SmartIR tries to adjust etc.
The key takeaway for people is that shutter speed DOES make a difference and does work as intended on these cameras. If you are not seeing any difference between a 1/30 and a 1/1000 shutter speed or 1/100 to 1/2000 then either something is wrong with your camera OR you need to double check your overall configuration. As I’ve mentioned here and before, critically dialing in your camera for your location, desired FOV, lighting on scene and most importantly the target requirements is an absolute must. Don’t ruin your captures by not dialing in correctly. On that note, this is why WDR and other backlight options should also be used VERY carefully as they will also introduce blur. Similarly as I mentioned above, keep NR under control, super low in the day (around 30-35 max) and no more than 40 at night. If using for LPR/NPR then you can even drop below 30 at night BUT I would never go less than 25 (this still allows the ALPR/ANPR traffic cams to work well) AND this will depend highly on your install location, lighting on scene etc. None of these cams are OR should be looked at as plug n play, they’re not as we’ve discussed many times. Dialing in your cams which includes shutter speed (remember to compensate for light impact) will take your useful captures to the next level.
Closing Thoughts
In summary, the important thing to remember is that you don’t / can’t control targets in your location BUT you do control how your cams are configured within that deployment and that is critical. You never know when someone will run up to an entrance way faster than anticipated or speed past an area at an angle that limits time in FOV. So, do you look to save bandwidth on the assumption that you may capture some of those events with critical frames OR do you set yourself up for success (as best you can) by dialing in your config to ensure your hit rate for ID frames improves and that short of a crazy event (like camera falling off of its install location), you’re covered. I’ll take and advocate for the latter and setting you up for success any day of the week in this business. So keep the above in mind, hope it helps
WC
