hikvision - extract thousands of images from camera sd card?

Symbiot78

n3wb
Joined
Jul 29, 2017
Messages
15
Reaction score
0
Hi

I have a hikvision camera with about a months worth of photos taken once pr. minute.

Is there an EASY way to get these images from the camera.

The sd card holds these 200Mb files which are not images but a .pic extension containing multiple images pr. file.

I can use the webbrowser but it can only search 4000 images and then if I do another search (date ranges) it will overwrite the previous images with the same names.

Yes.. I can create a multitude of folders but it will take forever and a day..

I tried IVMS-4200 but I can only find a way to download videos and NOT images..

Any and ALL help is appreciated.

I've tried asking on another forum but no luck.. hoping someone has tried something similar.
 

TonyR

IPCT Contributor
Joined
Jul 15, 2014
Messages
16,703
Reaction score
38,879
Location
Alabama
Not sure if this helps, but there's a video here. You log into cam with browser instead of using IVMS-4200. The guy moves waay too fast so get out your note pad or get ready with the pause key.
Skip ahead to about 1:40 to do what you're asking about, as before that it's about formatting, setting up snapshot intervals, etc.
 

djtentman

n3wb
Joined
Mar 9, 2018
Messages
10
Reaction score
3
Thanks, but that's how we were doing it. There is a limit of 100 photos that can be downloaded at a time that way.
 

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
I'm looking for an answer to this same question as well.
I've found some people who have done it with a Perl script, but that's a bit over my head:
That's a Python script and it works with a bit of tweaking. It's not that complicated and I've made a video tutorial showing how to use that Python script to extract jpg files from a Hikvision pic file.


Here is the modified code of the script I've used in the video.
Code:
#!/usr/bin/env python3

"""
Copyright (c) 2017 Graeme Smith
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
--------------------------------------------------------------------------------
The purpose of this script is to retreive the JPEG images embedded in a PIC
file created by a Hikvision CCTV.
Usage:
Required flags:
    -i or --input filename(s)     : list of PIC files to be decompressed
    -d or --directory directory   : directory to output the files, if the
                                      directory doesn't exist it will be
                                      created
Optional flags:
    -p or --prefix name           : filenames will start with this prefix
                                    and end in '-nnn.jpg' where nnn is the
                                    number of the image in the sequence.
                                    Defaults to 'picture'
    -e or --every n               : only output every nth image. Defaults
                                    to 1
If the script detects existing image files with the same prefix it will try to
continue the sequence.
"""

import argparse
import binascii
import mmap
import re
from pathlib import Path
from tqdm import tqdm

parser = argparse.ArgumentParser(description='Decompress Hikvision PIC Files')
optional = parser._action_groups.pop()
required = parser.add_argument_group('required arguments')

required.add_argument('-i', '--input', metavar="Filenames", nargs='+', required=True)
required.add_argument('-d', '--directory', metavar="Output Directory", required=True)
optional.add_argument('-p', '--prefix', metavar="Prefix", help="Filename Prefix (Default: Picture)", required=False, default="Picture")
optional.add_argument('-e', '--every', metavar="Number", help="Output every nth picture (Default: 1)", required=False, type=int, default=1)
parser._action_groups.append(optional) # added this line

args = parser.parse_args()

print ("\nHikvision PIC File Decompressor")
print ("-------------------------------")

count = 1
picture = 0

output_dir = Path(args.directory)
output_dir.mkdir(parents=True, exist_ok=True)

print("\nWriting files to: " + str(output_dir.absolute()))

filename = args.prefix + "-" + str(count) + ".jpg"

file_output = output_dir.joinpath(Path(filename).name)

if file_output.exists():
  filelist = output_dir.glob(args.prefix + "-*" + ".jpg")
  for filename in filelist:
    number = int(str(filename.name)[len(args.prefix)+1:-4])
    if number > count:
      count = number
  count += 1
  print("\nExisting Files Found - Starting at number:" + str(count))

for file_input in args.input:

  if Path(file_input).is_file():
    print ("\nDecompressing: %s" % file_input)
  else:
    print ("\nError: %s not found" % file_input)

  start = 0
  end = 0

  with open(file_input, 'rb') as f:
    f.seek(0, 2)
    num_bytes = f.tell()

    print ("File Size: %.2f MB" % float(num_bytes / 1024 / 1024))
    print()
    i = 0
    status = "No files written"
    t = tqdm(total=num_bytes, unit='B', unit_scale=True, unit_divisor=1024, desc="Progress")

    mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)

    while 1:

      f.seek(end)

      t.set_description(status)

      start = mm.find(b'\xFF\xD8', end)
      if start == -1:
        break

      end = mm.find(b'\xFF\xD9', start)

      if (picture % args.every == 0):

        filename = args.prefix + "-" + str(count) + ".jpg"

        filename_output = output_dir.joinpath(Path(filename).name)
        file_output = open(filename_output, 'wb')
        f.seek(start)
        jpeg_data = f.read(end - start)
        file_output.write(jpeg_data)
        file_output.close()
        status = "File " + filename + " written"
        count += 1
      picture += 1
      t.update(end - start)

  t.update(num_bytes - t.n)
  t.close()
  f.closed
 

Dazcroft

n3wb
Joined
Aug 24, 2018
Messages
5
Reaction score
0
Location
UK
Hi, This script has been a godsend, Is there any way to edit the script to do more than 1 at a time? I have no clue when it comes to python.
 

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
The tedious way is to add multiple input files after -i like this:
Code:
decompress.py -i "C:\Share\datadir0\hiv00000.pic" "C:\Share\datadir0\hiv00001.pic" -d "C:\Share\JPG"
The easy way is to create a batch script that runs the python script and drag and drop all the pic files over it.

When dragging and dropping over the batch script make sure that after you select all the pic files you drag by clicking on the first file (hiv00000.pic) so that the files are processed in alphabetical order. Doing so will ensure that the date and time in the output JPG files will be in correct order. So if you have three files to process hiv00000.pic hiv00001.pic and hiv00002.pic after you select them all you click on the first one hiv00000.pic and drag them all over the batch file.

To create the batch file first create an empty txt file and paste the code below into it. Change the path to the decompress.py and to the output JPG folder as needed. Save the file and rename its extension from txt to bat

Code:
@echo off
if [%1]==[] goto :eof
:loop
"C:\Share\decompress.py" -i "%~1" -d "C:\Share\JPG"
shift
if not [%1]==[] goto loop
echo ********************
echo ***** FINISHED *****
echo ********************
pause
 
Last edited:
Joined
Nov 15, 2018
Messages
5
Reaction score
0
Location
Russion federation
That's a Python script and it works with a bit of tweaking. It's not that complicated and I've made a video tutorial showing how to use that Python script to extract jpg files from a Hikvision pic file.


Here is the modified code of the script I've used in the video.
Code:
#!/usr/bin/env python3

"""
Copyright (c) 2017 Graeme Smith
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
--------------------------------------------------------------------------------
The purpose of this script is to retreive the JPEG images embedded in a PIC
file created by a Hikvision CCTV.
Usage:
Required flags:
    -i or --input filename(s)     : list of PIC files to be decompressed
    -d or --directory directory   : directory to output the files, if the
                                      directory doesn't exist it will be
                                      created
Optional flags:
    -p or --prefix name           : filenames will start with this prefix
                                    and end in '-nnn.jpg' where nnn is the
                                    number of the image in the sequence.
                                    Defaults to 'picture'
    -e or --every n               : only output every nth image. Defaults
                                    to 1
If the script detects existing image files with the same prefix it will try to
continue the sequence.
"""

import argparse
import binascii
import mmap
import re
from pathlib import Path
from tqdm import tqdm

parser = argparse.ArgumentParser(description='Decompress Hikvision PIC Files')
optional = parser._action_groups.pop()
required = parser.add_argument_group('required arguments')

required.add_argument('-i', '--input', metavar="Filenames", nargs='+', required=True)
required.add_argument('-d', '--directory', metavar="Output Directory", required=True)
optional.add_argument('-p', '--prefix', metavar="Prefix", help="Filename Prefix (Default: Picture)", required=False, default="Picture")
optional.add_argument('-e', '--every', metavar="Number", help="Output every nth picture (Default: 1)", required=False, type=int, default=1)
parser._action_groups.append(optional) # added this line

args = parser.parse_args()

print ("\nHikvision PIC File Decompressor")
print ("-------------------------------")

count = 1
picture = 0

output_dir = Path(args.directory)
output_dir.mkdir(parents=True, exist_ok=True)

print("\nWriting files to: " + str(output_dir.absolute()))

filename = args.prefix + "-" + str(count) + ".jpg"

file_output = output_dir.joinpath(Path(filename).name)

if file_output.exists():
  filelist = output_dir.glob(args.prefix + "-*" + ".jpg")
  for filename in filelist:
    number = int(str(filename.name)[len(args.prefix)+1:-4])
    if number > count:
      count = number
  count += 1
  print("\nExisting Files Found - Starting at number:" + str(count))

for file_input in args.input:

  if Path(file_input).is_file():
    print ("\nDecompressing: %s" % file_input)
  else:
    print ("\nError: %s not found" % file_input)

  start = 0
  end = 0

  with open(file_input, 'rb') as f:
    f.seek(0, 2)
    num_bytes = f.tell()

    print ("File Size: %.2f MB" % float(num_bytes / 1024 / 1024))
    print()
    i = 0
    status = "No files written"
    t = tqdm(total=num_bytes, unit='B', unit_scale=True, unit_divisor=1024, desc="Progress")

    mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)

    while 1:

      f.seek(end)

      t.set_description(status)

      start = mm.find(b'\xFF\xD8', end)
      if start == -1:
        break

      end = mm.find(b'\xFF\xD9', start)

      if (picture % args.every == 0):

        filename = args.prefix + "-" + str(count) + ".jpg"

        filename_output = output_dir.joinpath(Path(filename).name)
        file_output = open(filename_output, 'wb')
        f.seek(start)
        jpeg_data = f.read(end - start)
        file_output.write(jpeg_data)
        file_output.close()
        status = "File " + filename + " written"
        count += 1
      picture += 1
      t.update(end - start)

  t.update(num_bytes - t.n)
  t.close()
  f.closed
Hello, this program was wery helpful for me. But maybe you know how to save information about time of creation of snapshot? And put it in the jpg filename for example
 

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
I've made a new video tutorial in which I'm using a PERL script to extract the JPG files. This time the script is using the images capture date and time for the file names.

Edit: I've updated the script and the video tutorial. Scroll down this page to find the new one.
 
Last edited:
Joined
Nov 15, 2018
Messages
5
Reaction score
0
Location
Russion federation
I've made a new video tutorial in which I'm using a PERL script to extract the JPG files. This time the script is using the images capture date and time for the file names.
Hello. I've took this on ds-2cd2042wd-i.
F:\Cam1>Extract.pl "F:\Cam1\datadir0" "F:\Cam1\JPG"
PicFile: hiv00000.pic at 2816
2816: 48, 150011904, 1539094308, 1539098888, 0, 5356666, 0, 0, 49, 155320320, 1539098890, 1539103632, 0, 5356666, 0, 0
2896: 0, 5356666, 0, 0, 51, 144769024, 1539108196, 1539112616, 0, 5356666, 0, 0, 52, 108593152, 1539112618, 1539115934
Died at F:\Cam1\Extract.pl line 63.
It's seems very different with table in video :(
 
Last edited:

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
Hi, can you upload your index00p.bin file so I can take a look at it?
 

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
You need to change $headerSize = 11088;
This will fix the table.

If you have a lot of .pic files and you want to extract all the pictures you also need to change the value of $maxRecords.

For example if you have 3 .pic files (hiv00000.pic, hiv00001.pic, hiv00002.pic) you have to change $maxRecords to 4096 * number of .pic files, so 4096*3 = 12288, so $maxRecords = 12288;

If you have 8 .pic files 4096*8 = 32768 so $maxRecords = 32768;
 
Last edited:
Joined
Nov 15, 2018
Messages
5
Reaction score
0
Location
Russion federation
You need to change $headerSize = 11088;
This will fix the table.

If you have a lot of .pic files and you want to extract all the pictures you also need to change the value of $maxRecords.

For example if you have 3 .pic files (hiv00000.pic, hiv00001.pic, hiv00002.pic) you have to change $maxRecords to 4096 * number of .pic files, so 4096*3 = 12288, so $maxRecords = 12288;

If you have 8 .pic files 4096*8 = 32768 so $maxRecords = 32768;
Now only the first .pic file is extracted correctly. Then the same 1st .pic file continues to be extracted, but the JPG files continue to be named according to the second .pic file, and most of them are corrupted. May be easier to set the .pic file name as an input variable?
 
Joined
Nov 15, 2018
Messages
5
Reaction score
0
Location
Russion federation
You need to change $headerSize = 11088;
This will fix the table.

If you have a lot of .pic files and you want to extract all the pictures you also need to change the value of $maxRecords.

For example if you have 3 .pic files (hiv00000.pic, hiv00001.pic, hiv00002.pic) you have to change $maxRecords to 4096 * number of .pic files, so 4096*3 = 12288, so $maxRecords = 12288;

If you have 8 .pic files 4096*8 = 32768 so $maxRecords = 32768;
Found a solution: no need to change $maxRecords from 4096.
Thank you very much for this program!!!
 

Securame

Pulling my weight
Joined
Mar 25, 2014
Messages
664
Reaction score
214
Location
Barcelona, Spain
Awesome thread. I have 4 Hikvision cameras with micro SDs taking a snapshot every 5 minutos so I can later do a timelapse, this will surely come in handy.
Thank you!
 

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
Now only the first .pic file is extracted correctly. Then the same 1st .pic file continues to be extracted, but the JPG files continue to be named according to the second .pic file, and most of them are corrupted. May be easier to set the .pic file name as an input variable?
I've tested again and I've made a mistake the first time. There is no need to change maxRecords, it should be left as it is $maxRecords = 4096;

You need to have all the .pic files in the same directory as the index00p.bin file and they need to be consecutive and produced by the same camera. You can't just rename the .pic files to extract them one by one. If you rename hiv00001.pic to hiv00000.pic then the script will use the info for hiv00000.pic to extract the pictures from hiv00001.pic and this will result in corrupted JPG files.

The headerSize seems to be different depending on the camera model and can change even if you format the storage from the camera menu, so if for some of you the table looks wrong, you can post your index00p.bin file so I can give you the correct value for the headerSize. You can also find the value for the headerSize by analyzing the index00p.bin file with a HEX editor. I use FlexHEX.
 
Last edited:

Alex_N

Young grasshopper
Joined
Jun 20, 2018
Messages
42
Reaction score
22
Location
EU
I've updated the script and now it automatically finds the headerSize. There's no more need to use a HEX editor to find it or to change anything in the script's code.

I've also added the option to use a date and time interval limit when extracting the images. This saves a lot of time if you have lots of images stored but only want to extract some of them.

This is the updated video tutorial
2019 UPDATE: In the tutorial I've used ActiveState Perl, but the script doesn't work with the latest version. Make sure to download and install Strawberry Perl instead of ActiveState Perl.
Strawberry Perl for Windows


I've tested the script using a lot of different index00p.bin files to see if it fails to find the headerSize. It didn't, but if for some of you the extracted jpg files are corrupted (only half the image loads or doesn't load at all) you can send me your index00p.bin file and I will update the script.
 
Last edited:
Top