IP Cam Talk Custom Community DeepStack Model

I found a way to AI assist label the animals in the images using DS and animal V2.0 custom model, thanks to the below post. If someone can send some images so I can test how it work. Before sending the images please rename them for example deer (1), deer (2), fox (1), fox (2) below is a link to how to do this.

Use File Explorer To Batch Rename Files In Windows 10

cjowers said:
i managed to write in the YOLO automated labeling annotations into a python script that identifies the characters from plate images. it may take you some time to get it running / configured with your file system, but should help someone who wants to do something similar. (either automated labeling, or running the detection of a custom made OCR model on a large dataset of cropped images.
I used YOLO5m, ~200 images to train. results are good, but not perfect

***edited to update code massively - v20210712. way way better now!
*****edited with small updates - v20210713

Python: 
#Custom_LPR_OCR_detection_v20210713_wAutoYoloLabeling.py
#created by u\shallowstack for the purpose of either:  autolabeling for YOLO deepstack custom model for OCR,
# or for logging the detected characters in images within a file directory, with a number of filtering capabilties.

# before running this code, first run the below call in Powershell (not CMD) to start Deepstack custom model:
#PS C:\Windows\system32> deepstack --MODELSTORE-DETECTION "C:\AI_models\lpr_ocr\" --PORT 97
#change port and dir to whereever your custom (character recognition) AI model is...

#major changes to v20210712 -
# deals with overlapping detected characters. as i think these are common with custom OCR models.
# if too many detections, it now filters down to an expended number of Characters.
# now skips any left over yolo text files in the directory without error.  note: deletes txt file only when it needs to write a new one of the same name.
# better debugging info and code comments for usability

import requests
import os
import PIL
import sys
from PIL import Image

#CHANGE THESE GLOBAL VARIABLES for customized control
overlapping_threshold = 4           #how close the characters can be before the inferior one is ignored
min_conf_thres = 0.6                #if worst confidence is less than this, the result wont be written to output_logfile
min_len_thres = 3                   #if there is not more than this number of characters detected on the plate, the result wont be written to output_logfile
plate_len_threshold = 6              #if there is more character detected than this number, program will cut out the weakest confidence characters until value met.

#CHANGE THESE DIRECTORIES TO MATCH YOUR FILESYSTEM
input_directory = 'J:\\BlueIris\\Deepstack_LPR_dataset\\results_2021'                          #where large dataset of cropped plates live
output_logfile = 'C:\\Users\\BlueIrisServer\\Desktop\\logfile.txt'            #where python will save the log file

#change this file pointer to whereever your YOLO class file is.  it will sync up the character labels with the class numbering scheme
classYoloFilename = 'J:\\BlueIris\\Deepstack_LPR_dataset\\OCR_testing\\QLD Model\\train\\classes.txt'

#reads 'class' txt file, and puts into a list to be searched every time a label is given, with the index value of the list returned so it can be written to the YOLO file.
classLabels = tuple(open(classYoloFilename).read().split('\n'))
print(classLabels)       

#get # of x pixels for yolo file conversion from pixel # to %
def get_xnum_pixels(filepath):
    width, height = Image.open(filepath).size
    return width
#get # of y pixels for yolo file conversion from pixel # to %
def get_ynum_pixels(filepath):
    width, height = Image.open(filepath).size
    return height

#cleanup variables before run, just in case
resp_label = [0]
resp_conf = [0]
resp_pos = [0]
resp_data2write = [""]
resp_label.clear()
resp_conf.clear()
resp_pos.clear()
resp_data2write.clear()
YoloLabelFilepath = ""
data2write = ""

i = 0

#goes through entire input_directory and processes each file to look for alphanumeric characters
for filename in os.listdir(input_directory):
    #store full filepath of current file under inspection
    filepath = os.path.join(input_directory, filename)
   
    #skip file if it is a txt file
    if filename.rsplit('.',1)[1] == 'txt' :
        print(filename + " is a txt file, SKIPPED OCR DETECTION ON THIS FILE.")
        #os.remove(filepath)   #uncomment this if you want it to delete the text file from directory.  (untested)
        continue
    else :
        print(filename + " is not a txt file, sending to OCR detection AI...")

    #get # of x pixels for yolo file conversion from pixel # to %
    xSize = get_xnum_pixels(filepath)
    #get # of y pixels for yolo file conversion from pixel # to %
    ySize = get_ynum_pixels(filepath)

    #create new text file to store label annotations
    YoloLabelFilepath = filepath.rsplit('.',1)[0] + '.txt'
    image_data = open(filepath,"rb").read()

    #clear variables from last image data
    resp_label.clear()
    resp_conf.clear()
    resp_pos.clear()
    resp_data2write.clear()

    #posts to deepstack custom server and logs result
    response = requests.post("http://localhost:97/v1/vision/custom/yolo5m_best_20210623",files={"image":image_data}).json()     #change port 97 to whatever your deepstack custom server is on, and custom model name
    #print(response)

    #go through all detections in image and store to temporary lists for comparisons
    for detection in response["predictions"]:
        resp_label.append(detection["label"])
        resp_conf.append(round(detection["confidence"],2))
        resp_pos.append(detection["x_min"])
     
        #print annotation results for assisted labeling - for future incorporation into model
        #this will print a YOLO type txt file for each image processed, according to classes.txt file
       
        #some maths to get xy pixel values into YOLO format:  x center, y center, x width, y width (all as a % of total image size)
        xCenter = float(detection["x_min"] + detection["x_max"]) / float(2)
        yCenter = float(detection["y_min"] + detection["y_max"]) / float(2)
        xWidth = detection["x_max"] - detection["x_min"]
        yWidth = detection["y_max"] - detection["y_min"]

        xCenter = format(round(xCenter / xSize, 6), '.6f')
        yCenter = format(round(yCenter / ySize, 6), '.6f')
        xWidth = format(round(xWidth / xSize, 6), '.6f')
        yWidth = format(round(yWidth / ySize, 6), '.6f')

        #check the class list to see what index value the label is that was returned from detection
        ClassValue = classLabels.index(detection["label"])
        #format is :   class (not label) xcenter% ycenter% xwidth% ywidth%
        data2write = str(ClassValue) + " " + str(xCenter) + " " + str(yCenter) + " " + str(xWidth) + " " + str(yWidth) + " "

        # resp_xCenter.append(xCenter)
        # resp_yCenter.append(yCenter)
        # resp_xWidth.append(xWidth)
        # resp_yWidth.append(yWidth)
        # resp_ClassValue.append(ClassValue)
        resp_data2write.append(data2write)

    #sort all stored arrays (label, confidence, YoloData) according to x_min position array  (so it reads left to right like we see it)
    B1=resp_pos
    B2=resp_pos
    B3=resp_pos
    A=resp_label
    C=resp_conf
    D=resp_data2write

    #if NO detections made exist with debug info
    if len(C) == 0:
        print("No Char was found in image " + filename)
        continue
    else:
        pass

    #sort resp_label array
    zipped_lists1 = zip(B1,A)
    sorted_pairs1 = sorted(zipped_lists1)
    tuples = zip(*sorted_pairs1)
    B1, A = [list(tuple) for tuple in tuples]
    #sort resp_conf array
    zipped_lists2 = zip(B2,C)
    sorted_pairs2 = sorted(zipped_lists2)
    tuples = zip(*sorted_pairs2)
    B2, C = [list(tuple) for tuple in tuples]
    #sort resp_data2write array
    zipped_lists2 = zip(B3,D)
    sorted_pairs2 = sorted(zipped_lists2)
    tuples = zip(*sorted_pairs2)
    B3, D = [list(tuple) for tuple in tuples]

    #debug info only
    print(str(A) + ", " + str(B1) + ", " + str(C) + ", " + str(D))

  
    k=0
    #go through each position in the lists
    for m in B1 :
        i = 0
        if k == len(B1) :
            print("max of list reached already.")
            continue
        #check if any CHARs are overlapping according to their xmin value
        upperLim = m + overlapping_threshold
        lowerLim = m - overlapping_threshold
        for n in B1 :
            if i == len(B1) :
                print("max of list reached already.")
                continue
            if n < upperLim and n > lowerLim and i != k :
                if C[i] > C[k] :
                    #debug info only
                    print("deleted an overlapping CHAR: '" + str(A[k]) + "', " + str(B1[k]) + ", " + str(C[k]) + ", with yolo info: " + str(D[k]))
                    #remove detection # [n] from all lists
                    del B1[k]
                    del B2[k]
                    del B3[k]
                    del A[k]
                    del C[k]
                    del D[k]

                else:
                    #remove detection # [m] from all lists
                    #debug info only
                    print("deleted an overlapping CHAR: '" + str(A[i]) + "', " + str(B1[i]) + ", " + str(C[i]) + ", with yolo info: " + str(D[i]))
                    del B1[i]
                    del B2[i]
                    del B3[i]
                    del A[i]
                    del C[i]
                    del D[i]             
            else:
                pass
            i=i+1
        k=k+1
   
    #if results still more CHARs than expected, being deleting the lowest confidence items
    check_len = len(C)
    while check_len > plate_len_threshold :
        #remove detection # [i] from all lists
        i = C.index(min(C))   
        #debug info only
        print("more detections than expected (plate_len_threhold = " + str(plate_len_threshold) + "), so the following (low conf) CHAR was deleted : '" + str(A[i]) + "', " + str(B1[i]) + ", " + str(C[i]) + ", with yolo info: " + str(D[i]))
        #delete index i
        del B1[i]
        del B2[i]
        del B3[i]
        del A[i]
        del C[i]
        del D[i]
        check_len = len(C)


    #delete any exisiting yolo text file by that name
    if os.path.exists(YoloLabelFilepath) :
        os.remove(YoloLabelFilepath)
        print(YoloLabelFilepath + " file removed, so a new yolo txt file can be written.")
    else:
        pass
    #write to yolo text file       
    text_file = open(YoloLabelFilepath,"a")
    j=0
    for o in D :
        text_file.write(D[j] + '\n')
        j=j+1
    text_file.close()
   
    #debug
    print("I wrote all lines to Yolo .txt : " + str(D))
   
    #print results to log file (if looks OK)              change directory to where ever you like, or add info
    if C != [] and len(C)>min_len_thres:
        if min(C) > min_conf_thres :
            text_file = open(output_logfile,"a")
            text_file.write(filename)
            text_file.write("\n")
            text_file.write(str(A))
            text_file.write("\n")
            text_file.write(str(B1))
            text_file.write("\n")
            text_file.write(str(C))
            text_file.write("\n")
            text_file.write(str(D))
            text_file.write("\n")
            text_file.close()
        else:
            pass
    else :
        pass
   
    #if best found character really sucks, notify
    if max(C) < 0.1:
        print("No Char was found in image " + filename)
    else:
        print("COMPLETED : " + filename + " *** " + str(A) + " *** " + str(B1) + " *** " + str(C) + " *** " + str(D))
Click to expand...
 
  • Like
Reactions: cjowers, sebastiantombs and 105437
@MikeLud1 I know I have some fox images from overnight. I'll get them sent later today. Also, with the new AI you mentioned, it sounds like we will have one model to cover everything day and night. Is that right?
 
  • Like
Reactions: sebastiantombs
Going back to the animal model for a second, this is the first time I've ever had birds correctly identified. Impressive, Mike!
 
  • Like
Reactions: looney2ns and MikeLud1
MikeLud1 said:
Now with the newly found AI assist labeling I will now be able to add the dark custom model images to the new custom models using DS and dark.pt custom model.

The below is a sample image from the dark model, (A Reolink camera must have been used LOL)

View attachment 112700
Click to expand...


Sweet - are you going to be able to do that to both the custom model with and without animals?
 
That explains why I get "cat" "cat" when using dark, combined and animal.
 
Dark doesn't see the "cat", just combined and animal.
 
  • Like
Reactions: MikeLud1
Captured a deer overnight. One was identified correctly. Another was identified as a bird. These images are lo-res, since I can't get BI to create a hi-res alert image while I'm using sub-streams.

Screen Shot 2021-12-20 at 9.13.19 AM.pngScreen Shot 2021-12-20 at 9.14.02 AM.png
 
Last edited:
  • Like
Reactions: sebastiantombs and MikeLud1
I had a squirrel ID as a "bird" this morning using combined and animal.
 
  • Like
Reactions: MikeLud1
I know not to expect perfection from DS. No system is perfect. What I am happy with is that it is detecting things that were missed previously. That is an improvement. Even my tired, old, eyes can tell what I'm looking at definitely, as long as it's caught. That's the biggest hurdle in my mind.
 
  • Like
Reactions: jrbeddow, TBurt, kklee and 3 others
You must log in or register to reply here.
Top Bottom