Dedicated License Plate Cam project

bumped the resolution up past 1080p, now I am sending it 2304 × 1296 video and night time read accuracy went up dramatically..

How does that affect your processing time? Are you going to implement a hotlist/blacklist/whitelist to provide alerts say when the wife's car drives in or that you can put regos that the local Police issue keep a lookout for?

another bit of information concerning anyone wanting to do LPR over distance, with high shutter speeds weather is more of a problem than you may expect..

Hate to tell you this. It gets worse at night with the IR/car headlights illuminating the rain or, in your case, snow. An issue I have that will probably massively affect the performance of alprd is when I get rain on the front of the enclosure. Each rain drop acts like a tiny lens to spread the car headlights (think really severe IR halo effect) and effectively white out my camera until the headlight beams pass the camera at which point it is human readable.

It isn't so much the distance as the resolution over distance. You have high enough resolution to see the snow from lens to plate, whereas most wide angle lenses have enough resolution to see snow from lens to a few metres.
 
Last edited by a moderator:
not as dramatically as I expected, infact I am now at full 4MP and still getting plate processing times down around 200ms or less.. I have to recalibrate the day/night focus now because it actually changed the focal point, the FOV is a few degrees wider now.. but thats cool by me, just going to take a bit of time to get everything dialed in as well as it was before but I think I can get it done at full resolution.

Recalibrating everything is kind of a pain, to ensure I get high accuracy means I have to be sitting at the computer with live feed monitoring alot of traffic, day and night.. if you miss one out of 50 you have to wait for 50 cars to drive by before you find that edge case and try to figure out why.. then hope you fixed it correctly and when all is said and done your watching and waiting for enough traffic to be generated naturally.

and yeah right now its not even human readable, i just saw a car go by and alprd didnt spit anything out.. so went back checked the footage (720p), this is the cap I took manually.. but its allright, most plates are packed with snow right now so nothing could be done short of a microwave heat gun powerfull enough to stop the vehicle dead in its tracks :)

I am still trying to flush out how I want to store the data and the underlying logic, but yeah I'll have hotlists that will trigger a notification.. and room for additional details in the database, like make/model/color/company-orginization for manual input for residence that hit the system enough.. I am thinking 4 tables, one for captures and metadata, one for vehicles and specifics, one for re-mapping commonly misread plates either manually or semi-automatically (ie, B65AH5 = 865AHS; hitches/bike-racks that miss a character or two), and the last table for the shitlist/hotlist/automation.. if a match in the vehicle or remapping table is found then it'll just add the capture to the vehicle's register, otherwise if confidence level is high it will create a new vehicle with the information it knows, and if confidence is low or output non-standard it will be unmatched to a vehicle and flagged for manual reading/re-training

I will probably use it to sense our arrival for home automation, if the house is armed and it reads one of my plates come in late at night I could have it turn all the lights on for me or something like a passive geofence.. warn that the Code Nazi's/UPS/FedEx/Mother in law is coming.. its not a huge priority but I will design with this in mind.

right now I just have it updating the OSD on the camera, is great for testing.. car drives by and I can see the text change to its best guess.. if it dont then I go back and check footage and see if a plate was even visible or if I really missed it.. when conditions are decent it almost always due to lack of plates, temp tags can be in windows or out of state vehicles with no front plate.
 

Attachments

  • Galileo NVR-ALPR (720p)-2016-04-30-23-06-01.jpg
    Galileo NVR-ALPR (720p)-2016-04-30-23-06-01.jpg
    328 KB · Views: 115
Last edited by a moderator:
and yeah right now its not even human readable, i just saw a car go by and alprd didnt spit anything out.. so went back checked the footage (720p), this is the cap I took manually.. but its allright, most plates are packed with snow right now so nothing could be done short of a microwave heat gun powerfull enough to stop the vehicle dead in its tracks :)

You so know you want to. :nuts: The missus wouldn't mind too much if you took the magnetron out of the microwave in the kitchen would she?

Mother in law is coming.

The microwave gun could have further uses than just melting the snow off plates
nuts.gif
 
Last edited by a moderator:
  • Like
Reactions: nayr
Calibrating Masks, this is what I've been doing.. seems to work pretty well, if anyone has a better strategy I am open to alternatives and/or improvements.

you need sample imagery to build your mask.. rip some captures out of your recordings with vehicles in position where you want them captured or run openalprd basically default expecting it to miss stuff but still get you alot of imagery to work with after enough time..
look for:

  • Biggest Plate & Smallest Plate, due to distance and angles.. if your capturing both coming and going you need them for both all lined up in roughly the same vertical plane.
  • Highest Plate and Lowest Plate, big trucks and lil cars.. the extreme edges.
  • No plate image, an image with no vehicles or plates to read.. this is your baseline.

Import all the images into gimp/photoshop as separate layers, put your no plate image as the lowest/background layer and go through and cut everything but the plate area out of all the other layers, leaving the plates in the original location minus the vehicle..

you should now have the big and small plates both coming and going all transposed on-top of each-other, change layer visibility as desired, take your biggest plate size, calculate the width in pixels and add a guide line the same pixels to both the left and right of your plate group.. take your highest plate read and add a guide with a small margin of error, same with your lowest plate read.. you should now have your first mask defined.. this will get refined but its a great starting point.

create a white layer, with a black layer on top and push them to the top.. select the area inside your guides and cut out that section of the black layer, you should now have a black and white mask you can export to a jpeg and put in your openalpr.conf

play with the max_plate_width_percent & max_plate_height_percent options in openalpr.conf with your largest plate capture, get those numbers as small as reasonable while still detecting your plate then add a few percentage for a margin of error, thats unlikely to really be the absolute largest plate you have.

run your no-plate capture through openalprd after you have the above options tuned, take the total processing time on divide it by 1000, round the results down and this is your starting FPS.. so ~200ms would be 5fps, ~100ms would be 10fps, etc.. the smaller we get our mask the higher the FPS we can run.. the higher the FPS we can run the smaller our mask can be.. somewhere is perfection, and with all the variables changing you can just tune it in w/trial and error.. you'll get the feel pretty quickly if you made things better or worse.

fire up openalprd and wait for traffic!

3 options will likely play out:
1. Too much capture, your getting a half dozen plate captures for each vehicle.. so we can shrink our width down on mask incrementally and retry until we get desired # of images on average, and at least one capture for even the fastest speeders.
2. Just enough capture, you nailed it... just keep testing and make sure nothing slips past your mask.
3. Missing captures, your mask it too small and you dont have the performance for the FPS needed to get everyone in that area.. so try: expand the width, lower the fps, back off the resolution, increase zoom.
 
Last edited by a moderator:
Calibrating Masks, this is what I've been doing.. seems to work pretty well, if anyone has a better strategy I am open to alternatives and/or improvements.

you need sample imagery to build your mask.. rip some captures out of your recordings with vehicles in position where you want them captured or run openalprd basically default expecting it to miss stuff but

I've been following this thread fairly closely because I want to have a crack at this but the amount of work this part of the process is likely to take for Australian plates is putting me off a bit.

In Australia, state governments have worked out that there is money in number plates and because of that we have a stupidly large number of plates an anpr system will have to deal with.

https://www.myplates.com.au/products/myplates-product-brochure.pdf
http://www.transport.wa.gov.au/licensing/plateswa.asp
http://www.transport.sa.gov.au/ezyplates.sa.gov.au/WebContent/

Different Sizes, Shapes, colours, with between 1 and 9 characters

Open ALPR has, so the demo page says, an Australian number plate data base, but the demo hasn't been particularly successful reading the front 50mm digit plates from 600ppm images. It sees the plates but the reads are inaccurate. It chops off one the numbers most of the time. Ironically it usually reads the correct plate but that result isn't the best match. It chops off the first digit on the euro style plates as ell but there are no correct reads there. I am concerned that the Australia profile is limited at best and I am going to have to write all the various Australian number plate sizes and configurations into the database software, and then find some way to simulate each of those styles to create the mask, before it will be reliable.

Edit:

I had a look at the code.

https://github.com/openalpr/openalpr/commit/e55c4a7070af06568a82de7c67b189ecc920ad50

Doesnt appear to be a config file for the NSW 50mm Front plates as yet. The code looks like a Queenslander wrote it.

The config files look easy enough to modify with the correct dimensions etc of all the local plates, but I'm buggered if I have any idea how those runtime_data/region/auwide.xml and au.xml files came into being. That may as well be ancient greek.
 
Last edited by a moderator:
These are the droids your looking for: https://github.com/openalpr/openalpr/issues/292

Ive been reading on training the OCR engine in OpenALPR, its an ordeal thats for sure: https://github.com/openalpr/train-detector

I do feel your pain, colorado has a shit load of plate types and configs.. our standard plates read quite well, but my wife and I have 'State Park' plates and they struggle to read it.. then we have titty cancer plates, and dog plates, and kid plates, horse plates, and military plates, and blah (https://www.colorado.gov/pacific/dmv/node/40131/) and it seems all the graphics just make it harder to read programatically.. even though the fonts are identical.. I can only imagine how much trouble your in for hah

so far Texas is the easiest to read, all white plate.. all black text, no bullshit gfx.. perfect!
 
Last edited by a moderator:
  • Like
Reactions: SyconsciousAu
Thanks for the links.

Might be time to go down to the carpark at the local train station or set up on a busy intersection with a camera and collect several of hundred reference images of 372mm x 84mm front plates. That should be fairly easy as that size is on the front of about a third of all cars. With my DSLR I should be able to capture three lanes at once.
 
if the fonts are the same I guess it shouldn't really matter, what your really training is the font.. there is a pdf attached to that ticket that has a pure copy of the font, they were talking about printing out all the letters, photographing them back in as .jpg and running them through the binary font tools attached.. that seems like the best starting place, might not even need a bunch of sample plate images.

If you figure it out it'll be to the benifit of all Aussies, or if you cant it looks like its on the ToDo list and perhaps one of the Developers will find the time to get it trained correctly for Australia.

for posterity here is a simple NodeJS app for updating the OSD on the camera, and saving the results to a .json file.. once i get the database backend all flushed out I'l just write a quick tool to import all these .json files into the database so I can start off with all this data I am already gathering.

Code:
#!/usr/bin/nodejs
// Simple NodeJS + Express Server for ALPRD
// Updates OSD on Dahua Camera and saves data to .json files


var username = 'username'
var password = 'password'
var host = '192.168.1.100'
var path = '/var/lib/openalpr/platejson/'


// Begin App
var express = require("express");            // npm install express
var myParser = require("body-parser");            // npm install body-parser
var req = require('request');                // npm install request
var fs = require('fs');


// logging
require("console-stamp")(console, {            // npm install console-stamp
    pattern:"HH:MM:ss.l", 
    colors: {
        stamp: "green",
        label: "yellow",
    }
});


var app = express();


app.use(myParser.json({extended : true}));
app.post("/push/", function(request, response) {
    console.log(request.body.results[0].plate + ' (' + request.body.processing_time_ms + 'ms)');
    req('http://' + username + ':' + password + '@' + host + '/cgi-bin/configManager.cgi?action=setConfig&VideoWidget[0].CustomTitle[1].Text=' + request.body.results[0].plate);
    response.send("recieved");
    file = path + request.body.uuid + '.json';
    fs.appendFile(file, JSON.stringify(request.body), function () {
        response.end();
    });
});


app.listen(9000);
console.info('listening @ http://localhost:9000/push/')
 
submitted a pull request today that implements the pattern checking in alprd: https://github.com/openalpr/openalpr/pull/341

pattern checking on alprd is nice, the first result in the json array will be the first matching pattern, despite confidence.. so when it has highest confidence that: 63B-QR5 is correct, the pattern is invalid and 638-QRS will move to the top of the array.. the OSD display's accuracy went up with this patch.
 
ok I way underestimated the performance of alprd, lol.. by a huge margin.

when I sumbited that patch and was combing through code I found command line options for alprd, not documented anywhere.

alprd -f --clock

will not fork alprd into background and will spit out frame rate info.. I found on my system, at full resolution with my mask I was processing frames at ~45ms
INFO - Camera 1 processed frame in: 45.927 ms.
INFO - Camera 1 processed frame in: 46.8468 ms.
INFO - Camera 1 processed frame in: 44.9455 ms.
INFO - Camera 1 processed frame in: 44.7077 ms.
INFO - Camera 1 processed frame in: 43.8167 ms.
INFO - Camera 1 processed frame in: 45.6521 ms.
INFO - Camera 1 processed frame in: 46.5659 ms.
INFO - Camera 1 processed frame in: 46.6936 ms.
INFO - Camera 1 processed frame in: 43.6089 ms.
INFO - Camera 1 processed frame in: 44.7076 ms.
INFO - Camera 1 processed frame in: 44.703 ms.
INFO - Camera 1 processed frame in: 46.0946 ms.
INFO - Camera 1 processed frame in: 46.1838 ms.
INFO - Camera 1 processed frame in: 42.8784 ms.
INFO - Camera 1 processed frame in: 45.3709 ms.
INFO - Camera 1 processed frame in: 45.9721 ms.
INFO - Camera 1 processed frame in: 46.7635 ms.
INFO - Camera 1 processed frame in: 46.1351 ms.
INFO - Camera 1 processed frame in: 46.1646 ms.
INFO - Camera 1 processed frame in: 41.7338 ms

so with this in mind I bumped my frame rate up to 20FPS, and hah.. those edge cases I been trying to catch all vanished and its having no problem finding a plate in the mask..

the thing ive found with the mask is partial clipping, it has to be wide enough to basically get 2 reads for most vehicles.. one may be clipped on either side and the next one will be a complete.. before I commit to database I may store a value, set a timer and if a new value comes in before the timer it clears it and does a comparison, if one is a partial of the other it'll discard it and commit only the complete read.

my approach of using alpr's benchmarks for alprd proved to be grossly incorrect, it was telling me it'd take ~200ms to read a blank image and alprd is cooking through them at less than 1/4 that.
 
ok well that was just at night, the frame rate is highly variable throughout the day.. very fast at night, much slower in day.. i was not seeing that with alpr command line option, same speed day/night but even tweaking image settings can increase/reduce framerate.. the brighter the image the slower it goes.. black and white in the day is actually twice as slow (230-250ms per frame at dawn/dusk, ugh.. its already hard enough!)

so set your camera to a really high framerate and let alprd handle dropping what it cant keep up with..
Giving it a 4MP feed and scaling it to 1080p was hardly discernible in performance from giving it a 1080p feed and no scaling.. so no point in giving alprd anything but your highest resolution possible, as it will save that image unscaled to disk

so lesson is, set your camera to max resolution and high frame rate from the get go.. and with mjpeg this is going to be some serious bandwidth, I am removing the camera from my nvr and going to configure it as a generic camera with only the 720p h264 rtsp for recording, so it dont have to deal with the massive bitrates of the main stream. (~4MB/s or ~33Mbit)

resolution was making a bigger hit than I had realized, was up to 700ms on my 4mp image.. backed it down to 720p scaling (still saves 4mp jpg) and I am at about ~120ms in the day, sometimes up to 150ms, others down to 100ms depending on lighting.. down to under 30ms at night.

thinking of trying the cuda libs, can get a GTX 650 for ~$60, I believe its TDP is pretty low.. lower than my cpu
 
Last edited by a moderator:
if the fonts are the same I guess it shouldn't really matter, what your really training is the font.. there is a pdf attached to that ticket that has a pure copy of the font, they were talking about printing out all the letters, photographing them back in as .jpg and running them through the binary font tools attached.. that seems like the best starting place, might not even need a bunch of sample plate images.

In that case maybe I can go get 36 high res images of each character and see how that goes.


submitted a pull request today that implements the pattern checking in alprd: https://github.com/openalpr/openalpr/pull/341

pattern checking on alprd is nice, the first result in the json array will be the first matching pattern, despite confidence.. so when it has highest confidence that: 63B-QR5 is correct, the pattern is invalid and 638-QRS will move to the top of the array.. the OSD display's accuracy went up with this patch.

It will be interesting to see how the system deals with personalised and out of state plates doing that. I suspect you might get no result in that case.
 
it still returns results if non match pattern, unless you enable must_match_pattern, it just bumps matches up to the top of the results if there are any.

and here are my patterns
postprocess/us.patterns:co ###@@@
postprocess/us.patterns:co @@####
postprocess/us.patterns:co @@@###
postprocess/us.patterns:co @@@####
 
it still returns results if non match pattern, unless you enable must_match_pattern, it just bumps matches up to the top of the results if there are any.

and here are my patterns

Good to hear

###@@@
@@@###
@@##@@
@@@##@

@@##
##@@
##@@@
@@###
@@####
@###
@####
#@@#@@
#@@@###

That pattern list covers most standard content Australian plates with the Top 10 patterns being common in my state. I can see having that many on the list causing some issues, and you would be better off only using the three or four most common patterns in your area. (The top four bold ones for my state)
 
Ok So today I have uploaded a little over 400 cropped plate images of the local front plates to the FTP site at the open ALPR project. With luck they can use them to train the detector better. Seems a bit hit and miss at the moment.



Mini Front ALPR correct read.pngMini Front ALPR misread.png
 
Last edited by a moderator:
  • Like
Reactions: nayr
for posterity here is a simple NodeJS app ...

I'm a software developer, aws+nodejs and big homelab guy. I'm new to the cctv world but I'm really interested setting up aplrd and what you have going on with it and node. Can you give me a quick snapshot of how you have aplr running? Thanks.
 
got OpenCL (Nvidia CUDA) working with ALPRD, and its way fucking fast..

full 4MP resolution black and white image on a Zotac GTX 950 OC 2G was really loading the video card down but managing ~250ms per frame...

with my mask I can do full 4MP now in realtime, day or night.. no more missed plates I expect :)

black and white 4MP Video w/Mask (my worst case, used to be ~250ms @ 720p)
Code:
$ nvidia-smi Wed May 18 23:13:56 2016       
+------------------------------------------------------+                       
| NVIDIA-SMI 352.39     Driver Version: 352.39         |                       
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950     Off  | 0000:01:00.0     Off |                  N/A |
| 38%   51C    P0    43W / 141W |     54MiB /  2047MiB |     41%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0     30951    C   alprd                                           43MiB |
+-----------------------------------------------------------------------------+

alprd:
Code:
# alprd -f --clock
INFO - Running OpenALPR daemon in the foreground.
INFO - Using: /etc/openalpr/alprd.conf for daemon configuration
Missing config value for company_id
INFO - Using: /home/alprd/plateimages/ for storing valid plate images
INFO - country: us -- config file: /etc/openalpr/openalpr.conf
INFO - pattern: co
INFO - Stream 1: http://ptz:ptz@192.168.42.26/axis-cgi/mjpg/video.cgi
Loading detector mask: /etc/openalpr/mask.jpg


Use of OpenCL LBP detector selected in config file.
OpenCL device(s) found:
0 GeForce GTX 950 (OpenCL 1.2 CUDA)
INFO - Starting camera 1
INFO - Video stream connecting...
INFO - Video stream connected
INFO - Camera 1 processed frame in: 132.747 ms.
INFO - Camera 1 processed frame in: 54.1756 ms.
INFO - Camera 1 processed frame in: 58.727 ms.
INFO - Camera 1 processed frame in: 51.7811 ms.
INFO - Camera 1 processed frame in: 48.6139 ms.
INFO - Camera 1 processed frame in: 51.0526 ms.
INFO - Camera 1 processed frame in: 52.1215 ms.
INFO - Camera 1 processed frame in: 49.1735 ms.
INFO - Camera 1 processed frame in: 57.2218 ms.
INFO - Camera 1 processed frame in: 49.6315 ms.

at night with all black image its just tearing through it at full framerate and using 28W & 7% GPU
Code:
nvidia-smi Thu May 19 01:32:23 2016       
+------------------------------------------------------+                       
| NVIDIA-SMI 352.39     Driver Version: 352.39         |                       
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950     Off  | 0000:01:00.0     Off |                  N/A |
| 38%   45C    P0    28W / 141W |     53MiB /  2047MiB |      7%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0     31160    C   alprd                                           42MiB |
+-----------------------------------------------------------------------------+

I choose this card because it was affordable and power efficent, you could proabibly get pretty decent power consumption with a specific built machine.. a nuc could probably do pretty good for lower resolution.
 
Last edited by a moderator:
Interesting. Were you using OpenCL with Intel graphics before, or no OpenCL at all?

Edit: Hmm. It didn't enumerate any Intel graphics devices, so I'm guessing no.