Full ALPR Database System for Blue Iris!
- Thread starter algertc
- Start date
TheWaterbug
Known around here
I nuked the site from orbit (it's the only way to be sure), and started over. I'm pretty sure the incorrectly-created database was a problem, because as soon as the database engine was running, the dashboard looked like this:
instead of how it looked previously:
when it was trying to read tables/columns that didn't even exist.
And this was even before I reinstalled the AI engine and configured BI to use it.
So now I wait for vehicles, again . . . .
I'm almost tempted to drive back into work, just to trigger the cameras with a plate. Almost.
TheWaterbug
Known around here
Vettester
Getting comfortable
- Feb 5, 2017
- 918
- 901
I have copied and pasted this so many times . . . .
You have this in front of the key, right? "x-api-key:"
TheWaterbug
Known around here
Vettester
Getting comfortable
- Feb 5, 2017
- 918
- 901
TheWaterbug
Known around here
I have a plate!!!!!
Nuking the database and starting over was the last required fix. Thanks to everyone for all the help! I certainly would not have figured it out on my own. For anyone new to the thread, about 1/4 of my problems were due to following an out-of-date ReadMe.MD that had an obsolete docker-compose.yml, 1/2 were user error arising from not understanding the workflow and what other settings needed to be enabled, and the last 1/4 were user error from being stupid. I should probably compile my notes into a complete step-by-step on how to reinstall all of this if I ever need to.
Nuking the database and starting over was the last required fix. Thanks to everyone for all the help! I certainly would not have figured it out on my own. For anyone new to the thread, about 1/4 of my problems were due to following an out-of-date ReadMe.MD that had an obsolete docker-compose.yml, 1/2 were user error arising from not understanding the workflow and what other settings needed to be enabled, and the last 1/4 were user error from being stupid. I should probably compile my notes into a complete step-by-step on how to reinstall all of this if I ever need to.
Two days ago, I migrated all my image data. The earliest images were from June 2024 and it all showed correctly in the folders (2024/06, 07, etc., 2025/01). I confirmed everything was showing correctly in the program so went along and removed the Base64 data.
This morning I woke up to find that all images prior to 12/1/2024 are deleted.
I figured out why it happened - there's a new setting for image retention which is set to 3 (months). First, the setting can't be changed (it says, 'Saved' but it refreshes back to 3.) and second, that's not actually 3 months of images. It's all of 12/2024, all of 1/2025, but only 1 day of 2/2025. It really should still have most/all of 11/2024. I've added both bugs to the roadmap/tracker.
That aside, I'm now faced with a ton of missing images. I was intending to keep at least a year (or more) of plates. They don't appear in my recycle bin, so that's not an option. I do regular PC backups but the latest backup was this morning at 3:45, so after the deletion. And the other was just before I did the update and database migration on 1/30.
At this point, my best option seems to be to go to the backup before and just lose the plates from the last couple of days. I don't really want to restore the whole PC so does anyone know how to do a partial restore of just Docker? Specifically, where is Docker stored? What files constitute Docker? I didn't do any backups of containers with Docker but I'm fine with just wiping it completely back to the state of a couple days ago. Anyone know how to do that?
Edit: I think the answer is C:/Users/<username>/AppData/Local/Docker/wsl/disk/docker_data.vhdx
I'm in the process of restoring this right now.
Edit 2: That did put Docker back to the state on 1/30. I manually updated config/settings.xml to increase the retention period and then pulled the latest and reran the migration. I have all my images back (except for about 24 hours). Since, I have copies of that data from my notifications, I can manually call the API to restore the license plate reads. I highly recommend people update their image retention setting (manually) if they don't want to lose images. And always have a backup strategy in place before you need it.
This morning I woke up to find that all images prior to 12/1/2024 are deleted.
I figured out why it happened - there's a new setting for image retention which is set to 3 (months). First, the setting can't be changed (it says, 'Saved' but it refreshes back to 3.) and second, that's not actually 3 months of images. It's all of 12/2024, all of 1/2025, but only 1 day of 2/2025. It really should still have most/all of 11/2024. I've added both bugs to the roadmap/tracker.
That aside, I'm now faced with a ton of missing images. I was intending to keep at least a year (or more) of plates. They don't appear in my recycle bin, so that's not an option. I do regular PC backups but the latest backup was this morning at 3:45, so after the deletion. And the other was just before I did the update and database migration on 1/30.
At this point, my best option seems to be to go to the backup before and just lose the plates from the last couple of days. I don't really want to restore the whole PC so does anyone know how to do a partial restore of just Docker? Specifically, where is Docker stored? What files constitute Docker? I didn't do any backups of containers with Docker but I'm fine with just wiping it completely back to the state of a couple days ago. Anyone know how to do that?
Edit: I think the answer is C:/Users/<username>/AppData/Local/Docker/wsl/disk/docker_data.vhdx
I'm in the process of restoring this right now.
Edit 2: That did put Docker back to the state on 1/30. I manually updated config/settings.xml to increase the retention period and then pulled the latest and reran the migration. I have all my images back (except for about 24 hours). Since, I have copies of that data from my notifications, I can manually call the API to restore the license plate reads. I highly recommend people update their image retention setting (manually) if they don't want to lose images. And always have a backup strategy in place before you need it.
Last edited:
Just want to let everyone know I’m coming back to this very soon. Been slammed with work recently, but should be able to get a bunch of stuff worked out at the end of this coming week + the weekend.
Seems like there’s good advice being given to work through issues, but apologies to anyone having them in the first place. My focus in the next update is going to be making the installation and user experience as streamlined and simple as possible.
It’s always a bit challenging to release updates to software that is self hosted like this, so I’m doing my best and learning from some mistakes along the way. Web dev is more of a hobby for me, so I’m certainly not the most principled nor super experienced with this sort of release structure.
Still happily taking feature requests on the roadmap page also. Will get to them all soon along with reading back through the forum posts.
Thanks to everyone for trying it out! I’m glad it’s been useful.
Seems like there’s good advice being given to work through issues, but apologies to anyone having them in the first place. My focus in the next update is going to be making the installation and user experience as streamlined and simple as possible.
It’s always a bit challenging to release updates to software that is self hosted like this, so I’m doing my best and learning from some mistakes along the way. Web dev is more of a hobby for me, so I’m certainly not the most principled nor super experienced with this sort of release structure.
Still happily taking feature requests on the roadmap page also. Will get to them all soon along with reading back through the forum posts.
Thanks to everyone for trying it out! I’m glad it’s been useful.
TheWaterbug
Known around here
Please don't apologize for making free software!!!!! This is a great project. My neighborhood just got Flocked, and I'm very curious to see how well their very expensive system stacks up to yours.
Glad you like it
Yes it’s free, but I do still feel a bit of a sense of responsibility to at least not break other peoples installations. It’s not a chore for me at all. I wouldn’t do it if it were. It erks me on behalf of others when the process doesn’t work as intended.
It is pretty crazy how much companies charge for the fully integrated ALPR solutions compared to what we have here. There’s a thread on here somewhere about the recent vulnerability in the Motorola system. I find the CPAI model to work quite well and that would be thanks to @MikeLud1
Last edited:
Thank you so much for this approach and way of thinking! I believe everyone here truly appreciates it.
For updates where things might go wrong, would it be possible to tag Docker releases for easier rollbacks, or would that be too much effort?
Yeah I’ll do something and make sure it’s clear. I was thinking sort of by the same token, it might be nice to show some system info in the dashboard too. Storage usage, maybe a projection of how much storage you will use based on your traffic level, errors, users + their last login, etc.
I think I’ll keep the main version numbering the same and start tagging the docker images with the associated version number (this part is for a full update release). Then, also do something like a development branch which can be published as a nightly build, which is basically a pre-release potentially unstable version with changes soon to be in the main image.
That should let people roll back/choose specific versions if needed while also not messing with the stable image for any new users.
When I get to making an install and update script, I’ll try to add a rollback option too. I don’t think there would be any way to do it from inside of the app.
I’m gonna see if I can ditch the whole process of copying the docker compose and manually creating directories and turn the install process into a single command that will go get the files, show some pretty ascii art, ask you a few questions for how you want to install, then just take care of everything.
HomeAssistant app/integration or whatever it’s called should be feasible pretty soon. I’ll look into unraid also.
I’ll see if I can make a windows installer at some point too if people would want that.
Thats absolutely nuts that you had to restore the whole computer. Glad you were able to get them back, but wow that was a definite oversight. I’m gonna go ahead and fix that one right now.Two days ago, I migrated all my image data. The earliest images were from June 2024 and it all showed correctly in the folders (2024/06, 07, etc., 2025/01). I confirmed everything was showing correctly in the program so went along and removed the Base64 data.
This morning I woke up to find that all images prior to 12/1/2024 are deleted.
I figured out why it happened - there's a new setting for image retention which is set to 3 (months). First, the setting can't be changed (it says, 'Saved' but it refreshes back to 3.) and second, that's not actually 3 months of images. It's all of 12/2024, all of 1/2025, but only 1 day of 2/2025. It really should still have most/all of 11/2024. I've added both bugs to the roadmap/tracker.
That aside, I'm now faced with a ton of missing images. I was intending to keep at least a year (or more) of plates. They don't appear in my recycle bin, so that's not an option. I do regular PC backups but the latest backup was this morning at 3:45, so after the deletion. And the other was just before I did the update and database migration on 1/30.
At this point, my best option seems to be to go to the backup before and just lose the plates from the last couple of days. I don't really want to restore the whole PC so does anyone know how to do a partial restore of just Docker? Specifically, where is Docker stored? What files constitute Docker? I didn't do any backups of containers with Docker but I'm fine with just wiping it completely back to the state of a couple days ago. Anyone know how to do that?
Edit: I think the answer is C:/Users/<username>/AppData/Local/Docker/wsl/disk/docker_data.vhdx
I'm in the process of restoring this right now.
Edit 2: That did put Docker back to the state on 1/30. I manually updated config/settings.xml to increase the retention period and then pulled the latest and reran the migration. I have all my images back (except for about 24 hours). Since, I have copies of that data from my notifications, I can manually call the API to restore the license plate reads. I highly recommend people update their image retention setting (manually) if they don't want to lose images. And always have a backup strategy in place before you need it.
I think you’re one of the only people who backfilled recognitions from BI so hopefully pointing that out will save some people who wouldn’t have noticed until much later.
I just went back and looked at the posts from you and Mike with the code to send the images. It looks like you downloaded them first and labeled them - let me know if I’m wrong.
If anyone would be interested in adapting that code to try to get the alerts with the plate number/memo directly from the Blue Iris API (instead of manually downloading/labeling) and filling them into the database, that could be a great thing to add to the repository. I can integrate it into a page in the app too.
I’m gonna add the missing stuff to the readme also while I’m at it.
It would probably be good to make the whole thing a bit more clear.
I’m gonna add this image that @MikeLud1 sent + one showing the recording tab with jpegs enabled since a few people have been tripped up there.
A common roadblocks / troubleshooting section would be good to have also detailing BI configuration stuff or AI issues.
It would probably be good to make the whole thing a bit more clear.
I’m gonna add this image that @MikeLud1 sent + one showing the recording tab with jpegs enabled since a few people have been tripped up there.
A common roadblocks / troubleshooting section would be good to have also detailing BI configuration stuff or AI issues.
It wasn't a complete restore. It was the single file which is essentially the Docker image. The database on that date had all the images embedded in the base64 data, so I just had to rerun the migration to get the files back.
When I originally imported my old data, it was from my email notifications and I already had that method in place so it was easy to do just one additional day of recent plates that were missed.
But yes, if there were a way to pull that from BI, that could help others backfill their older records.
Ah ok I see, not quite as terrible. Clever fix!
I’ll try to add a simple backup option that dumps the plate_reads table (live feed) and zips the image directories then lets you download. You could also do this manually if you wanted to by copying the files from the container + creating a backup using pgadmin.
I think I’ll also add a confirmation step when changing the retention values since it’s a destructive action. Thanks for your suggestions also.
A general FYI about how the deleting/pruning works: the actual deletion is triggered by the plate-reads api route, so it’s not instant once you change the value in the settings. Next time you receive a plate, it will trigger the cleanup jobs with the whatever the current config value is at that time. i.e. if you quickly change it then change it back and no traffic goes by, nothing will happen. The max records prune also waits until the current value is 110% of your maximum value to avoid running the cleanup constantly.