MQTT payload issues (jpeg images) from Blue Iris. Maybe a Post issue?

[thewolf56]

A while back, I tried to setup Blue Iris and Home Assistant through the Blue Iris Integration in Home Assistant and MQTT to send me notifications through Home Assistant with the Blue Iris alert image attached. I had no success at the time, so I settled for setting up binary sensors (some with AI required objects) and simple payloads of "on". This worked perfectly and I was able to setup Home Assistant to take it's own snapshot of the image for the notification, but with the processing delays, this snapshot was not always accurate as to what actually triggered Blue Iris.

I recently started trying to get the Blue Iris Alert Image working again. Yesterday, I learned that a possible source of my issue was that the POST tab of my camera never had Post JPEGs set, so I set it and enabled the test automation in Home Assistant for the notification with the BI Alert Image.

An Amazon delivery driver came into my courtyard and I received a notification through Home Assistant with the Blue Iris alert image attached with the AI identification boxes on the jpeg. Great news. then my wife went to the courtyard to get the package and I got a new notification with the JPEG attached, only it was the same Blue Iris alert image with the Amazon driver. I've tried checking various settings, but even just using the testing section of the Alerts, I still only get the Amazon delivery driver photo.

I double checked the folder where Home Assistant stores the snapshot. The metadata for the photo gets updated as a new file is saved before being attached to the notification, but it's still the same image. I even tried changing the automation so that it saves the payload jpeg as a different name, but even the newly-named image has the same image.

Here is my On Alert settings

MQTT topic: BI/courtyard_camera/alert-image-b64
Post/payload: { "camera":"&CAM", "time":"%Y%m%d_%H%M%S", "dbid":"&ALERT_DB" }

Here is my Post settings for the camera



I would love to figure this out as there are a few cameras I would really like to add the BI alert images to the notifications. Any help is appreciated

 

[Jacoob]

Did you manage to get it working? I have it working on two cameras, but a third is doing exactly the same as you described. They are all set up identially so I'm really not really sure what else to try!

 

[thewolf56]

No, it is still not working. I've tried posting in a few different places, but still no help has been forthcoming. I did check the MQTT logs when trying to test and it was an error I've never seen before. I'm away from my computer right now, but I'll see if I can dig it up.

 

[The Automation Guy]

I experience the same issue with a camera that uses the pushover app to send me notifications of alerts. I will get the alert correctly, but the image attached is usually the previous alert's image (which might be hours or even days earlier). Rarely the image will be the same for multiple alerts before eventually changing to a newer (but not the correct) image. I think this happens when several alerts occur within a very short period of time and when this happens all those alerts have the same incorrect image attached. I don't use MQTT for this however which really points to the problem lying somewhere within the trigger/alert/notification setup (or underlying bug within this section of programming code) and not specifically with the communication method being used (ie MQTT or Pushover). I haven't been bothered to track the issue down myself, but perhaps that information can help point you in the right direction.

The strange thing is that my system worked reliably for a long time before starting to send out the wrong image. To my knowledge, nothing changed in my camera setup. I've surely upgraded BI over time, but the problem did not begin right after an update (that would have been an obvious red flag). It really seemed like the system worked correctly one day and then started to attach the wrong image to the notification the next day.

 

[thewolf56]

I found the error that was posted in MQTT

Exception in message_received when handling msg on ‘BI/courtyard_camera/alert-image-b64’: ‘b’{ “camera”:“Courtyard”, “time”:“20231102_105157”, “dbid”:"" }’’ Traceback (most recent call last): File “/usr/src/homeassistant/homeassistant/components/mqtt/debug_info.py”, line 43, in wrapper msg_callback(msg) File “/usr/src/homeassistant/homeassistant/components/mqtt/camera.py”, line 110, in message_received self._last_image = b64decode(msg.payload) ^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/base64.py”, line 88, in b64decode return binascii.a2b_base64(s, strict_mode=validate) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ binascii.Error: Invalid base64-encoded string: number of data characters (37) cannot be 1 more than a multiple of 4

 

[thewolf56]

So, I tried a different MQTT topic and payload that I had read as a possibility and it still didn't work. After a few days, it started working out of nowhere and even tried it on a 2nd camera (this time, it only took a couple of hours for it to work)

What worked this time:


Home Assistant:
I use a mqtt.yaml file, so it would be slightly different if you do everything in the config.yaml file

camera:
- topic: BlueIris/image/"Camera's Short Name in Blue Iris"
  name: Camera Name Alert MQTT Image
  unique_id: alert_camera_image
  image_encoding: b64

I'm still not sure what happened, but it has been working for over a week now. I have not upgraded Blue Iris at all since the issue started, but I have upgraded Home Assistant 2023.12.3 and HA's Mosquitto Broker add-on. The funny thing is, I only noticed that it started working because there was a picture in the Overview dashboard (I never use it but keep it around just in case) for the MQTT camera for the alert when I was looking for another recently-added device I was working on. I checked if the picture changed if I triggered it through Blue Iris (physically and not through the alert testing) and it did change. I disabled the home assistant action for the Home Assistant snapshot for notifications and tried the new one and it worked.

 

[TJ Martin]

Is there an automation you need to run in order to get the notifications sent to your phone? I'm currently using the snapshot automation and it is sending a picture notification but it seems the image is either before or after the trigger object. MQTT is setup and running, I just don't know how to retrieve it.

 

[thewolf56]

You do need an automation to take a snapshot of the MQTT camera and save it local to Home Assistant in a folder as one of the first actions in the automation and another action in the automation to actually send the jpeg to your phone.

The way mine is setup, there are a few moving parts and I'm sure there's a better, more efficient way to do it, but I could never get it working following the tutorial to the letter. I'll try to get on a PC tomorrow and post how my Home Assistant and my Blue Iris are setup so that it works in my system.

By the way, before I got it working, I did use the camera snapshot directly from the camera to Home Assistant and it was always a split second after the picture that would pop up from blue iris on my phone, and because it was from the camera, it didn't have the object detection boxes. Now that the MQTT camera works (well enough), I get the same notification image in Home Assistant that Blue Iris sends as a notification.

 

[TJ Martin]

Thanks for your response. Your correct regarding the snapshot response. No motion squares and is delayed. Oddly enough I have an entity card for the b64 image working which does show the motion squares. I just can't seem to get it to post in the message. Currently I have it under data as entity: camera.driveway_alert. I'm responding by phone and going off memory.

 

[thewolf56]

@TJ Martin

Sorry, I didn't get to my PC over the weekend.

1st, in my Home Assistant configuration.yaml, I added the line:

mqtt: !include mqtt.yaml

After restarting Home Assistant, I created an mqtt.yaml file (/homeassistant/mqtt.yaml) and added this code (where my camera's Short name is "Courtyard" in Blue Iris, not the regular name). Using the short name was key and letter case matters.

camera:
- topic: BlueIris/image/Courtyard
  name: Courtyard Camera Alert MQTT Image
  unique_id: alert_courtyard_image
  image_encoding: b64
binary_sensor:
- name: "Courtyard Camera Person"
  device_class: motion
  state_topic: "courtyard_camera_person"
  payload_on: "on"
  payload_off: "off"
  value_template: "{{value_json.state}}"
  off_delay: 30

Where the camera is the jpeg image coming from Blue Iris and the binary_sensor is the alert that AI has detected a person in the courtyard.

In Blue Iris, I setup MQTT and tested that it connected to my Home Assistant without issue (there are better tutorials out there than from me on this topic). Then make sure your camera is actually set to post the jpegs.



For Alert Settings, I used this for the camera image



And this for the Person detection




In Home Assistant, create a folder named "Security_Cameras" inside the folder "/config/www" so that you have the location "/config/www/Security_Cameras" available

I then created this automation (This is prior to version 2024.8.0 as I have not upgraded to the new version, yet)

alias: Person Detected in Courtyard
description: ""
trigger:
  - platform: mqtt
    topic: courtyard_camera_person
    payload: "on"
condition: []
action:
  - service: camera.snapshot
    target:
      entity_id: camera.courtyard_camera_alert_mqtt_image
    data:
      filename: /config/www/Security_Cameras/Courtyard_Camera_Alert.jpg
  - service: notify.mobile_app_my_android_phone
    data:
      message: Person Detected in Courtyard
      data:
        ttl: 0
        priority: high
        image: /local/Security_Cameras/Courtyard_Camera_Alert.jpg
        tag: person-detected-courtyard
        color: red
  - service: notify.alexa_media_kitchen_echo_dot
    data:
      message: Person Detected in Courtyard
      data:
        type: announce
    enabled: true
  - service: notify.living_room_nvidia_shield_pro
    data:
      message: Person Detected in Courtyard
      data:
        duration: 10
        fontsize: large
        position: bottom-right
        color: blue
        interrupt: 1
        image: /config/www/Security_Cameras/Courtyard_Camera_Alert.jpg
    enabled: true
mode: single

The first action takes a snapshot from the mqtt camera we created and saves the picture as "Courtyard_Camera_Alert.jpg" in the "/config/www/Security_Cameras" folder. This file gets overwritten every time there is a new alert.

The second action then sends a notification to my android phone with the image snapshot we had just saved in the previous action. I use the tag: so that it overwrites previous notifications containing that exact tag.

The third action I use to notify to my alexa in my kitchen that uses TTS to announce that someone is in the Courtyard.

The fourth action I left in the code as I use it to also have a pop-up on screen in my living room Nvidia Shield Pro with the Blue Iris alert image as well. I mainly use PLEX, but also use Netflix and Disney+ and it will pop-up the image like it is a PIP.

I actually took out more actions that this automation does that are conditional (also additional notifications to my wife).

I hope this helps.

 

[TonyR]

^^^^ Great!
Thanks for sharing that!

 

[TJ Martin]

Wow. This is a great write up and totally appreciated. I'm responding via mobile phone and will try this tonight. This is the first I've seen in any tutorial regarding posting from BI. I've also never seen your code that your using for MQTT. It's probably these reasons I couldn't get it to work most of the time. I did find a work around using Telegram. BI is posting directly to there and skipping Home Assistant but the issue remains where 85% of the images are to late like someone already walked by and it took a picture of an empty driveway. Does your way catch someone as expected and are you running AI, if so does it show the AI boxes? Lol. So many questions. Thank you so much for this great write up.

 

[thewolf56]

^^^^ Great!
Thanks for sharing that!

No problem. There are other tutorials out there, but never worked for mine. Had to work this one out on my own and may be missing some stuff.

 

[thewolf56]

Wow. This is a great write up and totally appreciated. I'm responding via mobile phone and will try this tonight. This is the first I've seen in any tutorial regarding posting from BI. I've also never seen your code that your using for MQTT. It's probably these reasons I couldn't get it to work most of the time. I did find a work around using Telegram. BI is posting directly to there and skipping Home Assistant but the issue remains where 85% of the images are to late like someone already walked by and it took a picture of an empty driveway. Does your way catch someone as expected and are you running AI, if so does it show the AI boxes? Lol. So many questions. Thank you so much for this great write up.

No problem. I was very frustrated when I could not get it to work for me. I tried using payloads that had a lot more information in them using templates, but I had to just try and copy and hope they worked (they didn't). Now that I have started using templates more in Home Assistant (and actually learning how to change things), I may try and use the templates again that contain more information in the payload.

Does your way catch someone as expected

Yes, it does.

and are you running AI, if so does it show the AI boxes?

Yes and yes. The thumbnail for the Blue Iris Alert Clip (in the Blue Iris program) typically matches the image that is sent to me through the Home Assistant notification, including the AI detection boxes. The AI detection boxes were what gave it away that what I had done actually worked.

The jpeg I have right now loaded in Home Assistant is not a great example. AI is at a slightly less than 300 ms processing time and I have tripwire-type zone for breaking the plane of the courtyard entrance (and the small path to it in front of my 1-car garage door), so if someone walks up the driveway in front of my 2-car garage and places a package over the pony wall of the courtyard (like the USPS guy that is my current alert image), the motion doesn't trigger until their hand breaks the plane at the wall and the image shows their back. I have other cameras that will see the approach before a person gets to the courtyard, but I need to get a GPU for the AI at some point to help with delay.

 

[tagmp]

Thanks so much thewolf56 for a detailed walkthrough! With the recent outage on the app I followed your instructions and am able to get alerts now. I don't have images pushing through. The /www folder isn't a thing on the HA OS for Raspberry Pi's apparently, and I can't seem to figure out how to get MQTT to store the pushed images anywhere accessible. Probably a permissions thing, I haven't tried the media folders yet. Nonetheless, I really appreciated this!