API for event stream (such as motion detection)

I'm developing a new NVR system. But it doesn't support motion detection yet, which is a must-have feature. I'd like to support on-camera motion detection because it should be possible without powerful NVR hardware, and it should be relatively easy for me to develop. Well, that's the theory...I'm struggling though.

Does anyone know a reliable way to get a stream of events (such as motion detected) from Dahua cameras?

I've done some research and have found four ways that "should" work:

1. ONVIF PullPoint Notifications
2. ONVIF Basic Notifications
3. ONVIF Metadata Streams
4. Dahua proprietary HTTP API

But as far as I can tell, they all have problems.

PullPoint Notifications

The camera supports up to five active subscriptions. You set one up with the CreatePullPointSubscription method, then use it with the PullMessages method (repeatedly), then tear it down with the Unsubscribe method. This does work...sort of.

The problem is that if my software crashes, I basically leak the subscription. If that happens five times, I have to reboot the camera before I can use it again. (Or track the URL to unsubscribe...but that's a whole mess because the URLs reset when the camera reboots, so some other client might be using a subscription with the same URL, etc.)

I want to build something rock-solid. Of course, I prefer my software / the hardware not crash, etc., but if it does, I want it to come back with no fuss, without leaking resources, etc.

ONVIF has a timeout associated with a subscription to address this problem. Once the timeout passes, the subscription goes away. If I could just keep the timeout only a minute or so in the future, there wouldn't be a problem. But...Dahua apparently implemented it wrong. If my PullMessages request is active for 11 seconds, the timeout advances 110 seconds. So before long it's days and days in the future, and there might as well not be a timeout. Grr.

ONVIF Basic Notifications

This approach requires the camera connect to your software. That's dumb, as described here. In my firewall setup, the cameras can't connect to anything, and I'd like to keep it that way.

ONVIF Metadata Streams

This seems to work fine with Hikvision cameras, but not Dahua ones. I can't seem to set up the stream to have anything useful in it. Detailed notes here[edit: fixed link; it went to the wrong thread before].

edit: actually, it does have motion in it! but I'd still like to get the SetMetadataConfig method to work so I can customize things. Grr.

Dahua proprietary HTTP API

I found a doc called DAHUA HTTP API FOR IPC Version 1.67, and I own a Synology NAS that appears to use this API, urls such as /cgi-bin/eventManager.cgi?action=attach&codes=[VideoMotion].

edit:

with firmware 2.622.0000000.18.R, this doesn't work at all. My Synology device doesn't work either, and there are forum posts about this.

with firmware 2.622.0000000.31.R, it sort of works. The big thing I need is a keepalive—making it send something every N seconds even when there's no event, so I can tell if I lost the connection to the camera or not. I see mentions on the Internet of a keepalive URL parameter, but it doesn't seem to do anything for me.

fwiw, Synology's setup has a dropdown for firmware versions:

• < FW V2.460.0000.10.R
• >= FW V2.460.0000.10.R
• >= FW V.640.0000002.0.R

so Dahua must have made a few significant API changes.

edit again: hmm, if I request event type All, as in:

curl -v --digest "http://$USER:$PASSWORD@$HOSTNAME/cgi-bin/eventManager.cgi?action=attach&codes=[All]"

it includes updates on time changes. With a 1-minute interval, that might work as a keepalive. My motion detection connection will drop if NTP is misconfigured or the server is down, which is not great, but it's better than nothing...

Why don't you use synology's surveillance station to detect movements? It also has a API to send notifications, alarms, etc.
My old DS412+ handles 7 streams right now (2x1080p,5x720p) without high cpu load.

I'm writing my own NVR. I don't want it to depend on someone else's, even if I happen to have one available at the moment.

slamb said:

ONVIF has a timeout associated with a subscription to address this problem. Once the timeout passes, the subscription goes away. If I could just keep the timeout only a minute or so in the future, there wouldn't be a problem. But...Dahua apparently implemented it wrong. If my PullMessages request is active for 11 seconds, the timeout advances 110 seconds. So before long it's days and days in the future, and there might as well not be a timeout. Grr.

Click to expand...

I don't know anything about this API or what PullMessages exactly does and how it's used but how about pro-actively unsubscribing and immediately re-subscribing after N PullMessage requests (or after having it active cumulatively for X seconds)? Wouldn't that reset the timeout to what you set it to when you (re-)subscribe?

SecuritySeeker said:

how about pro-actively unsubscribing and immediately re-subscribing after N PullMessage requests (or after having it active cumulatively for X seconds)? Wouldn't that reset the timeout to what you set it to when you (re-)subscribe?

Click to expand...

Thanks for the suggestion! Yeah, I could do that. Even better, every PullMessages response has the current time and timeout time (see an example at B.6.2.2 in this pdf), so I can subtract and tell when the timeout is getting too far out. But every un/re-subscribe would be a moment where I could miss an event. I could establish a new one, tear down an old, and so on, so I have 1 or 2 subscriptions active rather than 0 or 1. But that's complex and de-duplicating events would be kind of fuzzy (there's no unique id for every event or anything), so I'm not thrilled about it...

(I wish Dahua would fix their timeout advancing logic, but I don't think I have any way of getting a bug report to someone who would actually do anything about it.)

slamb said:

(there's no unique id for every event or anything), so I'm not thrilled about it...

Click to expand...

The pfd you linked to mentions a SubscriptionReference (and also a SubscriptionID):

First, a subscription is created and a subscriptionReference is returned, which is used in PullMessages requests to fetch the actual event notifications.

Click to expand...

If that works how I imagine it does it would at least be possible to know if an event is from the old or the new overlapping subscription. Perhaps you can try subscribing twice and see if you can compare the contents of an event message that has come in through both subscriptions to see if they are equivalent (if not identical) if receiving and processing an event twice is a problem.

SecuritySeeker said:

I don't know anything about this API or what PullMessages exactly does and how it's used but how about pro-actively unsubscribing and immediately re-subscribing after N PullMessage requests (or after having it active cumulatively for X seconds)? Wouldn't that reset the timeout to what you set it to when you (re-)subscribe?

Click to expand...

Right, when I receive a message I know what subscription it comes from. The lifecycle is something like this:

1. I create a subscription via CreateSubscription. It gives me a URL.
2. (in a loop) I call PullMessages on that URL, so it returns messages only associated with that subscription. (And hopefully the URL still represents my subscription. If the camera has rebooted, it might be someone else's subscription now.)
3. (hopefully, if I don't lose power, crash, etc.) I send an Unsubscribe to that URL. (And again hopefully it's still my subscription.)

The de-duplication would be between the two subscriptions. And yes, probably events would be equivalent but not identical. It'd have to be a little fuzzy. For example, they have timestamps, but it wouldn't surprise me at all if the two subscriptions gave different timestamps for the same event. The API implementations on these cameras are crap, unfortunately. (I'm not real fond of the ONVIF specification either, for that matter. The multipart/mixed streams are much simpler in that they don't require any state to be kept between HTTP requests, but ONVIF is based on SOAP, and SOAP doesn't support multipart/mixed.)

I did get Dahua's proprietary API to mostly work (see my edits to my initial post). I think I'll first see how well I can get that to work, and then come back to this multi-subscription de-duplication approach if necessary. Thanks again for the suggestion!

@slamb

Thanks for working on this. Check out the work that psyciknz has done with Dahua too. I've have created my own Dahau monitoring tool and a recent firmware update seems to have broken the eventManager API. I'm trying to find a solution.

I adapted the work of @johnnyletrois into AblazeWoland/dahua-ivs-watcher, you can check out the code on how to poll (IVS) events there. If you figure out how to do PushPoint notifications in e.g. Python, I think quite a few people here and elsewhere would appreciate if you published the code. I haven’t found anything in the wild so far, but the polling mechanism that I think was first published in still works. Until we have the firmware update @pcunite mentions, maybe.

@slamb
This is the latest API doc I have seen. The one you mentioned may be better.

A few thoughts:

Android TinyCam Pro receives on-camera motion detect alerts and stores video.

Perhaps you could set up camera for motion detect records to be sent via FTP or NAS, and do what you want when the records arrive.

Here's a thread you might find interesting on motion detection.

Update:
I reset the camera and that has fixed the eventManager API. I guess that is a requirement whenever doing so.

It’s hit or miss, IMO. I don’t do a reset normally but if something starts to act odd then I will.