Reading multiple RTSP streams synchronously

[LuBo]

Hello,

I am working on a project that involves reading frames from 8 IP cameras (all Dahua, but different models/firmwares) and bundling them into a packet for further processing.
If I simply read the incoming RTSP streams frame by frame with OpenCV's VideoCapture, they are out of sync by ca. 2 seconds. Getting the NTP timestamps out of the RTSP streams does not seem trivial (FFMPEG's libavformat provides some internal headers, but it's a mess to use them).

I was wondering, if there is any NVRs/DVRs that perform synchronization between several camera streams and provide any means to read the synchronized streams from software (Python, C++)?

Any ideas on this? Using "normal" IP surveillance cameras is a requirement, so I can not use any cameras with external trigger.

 

[TonyR]

What is your goal and how and where are these streams to be viewed?

 

[LuBo]

This "acquisition" module is part of a larger software project. Goal of the acquisition module is to provide an API with a method (get_frame_packet) that returns a packet of recent (say, not older than 500 ms) frames which are taken at the same point in time on camera side (say, within +- 1/fps seconds). This packet is then used for further analysis (object detection & tracking). In fact, I am also doing some more advanced stuff on the encoded stream, such as exporting H.264 motion vectors for every frame. But this can be more or less decoupled from the synchronization issue.

I am searching for a solution (software library or standalone software) which reads multiple streams, synchronizes them and provides output which I can further process with my own library. This output could be
1) Independent H.264 encoded RTSP streams. Of course, there is no guarantee here for synchronicity but since they come from the same physical sender...
2) Some return value of a library method which contains frames of all streams which are taken at the same time.

I was just wondering, if there is any obvious solution to this problem. Since, it should not be too uncommon to require synchronous streams.