How to: Use VLC to transcode IP video to an HTML5-compatible format.

bp2008

bp2008

Staff member
IPCT+ Member
Mar 10, 2014
12,783
14,319
USA
For a long time you've been able to stream camera video to a browser, but this has almost always involved using inefficient JPEG video or a plugin capable of actual video decoding.

But last week I discovered that right out of the box, VLC media player can produce and serve an HTML5-compatible video stream.

This works by having VLC transcode the video to a format the browser understands. The transcoding has to happen in real time, so a fast computer is needed.

Step 1

Run VLC with the following command. Modify the RTSP path as needed to point at one of your own cameras:

Code: 
vlc.exe rtsp://192.168.0.53/ :network-caching=1000 :sout=#transcode{vcodec=theo,vb=1600,scale=1,acodec=none}:http{mux=ogg,dst=:8181/stream} :no-sout-rtp-sap :no-sout-standard-sap :sout-keep

A VLC window should appear and the playback clock should begin incrementing, but no video will be shown since the command specifies the only output is http.

Step 2

Now you need a web page with an html5 video element on it. Just create an empty text file, paste the following line in it, and name the file video.html

HTML: 
<video id="video" src="http://localhost:8181/stream" type="video/ogg; codecs=theora" autoplay="autoplay"/>

Open this html page with a browser. It should immediately load the live video stream. No browser plugins required.

Some notes:

  • This method works by transcoding the video with the open/free "Theora" codec, wrapping it in the "ogg" container type. This transcoding is rather CPU intensive for high resolution video, but it is necessary for browser compatibility.
  • I only got this working in Chrome and Firefox. Internet Explorer does not support Theora video natively. Some mobile browsers don't support it either.
  • In theory most browsers should work with H.264 video in the MP4 container. In fact, in theory, this is the most compatible format around. But I was not able to make this work no matter what I tried.
  • You can tweak a lot of things in the command that runs VLC. For example if you change network-caching to a lower number, you get less video latency. But if you set it too low it may not work at all, or may cause problems if your network becomes congested. The vb=1600 argument specifies the bit rate (you can put whatever you like in there, too). You can even change the scale of the video or add width and height arguments to tweak the resolution. You can also change the output destination (:8181/stream) to change how VLC will listen for connections.
 
  • Like
Reactions: pcunite, handinpalm, newfoundlandplucky and 6 others
Thanks for answer. I tried to use another port instead of 8181 it doesnt allow that? My code is


vlc.exe rtsp://192.168.0.53/ :network-caching=1000 :sout=#transcode{vcodec=theo,vb=1600,scale=1,acodec=none}:http{mux=ogg,dst=:1910/stream} :no-sout-rtp-sap :no-sout-standard-sap :sout-keep

I got connection refused. Only port i can use is 8181?
 
@Ehtisham I tried to use that stream but it keeps returning http code 401 which means I am not authorized to view it. However once it did work just for no reason at all that I can tell. I couldn't make it happen again.

Anyway, to use the method I described four years ago you would need to put the remote stream address into the vlc command like this:

Code: 
vlc.exe http://ksportiptv.com:1557/SajidLhr/Cj6cZT2p6V/1345 :network-caching=1000 :sout=#transcode{vcodec=theo,vb=1600,scale=1,acodec=none}:http{mux=ogg,dst=:8181/stream} :no-sout-rtp-sap :no-sout-standard-sap :sout-keep

And then create the html file exactly as I described without replacing anything.

Code: 
<video id="video" src="http://localhost:8181/stream" type="video/ogg; codecs=theora" autoplay="autoplay"/>

It doesn't work 100% perfectly, and the vlc command I wrote above doesn't include an audio stream. So if you really need to view this in a web browser then you will have work to do.
 
Paul Haysom said:
Hi,

Thanks for this! Just what we needed after VLC plugins no longer work in most browsers!

Paul
Click to expand...

Yup. And luckily this is not a browser plug-in. It's a transcoder that converts RTSP into WebRTC. Is natively supported by most browsers. As with all transcoding it will consume quite a bit of power depending on the source video feed and it will lose some detail.

FYI Here is software that does something similar: mpromonet/webrtc-streamer
 
Last edited:
bp2008 said:
This works by having VLC transcode the video to a format the browser understands. The transcoding has to happen in real time, so a fast computer is needed.
Click to expand...
I'm going to try your recipe on my doorbell project. Thanks.

It should be possible to strip RTSP and substitute WebRTC mp4 envelope when fooling with raw H264 sourced video.

Want to run this reformatting pipeline on something like a pi zero without losing detail to transcoding. Would be eternally grateful if someone finds a working solution.
 
bp2008 said:
This has nothing to do with WebRTC (thank goodness. WebRTC is huge and bloated when all you want to do is stream a video).
Click to expand...
Yup, you are right. My bad. The video tag hides a lot of the underlying infrastructure. Also, seems like a reasonable trade-off should you just want to stream live video and don't care about the limitations that you carefully documented.

AFAIK the key to a low latency and low power proxy seems to hide in the details of controlling the mp4 moov header when re-framing and re-buffering the H264 raw stream.
 
Well you can't live-stream an MP4. The file needs to be finished before it is valid.

The Media Source Extensions API (used by lots of streaming video companies) lets you feed fragmented mp4 files which can be as little as one video frame each. So you can stream in your video in any format you want, then create one "fmp4" out of each video frame and feed it into MSE. This is good for low latency, but only webkit browsers really handle it well. MS Edge (pre-webkit) forces you to buffer several seconds of video, which makes realtime streaming terrible. Firefox is even worse. Every time Firefox finishes decoding the frames you have submitted so far, it can only resume by going back to a keyframe. So if you submit video one frame at a time to achieve the best latency, Firefox will go full-blown idiot mode and continuously decode old video, raising your resource utilization to obscene amounts (along with bad stuttering playback) and it gets worse the longer your keyframe interval is. So most players using Media Source Extensions make sure that every fragmented mp4 they build starts with a keyframe. This imposes a minimum latency equal to your keyframe interval.
 
  • Like
Reactions: newfoundlandplucky
bp2008 said:
Well you can't live-stream an MP4.
Click to expand...

I interpret this to mean you can't livestream non-fragmented MP4. HTML5 video does not handle fragmented-MP4. Instead many solutions use Media Source Extensions API. Using the HTML5 video tag is a lot simpler. Makes sense.

In addition to trying out your VLC recipe I intend to look at this: Streaming via RTSP or RTP in HTML5. I don't know yet if it uses Media Source Extensions or WebRTC. The extra complexity may help the system run with fewer resources.

bp2008 said:
So most players using Media Source Extensions make sure that every fragmented mp4 they build starts with a keyframe. This imposes a minimum latency equal to your keyframe interval.
Click to expand...

I'm looking into this issue right now. Thanks for the anecdotes. Sorry for polluting your thread with this level of detail but glad you're willing share. Thanks.
 
newfoundlandplucky said:
I interpret this to mean you can't livestream non-fragmented MP4. HTML5 video does not handle fragmented-MP4. Instead many solutions use Media Source Extensions API. Using the HTML5 video tag is a lot simpler. Makes sense.
Click to expand...

Correct. A way to live stream with just the HTML5 video tag would be great though :)

newfoundlandplucky said:
In addition to trying out your VLC recipe I intend to look at this: Streaming via RTSP or RTP in HTML5. I don't know yet if it uses Media Source Extensions or WebRTC. The extra complexity may help the system run with fewer resources.
Click to expand...

HLS is pretty widely supported thanks to javascript players which can convert short .ts clips to fragmented mp4 and feed them to Media Source Extensions (HLS sends a series of short .ts videos). I prefer the Clappr player for doing this.
 
  • Like
Reactions: newfoundlandplucky
bp2008 said:
HLS is pretty widely supported thanks to javascript players which can convert short .ts clips to fragmented mp4 and feed them to Media Source Extensions (HLS sends a series of short .ts videos). I prefer the Clappr player for doing this.
Click to expand...
An ultra low power low latency solution seems to require something like Clappr plus an RTSP proxy. Weird because I don't quite understand why an efficient embedded linux proxy isn't already available as an open source project.

Your earlier post helped me understand how to modify webrtc streamer to identify key frames (for my camera they are approximately 2s apart) and attempt to re-frame the raw H264 at each IDR. Need to understand how to box these frames into an fMP4 stream. Not certain if the WebRTC player API will accept fMP4 or if I have to switch to Media Source Extensions API.

For my prototype I will likely stick to the simple HTML5 video tag and VLC. I can make things complicated later.
 
bp2008 said:
Correct. A way to live stream with just the HTML5 video tag would be great though :)

HLS is pretty widely supported thanks to javascript players which can convert short .ts clips to fragmented mp4 and feed them to Media Source Extensions (HLS sends a series of short .ts videos). I prefer the Clappr player for doing this.
Click to expand...
Well. I figured out how to box the H264 stream using MP4 but the results are a mixed bag; results are consistent with your description.

MP4 movie fragments are well formed when the sample table starts with a key frame followed by the non-IDR samples. Using this method it takes very few CPU cycles to box the H264 stream into an MP4 header + subsequent MP4 movie fragments. Development is still in progress but my assumption is that the live stream requires a delay of slightly less than two complete H264 key-frame sequences for reliable playback.

It'll be interesting to see if I can use the high power transcoder when low latency is important and then switch to low power re-framing for background monitoring.
 
For reliable, efficient playback in all browsers, then yes, some delay is necessary. In Firefox I believe you could get away with just one keyframe interval of delay. Technically you can get away with less but it will waste resources due to the issue I mentioned before. Edge is different and I don't fully understand how. I can feed it a series of fMP4 each containing a single H.264 frame, where the keyframe interval is 15+ seconds and it will only force a delay in the area of 2-3 seconds. Other browsers tend to handle this method of feeding in frames just fine and have negligible delay.

Still, the very lowest delay comes from skipping MSE entirely and using a different H.264 decoder. bp2008/ui3 has three different ways to play H.264 video, each with their own pros and cons.
 
  • Like
Reactions: hasen
You must log in or register to reply here.
Top Bottom