Search the Asterisk Blog

Asterisk 15: Multi-stream Media and SFU

By Kevin Harwell

In a previous post some of the upcoming changes made for Asterisk 15 have been discussed. Specifically, one of the items mentioned is the beginnings of a multi-stream media framework. Starting in 15, groundwork has been laid that greatly enhances media flow in Asterisk. Taking advantage of this new architecture, changes have also been made that now allow Asterisk to act as a selective forwarding unit (SFU) with regards to video.

Stream the Streams

In Asterisk, streams are simply logical flows of media. In previous versions, prior to 15, only a single pipe is used to exchange media between endpoints for a session. And while different media types are handled (audio/video/text), only a single instance of each type is supported. For instance, only one flow of video is allowed, thus limiting its usefulness in conferencing applications.

However, in Asterisk 15 the single pipe has been swapped for multiple ones. For supported channel drivers (currently only PJSIP) it’s now possible to convey multiple streams of media for multiple media types. For example with regards to video, a signaling single session is now capable of negotiating, and then sending and receiving both VP8 and H.264.

To enable multi-stream support in the PSJIP channel driver you’ll need to set max_audio_streams and max_video_streams options for a given endpoint to something greater than “1”, which is the default. Note, there is hard limit, set within pjproject, to the total number of streams allowed (for all combinations). If using “pjproject bundled” for Asterisk this number is currently set to 16.

Selective Forwarding Unit

So that multi-streaming stuff is all good, but what’s a practical use case? I’m glad you asked. As mentioned, with respect to video, Asterisk can now act as a selective forwarding unit (SFU). Meaning it is capable of processing multiple video streams, and then selecting which video streams are forwarded to which endpoint. This of course is directly applicable to video conferencing.

By configuring app_confbridge correctly, and setting the video_mode appropriately (to “sfu”) you can enable multi-user video conferencing within Asterisk that allows each participant to see all other participants:

Stop, Demo Time

We created a demo/example WebRTC application called:

Cyber Mega Phone 2000Or CMP2K for short. And while you can’t touch the Hammer I encourage you to download and interact with the demo. To do so, start by configuring your Asterisk 15+ server for WebRTC and set up one or more PJSIP endpoints. If you are unsure how to do that then this guide will show you how. However, instead of using SIPML5 we’ll be using CMP2K as the client instead. To begin, here is the http configuration settings I used (http.conf):

Here is the pjsip configuration I used – don’t forget to set the max_audio_streams and max_video_streams options as mentioned above (pjsip.conf):

Next configure a confbridge and set the video_mode to “sfu” (confbridge.conf):

Lastly add some dialplan to call into the StreamEcho application and/or the conference (extensions.conf):

Now that Asterisk has been setup, start it, and then open the “index.html” file (in either Firefox or a Chrome/Chromium browser) that’s in the top level directory of CMP2K. You should see something like the following:

cyber mega phone 2000

It’s quite a fancy application I know! Press the “account” button and enter your PJSIP endpoint information as well as the extension to dial. Note, the grey text in the account settings are the defaults used. If you specified different values for your endpoint configuration you’ll need enter a value overriding the default. “ID” is your endpoint name. I didn’t attach any authorization settings to my endpoint so the name and password are not applicable.

cyber mega phone 2000 account settings
Close the window and press the “connect” button. You should now be connected to Asterisk via a websocket, and registered if your endpoint settings required it.

NOTE: Having trouble connecting? You may need to import your self-signed certificate into your browser’s keychain. However, for now it’s probably easier to just open a separate browser tab and point it to Asterisk’s HTTP server’s TLS port and WS path, e.g. http://[ ip of asterisk server ]:8089/ws, and you can manually confirm the security exception from there.

Press the “call” button. If you dialed the echo application you should see “n” number of video streams being echoed back to you:

multi-media echo

If you called the video-conference extension (If you didn’t try now. Hangup, open the account settings again, and enter the new extension) you should see yourself only. At least until others show up. I found a couple of friends willing to join my conference:

multi-media conference

In the above you’ll note that there is an “Audio Only” section with no video. This is the audio stream of the conference bridge. Muting it mutes the audio on the bridge itself. Meaning you’ll stop hearing audio from Asterisk.

Where to From Here

As indicated earlier, the new multi-stream media work in Asterisk 15 is a great start. A solid foundation has been established, and we’ve just seen that Asterisk can now act as an SFU giving users a nice video conferencing experience. More work is surely to come in this area and others as the 15 branch continues to be developed, so be sure to keep a close eye on things.

There Are 10 Comments

  • Yasin says:

    it is great job!

  • marek cervenka says:

    looks great!

    can be used with JsSIP? or are there some requirements?

    • Kevin Harwell says:

      Yes it can be used with JsSIP. As a matter of fact that’s the library we used when building CMP2K. If you take a look in the “lib” directory in CMP2K’s source you’ll see it as a dependency.

  • Ray Jender says:

    So I have a WebRTC program that I am working on, but am stuck on the bandwidth issue. The program can show from 1 to 5 streams at a time. Picture a news broadcast or talk show with the host broadcaster and up to 4 guests. Broadcaster and guests can see and talk to each other.

    The possibility of hundreds or even thousands of viewers. Viewers would be one-way only for audio and video.

    Would Asterisk 15 be able to handle this? And how would I interface my existing code to Asterisk?

    Thanks,

    Ray

    • Kevin Harwell says:

      In theory, Asterisk 15 should be able to handle that scenario. The functionality is in place, however for the specified use case it has not been tested. That being said you are still going to run up against bandwidth issues. You’d probably be better off using something like a CDN (Content Delivery Network) instead to deliver the video.

  • sles says:

    I think it’s better to add MCU function…

  • Ankit says:

    Hi,
    I followed your steps for video conferencing and reached up to connect pjsip endpoint with Asterisk in UI but not able to call.(call button not working for me).

    in cyber_mega_phone.js on the line number 78 it stuck:
    console.log(‘new session – ‘ + rtc.direction + ‘ – ‘ + rtc);

    please guide

    • Kevin Harwell says:

      It’s hard to tell what your problem could be. If you can’t click the call button at all (disabled) then either you are not connected (websocket failed) or you are not registered (if setup to register). If you can click the call button, but it fails then you need to look at the SDPs going to/from browser and Asterisk.

      This comment section is not conducive to these types of discussions. If you’d like to follow up please post your question on https://community.asterisk.org/

  • Hangs up after a few seconds with: Reason: SIP ;cause=408; text=”RTP Timeout”
    I use the same webrtc enabled endpoint for other tests and I know it works. I can connect and see my video for a few seconds before getting disconnected. I cannot hear any audio at all while the connection is active. Any pointers on where to check for a solution?

Add to the Discussion

Your email address will not be published. Required fields are marked *

About the Author

Kevin Harwell

Kevin is a Software Developer at Digium. He has a diverse background in the software industry and has worked on an assortment of projects. Since joining the Asterisk team a few years ago he has been a frequent contributor to a variety of areas within the project. He also can usually be seen with a cup of hot tea.

See All of Kevin's Articles

More From
The Digium Blog

  • No items