Iñaki Baz José Luis Millán
@ibc_tw @jomivi
BUILDING MULTI-PARTY VIDEO APPS WITH
MEDIASOUP
BACKGROUND
WHAT WE DO
▸ RFC 7118 “The WebSocket protocol as a Transport for SIP”
▸ JsSIP “The JavaScript SIP library”
▸ OverSIP (first SIP proxy with WebSocket support)
▸ mediasoup “Cutting Edge WebRTC Video Conferencing”
MEDIASOUP…
▸ is:
▸ a client and server side library
▸ a muti-party video component
▸ is not:
▸ an application itself
▸ an end product
WEBRTC SERVER TOPOLOGIES
MULTIPOINT CONTROL UNIT (MCU)
Participants send their media to the server
Participants receive others media in a single
stream, mixed by the server
✓ Clients need to handle a single remote stream
✓ Server performs transcoding
✓ Low download link required
๏ CPU intensive in server side
๏ High latency
๏ Fixed remote participants representation
๏ Low flexibility in client side
WEBRTC SERVER TOPOLOGIES
SELECTIVE FORWARDING UNIT (SFU)
Participants send their media to the server
Participants receive others media in separate
streams, one each
✓ Server simply routes. High throughput, low latency
✓ Low CPU usage in server side
✓ Client can decide what streams to receive
✓ Client/Server can choose quality for each stream
๏ Higher download link required
๏ No transcoding
MEDIASOUP
SERVER
MEDIASOUP SERVER
▸ Programmable WebRTC Selective Forwarding Unit (SFU)
▸ Written in C++ in its core, using libuv for asynchronous IO
▸ Written in JavaScript ES6 in the surface
▸ Offers a ORTC like API (no SDP but RTC Objects)
▸ Presented as a Node.js module
$ npm install mediasoup
MEDIASOUP SERVER
NODE.JS MODULE ARCHITECTURE
▸ Server instance launches the C++ workers
▸ Rooms are created within a server
▸ Peers are created within a room
MEDIASOUP SERVER
MEDIASOUP PEER
▸ WebRTC endpoint in the server side
▸ Interacts with a remote endpoint (browser, native client)
▸ Handles transports, producers and consumers
▸ A Transport represents the channel for ICE, DTLS, SRTP
▸ A Producer represents a media track produced by the remote
peer
▸ A Consumer represents a media track produced by other peer
and consumed by this one
MEDIASOUP
CLIENT
MEDIASOUP CLIENT
▸ client-side javascript SDK
$ npm install mediasoup-client
$ bower install mediasoup-client
▸ Abstracts the app from the underlaying WebRTC device
▸ SDP specifics, WebRTC API, ORTC API
▸ Handles message exchange with mediasoup server
MEDIASOUP CLIENT
MEDIASOUP CLIENT SDK ARCHITECTURE
▸ Room representing the room in mediasoup server
▸ Local peer representing the local WebRTC endpoint
▸ It consists of Transports and Producers
▸ Remote Peers are added to the room as they join
▸ They consist of Consumers
MEDIASOUP
CLIENT AND
SERVER
INTERACTION
MEDIASOUP CLIENT AND SERVER MESSAGE EXCHANGE
MEDIASOUP CLIENT AND SERVER MESSAGE EXCHANGE (SIMPLIFIED)
BUILDING THE
APPLICATION
APPLICATION
APPLICATION EXAMPLE
▸ Campus X has decided to offer online live classes
▸ Teacher talks, students listen and see the teacher’s
webcam
▸ Students can “raise the hand” when they want to talk
▸ If they are granted permission, they talk and are seen
TEACHER LOGS IN THE CAMPUS AND CREATES THE ‘MASTERCLASS’ ROOM
TEACHER JOINS THE ROOM
TEACHER STARTS SENDING MEDIA
STUDENT JOINS THE ROOM AND STARTS RECEIVING TEACHER’S MEDIA
STUDENT REQUESTS PERMISSION FOR TALKING
STUDENT STARTS SENDING MEDIA
TEACHER STARTS RECEIVING STUDENT’S MEDIA
DEMO
DEMO
FIRSTSIGHT
▸ https://firstsight.mediasoup.org
▸ Join using desktop or Android Chrome/Firefox
