There is no standardized signaling protocol for WebRTC applications. For this reason, the UV4L Streaming Server attempts to support or leverage a variety of them so that they can be used in different scenarios:
- built-in, “proprietary” message passing over websocket;
- Janus, a general purpose WebRTC Gateway;
- PeerJS (work-in-progress);
- XMPP Jingle (experimental, out of the scope of this doc.)
Built-in signaling
This signaling protocol is available with the UV4L Streaming Server module and allows a peer (e.g. a browser) to initiate a bidirectional, p2p audio-video-data call session with an uv4l instance. This protocol is based on message passing over websocket, which must stay open for the whole duration of the session. Messages are in JSON format and always include a mandatory what field identifying a request (from the initiator to uv4l) or an answer or message (from uv4l to the initiator), a data field specifying the content of the request or the answer, and may include an options field if a request supports it (e.g. a call request).
- call request. This is the very first message in a session that the initiator has to send to uv4l to start a call:
{
what: "call",
options: {
force_hw_vcodec: true,
vformat: 30,
trickle_ice: true
}
}
At the moment of writing, options may only include whether uv4l has to use the hardware encoder and decoder (if supported by the platform on which uv4l is running), what resolution and frame rate shall be used in this case and if trickle ICE optimization has to be used or not (see also the iceCandidate message below). Note that all other options affecting a WebRTC session can be modified, before it begins, through the RESTful API.
- offer answer. After the call request has been sent, uv4l answers with an offer containing its SDP (the string reported as <uv4l-sdp> for simplicity below) :
{ what: "offer", data: "<uv4l-sdp>" } generateIceCandidates request. Deprecated as implicit in the call request, so this is no longer needed and does nothing at the moment. (Old behaviour: after receiving the SDP from uv4l, the initiator had to ask uv4l to start to generate its own list of ICE candidates):{ what: "generateIceCandidates" }- answer request. The initiator can in turn provide its local SDP with the following message:
{ what: "answer", data: "<local-sdp>" } - iceCandidates message. When trickle ICE is not used, after uv4l has automatically gathered all the ICE candidates, they are passed to the initiator all together as an array. As collecting all the candidates beforehand might require several seconds depending on the network environment, it is suggested to turn on and use the trickle_ice optimization (see the call and iceCandidate requests):
{ what: "iceCandidates", data: <array-of-uv4l-ice-candidates> } - iceCandidate message. When trickle ICE is used, whenever a new ICE candidate becomes available, uv4l immediately sends it to the initiator (an empty string in data means that the gathering has completed). Remember that, according to the WebRTC standard, a candidate can be added to the peer connection only after the remote description has been set, otherwise is an error. This is how the message looks like:
{ what: "iceCandidate", data: <single-uv4l-ice-candidate or empty string> } - addIceCandidate request. The initiator tells uv4l to add the one local ICE Candidate specified. This should be typically sent for each local ICE Candidate being generated by the initiator:
{ what: "addIceCandidate", data: "<local-ice-candidate>" } - message message. Messages from uv4l during the session (typically errors), before possibly unilaterally closing the socket, come in the following form:
{ what: "message", data: "<error-message>" } - hangup request. This can be sent by initiator to signal that the call has terminated. Alternatively, to hangup the initiator can just close the websocket:
{
what: "hangup"
}
As you can see the protocol is reasonably simple and can be easily adapted or wrapped into more sophisticated signaling protocols for interoperability with third-party infrastructures.
For a possible implementation of the above protocol in Javascript expand the link below, while an example of use can be found in this ReactJS-based web application or in this simple web application:
RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection;
RTCSessionDescription = window.RTCSessionDescription;
RTCIceCandidate = window.RTCIceCandidate;
function signal(url, onStream, onError, onClose, onMessage) {
if ("WebSocket" in window) {
console.log("opening web socket: " + url);
var ws = new WebSocket(url);
var pc;
var iceCandidates = [];
var hasRemoteDesc = false;
function addIceCandidates() {
if (hasRemoteDesc) {
iceCandidates.forEach(function (candidate) {
pc.addIceCandidate(candidate,
function () {
console.log("IceCandidate added: " + JSON.stringify(candidate));
},
function (error) {
console.error("addIceCandidate error: " + error);
}
);
});
iceCandidates = [];
}
}
ws.onopen = function () {
/* First we create a peer connection */
var config = {"iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]};
var options = {optional: []};
pc = new RTCPeerConnection(config, options);
iceCandidates = [];
hasRemoteDesc = false;
pc.onicecandidate = function (event) {
if (event.candidate && event.candidate.candidate) {
var candidate = {
sdpMLineIndex: event.candidate.sdpMLineIndex,
sdpMid: event.candidate.sdpMid,
candidate: event.candidate.candidate
};
var request = {
what: "addIceCandidate",
data: JSON.stringify(candidate)
};
ws.send(JSON.stringify(request));
} else {
console.log("end of candidates.");
}
};
if ('ontrack' in pc) {
pc.ontrack = function (event) {
onStream(event.streams[0]);
};
} else { // onaddstream() deprecated
pc.onaddstream = function (event) {
onStream(event.stream);
};
}
pc.onremovestream = function (event) {
console.log("the stream has been removed: do your stuff now");
};
pc.ondatachannel = function (event) {
console.log("a data channel is available: do your stuff with it");
// For an example, see https://www.linux-projects.org/uv4l/tutorials/webrtc-data-channels/
};
/* kindly signal the remote peer that we would like to initiate a call */
var request = {
what: "call",
options: {
// If forced, the hardware codec depends on the arch.
// (e.g. it's H264 on the Raspberry Pi)
// Make sure the browser supports the codec too.
force_hw_vcodec: true,
vformat: 30, /* 30=640x480, 30 fps */
trickle_ice: true
}
};
console.log("send message " + JSON.stringify(request));
ws.send(JSON.stringify(request));
};
ws.onmessage = function (evt) {
var msg = JSON.parse(evt.data);
var what = msg.what;
var data = msg.data;
console.log("received message " + JSON.stringify(msg));
switch (what) {
case "offer":
var mediaConstraints = {
optional: [],
mandatory: {
OfferToReceiveAudio: true,
OfferToReceiveVideo: true
}
};
pc.setRemoteDescription(new RTCSessionDescription(JSON.parse(data)),
function onRemoteSdpSuccess() {
hasRemoteDesc = true;
addIceCandidates();
pc.createAnswer(function (sessionDescription) {
pc.setLocalDescription(sessionDescription);
var request = {
what: "answer",
data: JSON.stringify(sessionDescription)
};
ws.send(JSON.stringify(request));
}, function (error) {
onError("failed to create answer: " + error);
}, mediaConstraints);
},
function onRemoteSdpError(event) {
onError('failed to set the remote description: ' + event);
ws.close();
}
);
break;
case "answer":
break;
case "message":
if (onMessage) {
onMessage(msg.data);
}
break;
case "iceCandidate": // received when trickle ice is used (see the "call" request)
if (!msg.data) {
console.log("Ice Gathering Complete");
break;
}
var elt = JSON.parse(msg.data);
let candidate = new RTCIceCandidate({sdpMLineIndex: elt.sdpMLineIndex, candidate: elt.candidate});
iceCandidates.push(candidate);
addIceCandidates(); // it internally checks if the remote description has been set
break;
case "iceCandidates": // received when trickle ice is NOT used (see the "call" request)
var candidates = JSON.parse(msg.data);
for (var i = 0; candidates && i < candidates.length; i++) {
var elt = candidates[i];
let candidate = new RTCIceCandidate({sdpMLineIndex: elt.sdpMLineIndex, candidate: elt.candidate});
iceCandidates.push(candidate);
}
addIceCandidates();
break;
}
};
ws.onclose = function (event) {
console.log('socket closed with code: ' + event.code);
if (pc) {
pc.close();
pc = null;
ws = null;
}
if (onClose) {
onClose();
}
};
ws.onerror = function (event) {
onError("An error has occurred on the websocket (make sure the address is correct)!");
};
this.hangup = function() {
if (ws) {
var request = {
what: "hangup"
};
console.log("send message " + JSON.stringify(request));
ws.send(JSON.stringify(request));
}
};
} else {
onError("Sorry, this browser does not support Web Sockets. Bye.");
}
}
Janus WebRTC Gateway
At the moment of writing, the UV4L Streaming Server supports the videoroom plugin:
This is a plugin implementing a videoconferencing SFU (Selective Forwarding Unit) for Janus, that is an audio/video/data router. This means that the plugin implements a virtual conferencing room peers can join and leave at any time. This room is based on a Publish/Subscribe pattern. Each peer can publish his/her own live audio/video/data feeds: this feed becomes an available stream in the room the other participants can attach to. This means that this plugin allows the realization of several different scenarios, ranging from a simple webinar (one speaker, several listeners) to a fully meshed video conference (each peer sending and receiving to and from all the others).
UV4L natively implements the underlying signaling protocol required to interact with a given videoroom hosted by a Janus Gateway and provides two alternative interfaces to do this via HTTP(S).
One is “action-based”, meaning that you essentially invoke a GET request towards the UV4L Streaming Server specifying, in the parameters of the request, whether you want to start a new session or stop the one in progress. Other parameters indicate the gateway URL, the room number, the username to be shown to others in the room, and some other optional things such as whether you want to be a publisher only, a subscriber or both. For more detailed informations with an example, please read this.
The second interface is based on ad-hoc RESTful API. It gives a more fine-grained control over the whole session and can be used to develop more sophisticated and interactive web applications. The documentation can be found here.
Note that the two interfaces described above are mutually exclusive, thus they cannot be used at the same time to control the same session.