WebRTC 24.12 ONVIF™ www.onvif.org December 2024 2008-2024 ONVIF™ All rights reserved. Recipients of this document may copy, distribute, publish, or display this document so long as this copyright notice, license and disclaimer are retained with all copies of the document. No license is granted to modify this document. THIS DOCUMENT IS PROVIDED "AS IS," AND THE CORPORATION AND ITS MEMBERS AND THEIR AFFILIATES, MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR TITLE; THAT THE CONTENTS OF THIS DOCUMENT ARE SUITABLE FOR ANY PURPOSE; OR THAT THE IMPLEMENTATION OF SUCH CONTENTS WILL NOT INFRINGE ANY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. IN NO EVENT WILL THE CORPORATION OR ITS MEMBERS OR THEIR AFFILIATES BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, ARISING OUT OF OR RELATING TO ANY USE OR DISTRIBUTION OF THIS DOCUMENT, WHETHER OR NOT (1) THE CORPORATION, MEMBERS OR THEIR AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES, OR (2) SUCH DAMAGES WERE REASONABLY FORESEEABLE, AND ARISING OUT OF OR RELATING TO ANY USE OR DISTRIBUTION OF THIS DOCUMENT. THE FOREGOING DISCLAIMER AND LIMITATION ON LIABILITY DO NOT APPLY TO, INVALIDATE, OR LIMIT REPRESENTATIONS AND WARRANTIES MADE BY THE MEMBERS AND THEIR RESPECTIVE AFFILIATES TO THE CORPORATION AND OTHER MEMBERS IN CERTAIN WRITTEN POLICIES OF THE CORPORATION. 24.06 Jun 2024 Fredrik Svensson, Jonas Cremon, Karin Hedlund, Hans Busch First release. 24.12 Dec 2024 Jean-Francois Levesque, Jose Melancon Add requirements for device supporting ICE candidates. Add RTCP & I-Frame request requirements 25.06 June, 2025 Tomasz Zajac Added data channels. This document defines how WebRTC and related protocols are to be used for ONVIF clients and devices to establish a peer-to-peer connection between a client and a device using a signaling server. Sending PTZ and other commands via WebRTC datachannels is outside of the scope of this version of the specifcation. W3C WebRTC Specification: Real-Time Communication in Browsers <> IETF RFC 6455 - The WebSocket Protocol <> IETF RFC 6544 - TCP Candidates with Interactive Connectivity Establishment (ICE) <> IETF RFC 8829 - JavaScript Session Establishment Protocol (JSEP) <> IETF RFC 5245 - Interactive Connectivity Establishment (ICE) <> IETF RFC 8866 - SDP: Session Description Protocol <> IETF RFC 8834 - Media Transport and Use of RTP in WebRTC <> IETF RFC 8839 - Session Description Protocol (SDP) Offer/Answer Procedures for Interactive Connectivity Establishment (ICE) <> IETF RFC 8840 - A Session Initiation Protocol (SIP) Usage for Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (Trickle ICE) <> IETF RFC 8445 - Interactive Connectivity Establishment (ICE) <> IETF RFC 8489 - Session Traversal Utilities for NAT (STUN) <> IETF RFC 8656 - Traversal Using Relays around NAT (TURN) <> IETF RFC 7519 - JSON Web Token (JWT) <> IETF RFC 6749 - The OAuth 2.0 Authorization Framework <> IETF RFC 6750 - The OAuth 2.0 Authorization Framework: Bearer Token Usage <> IETF RFC 7064 - URI Scheme for the Session Traversal Utilities for NAT (STUN) Protocol <> IETF RFC 7065 - Traversal Using Relays around NAT (TURN) Uniform Resource Identifiers <> IETF RFC 4585 - Extended RTP Profile for Real-Time Transport Control Protocol (RTCP)-Based Feedback <> IETF RFC 5104 - Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF) Profile <> JSON-RPC 2.0 <> OpenID Connect Core <> ONVIF Security Service Specification <>

Signaling Server A server that manages the WebRTC connections between clients and devices. Peer ID A peer's assigned ID provided by the signaling server. Session ID Identifier for a connection between peers.

ICE Interactive Connectivity Establishment NAT Network Address Translation SDP Session Description Protocol STUN Session Traversal Utilities for NAT TURN Traversal Using Relays around NAT WebRTC Web Real-Time Communication

The WebRTC standard includes APIs for communicating with an ICE Agent, but the signaling component is not part of it. Signaling is needed in order for two peers to share how they should connect. Signaling can be implemented in many different ways, and the WebRTC standard doesn't recommend any specific solution. This specification contains documentation and examples of the signaling protocol used in ONVIF to set up a WebRTC peer-to-peer connection. The setup involves three participants: client, device and signaling server.

In this figure, The client represents a user who initiates the WebRTC session. The device is the resource delivering the media, for example a camera. The signaling server is the mediator that both client and device connect to in order to establish a peer-to-peer connection between them. The Signaling Protocol described in this specification details the data exchange between client and device via the signaling server. Once a peer-to-peer connection has been established, how WebRTC is used is described in the WebRTC usage chapter. The Signaling Protocol defines the messages between a client and a device with the intention of establishing a WebRTC peer-to-peer connection between the client and a device. The messages are always sent via the signaling server called server from now on. Once a peer-to-peer connection has been established the connection with the server can be dropped without affecting the peer-to-peer connection.

Both a client and a device need to connect to the server by setting up a WebSocket connection according to [RFC 6455]. The WebSocket sub protocol shall be set to 'webrtc.onvif.org'. A device shall connect to a signaling server as soon as the configuration contains a valid URI. In case a connection is dropped the device shall reconnect automatically. Each client should use an individual ascending interval strategy to avoid that all clients reconnect at the same time. Both a client and a device shall provide their JWT access token using the Authorization header or via query parameter on the WebSocket URI as defined by [RFC6750] Section 2.3. Once the WebSocket is open, both device and client shall immediately send a register command as specified in section when they receive the HTTP 101 switching protocol.

The communication protocols shall use JSON-RPC version 2 over a WebSocket connection . Parameters shall be passed through an Object by-name [JSON-RPC section 4.2]. The table below provides the encodings to be used. Type Encoding JWT Encoded according to RFC 7519 SDP SDP according to RFC 8866 with \r\n escaping IceCandidate JSON encoding according to W3C WebRTC RTCIceCandidateInit RTCIceServer Connection parameters for a STUN or TURN server as defined by W3C WebRTC DataChannelSubprotocol Data channel subprotocol identifier The following sections specify the parameters of the individual commands.

A signaling server shall expect this command to be sent once before any other command is sent over the WebSocket connection. The signaling server shall verify the validity of the access token. Token details are outside of the scope of this specification. request authorization [JWT] Access token that authorizes the device or client. id optional [string] The unique ID of the device or client. name optional [string] The human readable name of the device or client. response id [string] The ID assigned by the signaling server to the endpoint. faults 401 Authorization failed The JWT token cannot be verified, does not include the required claims or has expired.

An ONVIF compliant signaling server and device shall support this command to establish a streaming session between two peers. request peer optional [string] The ID of the peer to connect to. authorization [JWT] Access token that authorizes the device or client. session optional [string] The ID assigned by the signaling server to the session. profile optional [string] Optional token of the media streaming profile to use. iceServers optional unbounded [RTCIceServer] List of STUN and TURN servers to be used. response session optional [string] The ID assigned by the signaling server to the session. iceServers optional unbounded [RTCIceServer] List of STUN and TURN servers to be used. faults 401 Authorization failed The JWT token cannot be verified, does not include the required claims or has expired. 403 Forbidden The client is not authorized to connect to the provided peer. 480 Temporary unavailable. The target device is currently unavailable. An ONVIF compliant signaling server shall provide the session ID and a list of STUN and TURN servers in both request and response such that both device and client can use them. A compliant client and device shall support password based authentication of TURN server. The urls parameter of the iceServers shall always be encoded as array of strings. The optional shortcut defined by W3C WebRTC specification of passing a single string shall not be used. An ONVIF compliant device shall issue an invite method immediately after responding to this method.

An ONVIF compliant signaling server shall support this command to establish a streaming session between two peers. It shall forward this command to the client and correspondingly route the response back to the device. The dynamics and the format of the SDP records are defined in RFC 8829. Usage of WebRTC data channels is specified in section . request session [string] The ID assigned by the signaling server to the session. offer [SDP] The SDP offer provided by the device. subprotocols optional unbounded [DataChannelSubprotocol] List of data channel subprotocols to be allowed response answer [SDP] The SDP answer provided by the client. subprotocols optional unbounded [DataChannelSubprotocol] List of data channel subprotocols to be allowed faults 400 Bad Request Invalid session ID or SDP payload. 408 Request Timeout The peer did not react. 410 Gone The peer is no more available.

An ONVIF compliant signaling server, device and client shall support this command to signal new ICE candidates for the trickleICE procedure as defined in RFC 8840. A signaling server shall relay this command unaltered to the peer. request session [string] The ID assigned by the signaling server to the session. candidate [IceCandidate] The ICE Candidate SDP update for the peer. response <none> faults <none> Both client and device produce ICE candidates and send them to each other, via the server. Once all ICE candidates have been sent by a peer, it shall send a last trickle notification with an empty ICE candidate content to indicate that all candidates have been sent according to RFC 8838. NOTE: Note that ICE candidates can arrive before the SDP offer and the implementing client needs to handle this. If everything works as it should, a peer-to-peer WebRTC session can be set up between client and device. After the session has been established the client can terminate the WebSocket session to the signaling server and the peer-to-peer connection will not be affected.

An ONVIF compliant signaling server, device and client shall support sending or receiving notifications signaling that an error has occurred. This message is a [JSON-RPC] "Request notification" and shall include [JSON-RPC] "Error object" in its params field. request code [int] A number that indicates the error type that occurred. message [string] A string providing a short description of the error. The message should be limited to a concise single sentence. session [string] The ID assigned by the signaling server to the session. response <none> faults <none> The following table defines possible error codes: Signaling Error Error Code Reason Insufficient resources 1001 The peer does not have sufficient resources. Peer disconnected 1002 A peer in the session has closed its websocket connection with the Signaling Server. Malformed candidate 1003 The trickle ICE candidate received is malformed.

The following example shows the flow using JSON-RPC 2.0. Note that for improved readability the mandatory JSON version string is not shown. Server: { "method": "register", "params": {"authorization": "JWT..."}, "id":1} Server -> Client: { "result": { "id": "client-a1" }, "id": 1} Device -> Server: { "method": "register", "params": {"authorization": "JWT..."}, "id":1} Server -> Device: { "result": { "id": "device-b1" }, "id": 1} Client -> Server: { "method": "connect", "params": {"authorization": "JWT...", "peer": "device-b1"}, "id":2} Server -> Device: { "method": "connect", "params": {"session": "s1", "iceServers": [{"urls": ["stun:1.2.3.4"]}], "id":2} Device -> Server: { "result": {}, "id": 2} Server -> Client: { "result": {"session": "s1", "iceServers": [{"urls": ["stun:1.2.3.4"]}]}, "id": 2} Device -> Server: { "method": "invite", "params": {"session": "s1", "offer": "SDP...", "subprotocols": ["proto1"]}, "id":3} Server -> Client: { "method": "invite", "params": {"session": "s1", "offer": "SDP...", "subprotocols": ["proto1"]}, "id":3} Client -> Server: { "result": {"answer": "SDP...", "subprotocols": ["proto1"]}, "id": 3} Server -> Device: { "result": {"answer": "SDP...", "subprotocols": ["proto1"]}, "id": 3} Device -> Server: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} Server -> Client: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} Client -> Server: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} Server -> Device: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} Client -> Server: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} Server -> Device: { "method": "trickle", "params": {"session": "s1", "candidate": {...}}} ... ]]>

This chapter describes ONVIF specific usage of WebRTC regarding how to send data between the client and device on the peer-to-peer connection.

Devices may define profiles for different streaming scenarios that can be chosen by the client based on the available bandwidth or other limiting factors. For example, a specific profile for mobile clients, another one for high quality streaming, etc. Devices should estimate available uplink bandwidth and reduce stream bitrate as needed to avoid packet loss. It may be achieved by switching the currently streamed profile into a profile with a lower bitrate encoding. Devices are allowed to switch only between profiles with the same video source and same video codec. Once there is enough available bandwidth, devices should aim to switch back to the initially selected profile.

Devices shall be able to exchange RTCP feedback with their peer as defined in RFC 8834. In particular, devices shall handle FIR messages, as defined in RFC 5104, and PLI messages, as defined in RFC 4585, and attempt to provide a synchronization point according to the semantics of these requests.

A device shall support generating and receiving host, server reflexive and relay ICE candidates as defined in [RFC 8445]. A device shall handle passive ICE TCP candidates as defined in [RFC 6544]. On reception, it shall generate an active ICE TCP candidate to match and establish a TCP connection to the endpoint announced by the candidate. A device is not required to listen on a TCP endpoint and generate a passive ICE TCP candidate.

When using WebRTC with a web browser only certain codecs will work. Because of this ONVIF recommends to use the h.264 codec. In line with WebRTC standard, ONVIF also recommends to always use congestion control to ensure that WebRTC cannot be used to flood the network.

When using WebRTC with a web browser only certain codecs will work. Because of this ONVIF recommends to use the Opus or PCMU codec. For two-way audio it's recommended to use echo cancellation to avoid feedback loops.

If the device supports WebRTC data channels, it shall signal this in the SDP offer of the invite method. If the client supports data channels and intends to use them during the WebRTC session, it shall signal this in the SDP answer. ONVIF-defined data channel subprotocol names shall have “vnd.onvif” prefix. Usage of this prefix is restricted to ONVIF-defined subprotocols only. Before creating and using any data channel subprotocol, the device and client shall exchange a list of allowed data channel subprotocols in the invite method. Only subprotocols provided by both the device and client during the invite method shall be allowed to be enabled during the WebRTC session. At any time after obtaining a WebRTC connection, a client may request enabling a pre-negotiated data channel subprotocol by creating a new data channel with the specified subprotocol identifier. Device shall enable the requested data channel subprotocol only after ensuring it was negotiated during the SDP exchange; otherwise, it shall close the data channel.

Subprotocol identifier: vnd.onvif.metadata+gzip The Metadata data channel subprotocol enables streaming XML Metadata over WebRTC data channels. An ONVIF compliant device supporting streaming of metadata shall support gzip compressed metadata as signaled via the subprotocol vnd.onvif.metadata+gzip. A device shall use the MetadataConfiguration from the media streaming profile selected for the WebRTC session and ignore the CompressionType setting. The data channel Metadata payload is a binary data channel message starting with a GZIP header according to RFC 1952, followed by the compressed data. Messages may be fragmented as needed, and the client shall determine the beginning of a new GZIP payload based on the existence of the GZIP header.