WebRTC-Observer

Observer is a server-side service for monitoring WebRTC applications. You can run it in a docker container with a command docker run observertc/observer. The service by default is listening on the port 7080.

To see ObserveRTC integrations and examples, check out the full stack examples repository.

Table of Contents:

• Overview
• Scalability
• Configurations
• Micronaut configs
• Management endpoints configs
• Observer configs
• Hamok configs
• Sinks
• LoggerSink
• Apache Kafka
• Mongo Database
• AWS Firehose
• AWS S3 Bucket
• AWS Credential Settings
• Performance considerations
• Compatibility
• Getting Involved
• Changelog
• License

Overview

Observer is a server-side service for monitoring WebRTC applications. The service receives Samples from Monitors. Based on the received samples, the observer:

• Identify calls
• Create events (call started, call ended, client joined, client left, etc.)
• Match clients' inbound-rtp sessions with remote clients' outbound-rtp sessions
• Match SFUs RTP sessions with browser-side clients' in-, and outbound-rtp sessions.

Receiving Samples

The observer accepts Samples on the following endpoints:

• REST: http(s)://${OBSERVER_ADDRESS}:{PORT}/rest/samples/{serviceId}/{mediaUnitId}
• Websocket: ws(s)://${OBSERVER_ADDRESS}:{PORT}/samples/{serviceId}/{mediaUnitId}

Sending Reports

The observer create Reports based on the obtained events, matched sessions and received samples. The reports are forwarded to Sinks. Currently, the following type of sinks are supported: * KafkaSink: Apache Kafka Integration * MongoSink: Mongo Database integration * FirehoseSink: AWS Firehose integration * AwsS3Sink: AWS S3 integration * WebsocketSink: Websocket Sink

Scalability

The service uses a distributed in-memory object storage hamok as its base to store information about calls while clients and SFUs are sending samples. If the service need to be scaled horizontally you need to setup hamok in your configuration. Additionally, you can take a look at the section
hamok configurations to have more information about the built in discovery and communication endpoint features.

Some useful tips you can find how to design your service in terms of traffic at the performance consideration section.

Configurations

At startup time the application fetches the configuration and sets up the service. The default configuration the observer starts with can be found here.

The configuration file is divided to the following main parts: * micronaut: defines the server base properties (listening port, REST authentication methods, etc...) * endpoints: defines the management endpoints of the service. * observer: defines the observer functionalities

Micronaut configs

The application is written by using micronaut framework. Micronaut defines the base properties of the application including which port it listens on for REST or websocket requests. The complete guide to configure a micronaut application can be found here.

Here is an example for the micronaut configuration part:

micronaut:
  server:
    # defines the port the service is listening on 
    port: 7080
  metrics:
    # enable metric collection of the service (cpu usage, memory usage, etc.)
    enabled: True
    export:
      # Sets an exporting methods of the collected metrics
      prometheus:
        enabled: true
        descriptions: true
        # Sets the frequency to collect the metrics
        step: PT1M
  # Sets authentication methods to REST API endpoints
  # for complete guideline to setup the security for REST API requests check the
  # documentations https://micronaut-projects.github.io/micronaut-security/3.4.1/guide/configurationreference.html
  security:
    enabled: false

Management endpoints configs

The application have management endpoints. Management endpoints can be reached on a different port number than other micronaut endpoints. This allows for example to integrate the service into a container orchestration systems like kubernetes.

Here is an example for the endpoints configuration part:

endpoints:
  all:
    # defines the port number for the management endpoints
    port: 7081
  prometheus:
    # indicate if /metrics endpoint is provided or not
    enabled: true
    # indicate if the provided prometheus endpoint requires authentication or not
    # the authentication method is defined in the micronaut.security part
    sensitive: false
  health:
    # indicate if the service provide /heath endpoint.
    enabled: true

Observer configs

The observer configuration can be divided into the following components: * security: define properties of security related processes (hash algorithm, hash salt, allowed serviceIds, etc.). * sources: define properties of endpoints collecting Samples (for example enable or disable collecting for REST or Websocket) * sinks: define connections observer generated Reports are forwarded to (mongoDB, kafka, etc.). * repository define properties of the repository the observer stores data in (media track expiration time, peer connection expiration time). * buffers defines the properties of the internal buffers (samples buffering time, reports buffering time, etc.) * hamok: defines the settings of the hamok distributed object storage the service uses.

Here is an example for the observer configuration part:

```yaml observer:

# configuration to expose metrics metrics: # the prefix for metrics exposed by the observer processes prefix: "observertc" # the name of the tag for serviceId used where it is exposed serviceIdTagName: "service" # the name of the tag for mediaUnitId used where it is exposed mediaUnitIdTagName: "mediaUnit" # metrics exposed from evaluators evaluatorMetrics: # flag indicate if evaluator metrics are exposed or not enabled: false reportMetrics: # flag indicate if report metrics are exposed or not enabled: false sinkMetrics: # flag indicate if sink metrics are exposed or not enabled: false sourceMetrics: # flag indicate if source metrics are exposed or not enabled: false repositoryMetrics: # flag indicate if repository metrics are exposed or not enabled: false # sets the period for exposing metrics about the number of entries the repository holds exposePeriodInMins: 5

sources: # flag indicate if sfu sampels are accepted or not acceptSfuSamples: true # flag indicate if client samples are accepted or not acceptClientSamples: true # if it sets to true then observer overrides the samples provided timestamp to its own, to simplify the problem # of different clients reporting timestamp in different time zones. useServerTimestamp: false # settings related to REST API accepting Samples rest: # indicate if Samples are accepted through REST or not enabled: true # settings related to Websockets accepting Samples websocket: # indicate if Samples are accepted through websockets or not enabled: true # this defines how many remote observer peers must be connected to the grid before # the websocket starts accepting sessions # this configuration is useful to avoid a huge amount of session directed to the first # available observer instance minRemotePeers: 0

repository: # indicate if the repositories storages in hamok should use distributed backup service or not. # NOTE: Distributed backup service increases traffic inside the cluster and volume of data stored in the memory useBackups: false # the maximum idle time for a call object after which it is removed from the repository callMaxIdleTimeInS: 300 # the maximum idle time for an SFU object after which it is removed from the repository sfuMaxIdleTimeInS: 300 # the maximum idle time for a Sfu Transport object after which it is removed from the repository sfuTransportMaxIdleTimeInS: 600 # the maximum idle time for a client object after which it is removed from the repository clientsMaxIdle: 300 # the maximum idle time for a peer connection object after which it is removed from the repository peerConnectionsMaxIdle: 300 # the maximum idle time for an inbound track object after which it is removed from the repository inboundTracksMaxIdle: 300 # the maximum idle time for an outbound track object after which it is removed from the repository outboundTracksMaxIdle: 300 # the maximum idle time for a sfu inbound rtp pad object after which it is removed from the repository sfuInboundRtpPadMaxIdleTimeInS: 300 # the maximum idle time for a sfu outbound rtp pad object after which it is removed from the repository sfuOutboundRtpPadMaxIdleTimeInS: 300 # the maximum idle time for a sfu sctp channel object after which it is removed from the repository sfuSctpChannelMaxIdleTimeInS: 300

buffers: # settings of the buffer collect samples from the sources samplesCollector: # the maximum number of items of the buffer maxItems: 10000 # the max time the buffer can hold an item in milliseconds maxTimeInMs: 10000 # settings of the buffer holds item between evaluators debouncers: # the maximum number of items of the buffer maxItems: 10000 # the max time the buffer can hold an item in milliseconds maxTimeInMs: 1000 # settings of the buffer collect reports from evaluators reportsCollector: # the maximum number of items of the buffer maxItems: 10000 # the max time the buffer can hold an item in milliseconds maxTimeInMs: 10000

# sets up the security methods and values for the observer analysis security: # flag indicating if fetched configurations should be written to the console or not printConfigs: True # obfuscation methods related settings obfuscations: # the used hash algorithm for anonymization hashAlgorithm: SHA-256 # the added extra salt for the hash in anonymization process hashSalt: "MySalt"

# settings for analysing Samples evaluators: # settings for component responsible for updating call entity objects callUpdater: # defines the strategy to assign callId to clients. # possible values: # - MASTER: the Observer responsible to assign new callId to clients, # and only the observer can end a call # - SLAVE: the client is responsible to provide callId, and whenever a new callId # is provided for a room, the previous call is closed callIdAssignMode: master # settings for component analyse the samples from # client applications clientSamplesAnalyser: # drop reports created from inbound-audio and video if # their outbound pair has not been found
dropUnmatchedReports: false

# settings for component analyse the samples from SFUs
sfuSamplesAnalyser:
   # drop reports created from inboundRtpPad if their sfuStreamId
   # been not registered by a client
   dropUnmatchedInboundReports: false
   # drop reports created from outboundRtpPad if their sfuSInkId
   # been not registered by a client
   dropUnmatchedOutboundReports: false
   # drop reports created from internal inboundRtpPad samples if their counterpart
   # internal outbound rtp pads are not matched
   dropUnmatchedInternalInboundReports: false

# obfuscator component settings
obfuscator:
  # indicate if it is enabled or not
  enabled: false
  # indicate how the ICE addresses has been obfuscated.
  # possible values are: none, anonymization
  iceAddresses: none
  # indicate how the room ids are obfuscated.
  roomId: none
  # indicate how the user ids are obfuscated.
  userId: none

# Setup the sinks to forward Reports. # Detailed description is in the Sinks section sinks: {}

# settings related to hamok distributed object storage # and discovery of other observer instances to create a grid hamok: # settings related to the storage grid the observer shares objects through storageGrid: # the retention time in minutes the Raft keeps logs for # this determines how long a new peer can join to the cluster without # issuing a storage sync. If a peer joins after a retention time, # it issues a storage sync which requires the leader to synchronize all # storage with the n