livesim2 - the second generation DASH Live Source Simulator

livesim2 is a new and improved version of the [DASH-IF live source simulator][1].

Test it at https://livesim2.dashif.org or set up your own server using your own DASH content or test content available at livesim-content. See the wiki for more info.

As the original simulator ([livesim1][1]), the output is a wall-clock (UTC) synchronized infinite linear stream of segments. This achieved by looping input VoD DASH assets, and changing time stamps so that an infinite "live" stream is available. The synchronization is done modulo asset duration, for example: a 1-hour asset restarts every hour on full hours, and a 30s asset restarts every 30s on full and half minutes. If there is a clock display in the video, and the length is full minutes or similar, it is therefore easy to directly see how long the system delay is from publishing to screen presentation. The very short example assets bundled with the code are only 8s long, which means that they restart every time the UTC time is a multiple of 8s, relative to the Epoch start 1970-01-01:00:00:00Z.

To provide full UTC time stamps on-screen and the possibility to test subtitles, livesim2 has a new feature for generating subtitles for any number of languages. This is done by a URL parameter like /timesubsstpp_en,sv which will result in two stpp (segmented TTML) subtitle tracks with with language codes "en" and "sv", respectively. There is a corresponding setting for wvtt (segmented WebVTT) subtitles using /timesubswvtt_en,sv.

The new livesim2 software is written in Go instead of Python and designed to handle content in a more flexible and versatile way. It is intended to be very easy to install and deploy locally since it is compiled into a single binary that serves the content via a built-in performant HTTP/2 server. There is also a very simple way of setting up HTTPS using Let´s Encrypt.

Similarly to [livesim1][1], the output is highly configurable by adding parameters inside the URLs. These parameters are included not only in the MPD requests, but in all segment requests allowing the server to be stateless, and be able to generate streams with a huge number of parameter variations. Currently, not all parameters of [livesim1][1] are implemented, but there are also new parameters like the generated subtitles mentioned above.

The [URL wiki page][urlparams] lists what is available and the served page /urlgen makes it easy to construct URLs to play the content with specific parameters set.

Beside livesim2 there is a tool called dashfetcher in this repo. That tool can be used to download the MPD and all segments of a DASH VoD asset.

livesim2 server

The server is configured in one or more ways in increasing priority:

1. Default values
2. In a config file
3. Via command-line parameters
4. With environment variables

Major values to configure are:

• the top directory vodroot for searching for VoD assets to be used
• the HTTPS domains if Let's Encrypt automatic certificates are used
• certpath and keypath if HTTPS is used with manually downloaded certificates
• the HTTP/HTTPS port if domains is not being used (default: 8888)

Once the server is started, it will scan the file tree starting from vodroot and gather metadata about all DASH VoD assets it finds. Currently, only source VoD assets using SegmentTimeline with $Time$ and SegmentTemplate with $Number$ are supported.

Command-line parameters

A complete list of parameters, and their access via the command line looks like:

  --certpath string      path to TLS certificate file (for HTTPS). Use domains instead if possible
  --cfg string           path to a JSON config file
  --domains string       One or more DNS domains (comma-separated) for auto certificate from Lets Encrypt
  --host string          full scheme://host used in MPD Location/BaseURL. If empty, auto-detected from the request (honors X-Forwarded-Proto)
  --keypath string       path to TLS private key file (for HTTPS). Use domains instead if possible.
  --livewindow int       default live window (seconds) (default 300)
  --logformat string     log format [text, json, pretty, discard] (default "text")
  --loglevel string      log level [DEBUG, INFO, WARN, ERROR] (default "INFO")
  --maxrequests int      max nr of request per IP address per 24 hours
  --playurl string       URL template to play mpd. %s will be replaced by MPD URL (default "https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html?mpd=%s&autoLoad=true&muted=true")
  --port int             HTTP port (default 8888)
  --repdataroot string   Representation metadata root directory. "+" copies vodroot value. "-" disables usage. (default "+")
  --reqlimitint int      interval for request limit i seconds (only used if maxrequests > 0) (default 86400)
  --reqlimitlog string   path to request limit log file (only written if maxrequests > 0)
  --timeout int          timeout for all requests (seconds) (default 60)
  --vodroot string       VoD root directory (default "./vod")
  --writemissingrepdata  Write representation metadata only if missing (does not override existing)
  --writerepdata         (Re)generate and write representation metadata, overwriting any existing files

Quicker load by using metadata files

For assets with many segments, the scanning process can take a considerable time. The possibility to generate and read extra representation metadata files has therefore been added. For representation repX, the corresponding metadata file is repX_data.json.gz. As the file extensions indicates, these files are gzipped JSON files. To generate such files, turn on writerepdata (always (re)generates and overwrites the files) or writemissingrepdata (writes only the files that are missing, leaving existing ones untouched). The root directory for such files is by default the same as the VoD root directory, meaning that the metadata files will be in the same directories as the corresponding MPDs. However, it is possible to use another path, by specifying repdataroot.

Once the server has started, it is possible to find out information about the server and the assets using the root HTTP endpoint

• /

that in turn points to:

• /urlgen
• /assets
• /config
• /healthz
• /metrics

and links to the Wiki page for more information.

It is also possible to explore the file tree and play Vod assets by starting at

• /vod/...

Finally, any VoD MPD like /vod/cfhd/stream.mpd is available as a live stream by replacing /vod/ with livesim2 e.g. /livesim2/cfhd/stream.mpd.

Backwards compatibility with livesim

For backwards compatibility with the first version of livesim where /livesim was used as a prefix for simulated live output, and /dash/vod was the path to the VoD assets, these two paths are redirected by the server with an HTTP 302 response as:

/livesim/* -> /livesim2/*
/dash/vod/* -> /vod/*

MPD Restrictions

The following restrictions apply to the VoD manifest to be used with livesim2

• live-profile (separate segments)
• one Period with all representations of "same" duration
• no BaseURL elements
• no Location elements
• initialization and media attributes in SegmentTemplate on AdaptationSet level

Special Time Test Parameter nowMS

The query string parameter ?nowMS=... can be used in any request to set the wall-clock time that livesim2 uses as reference time. The time is measured with respect to the 1970 Epoch start, and makes it possible to test time-dependent requests in a deterministic way.

Get Started

Install Go 1.25 or later.

Then run

> go mod tidy

to fetch and install all dependencies.

To build dashfetcher and livesim2 you can use the Makefile like

> make build

to create binaries in the /out directory with embedded version numbers.

During development it may be easier to use the usual go commands:

> cd cmd/dashfetcher
> go build .
> cd ../../cmd/livesim2
> go build .

or compile and run directly with go run ..

HTTPS with automatic certificates

To enable HTTPS in an easy manner, make sure that you have DNS pointing to your machine, and that ports 80 and 443 are forwarded. Then use the parameter --domains=your.domain.com,second.domain.com to automatically fetch TLS certificates from Let's Encrypt for your domains to this machine. The certificates are automatically renewed before they expire.

HTTPS with manual certificates

The old-fashioned way of using manually acquired TLS certificates is also supported. Use the two parameters certpath and keypath to point to the respective files, and set the port to 443.`

Content

The content must be a DASH VoD asset in isoff-live format (individual segment files) with either SegmentTimeline with $Time$ or SegmentTemplate with $Number$. The video segments duration must have an exact average duration so that the total duration is the number of segments time that average. The total duration must be an integral number of milliseconds. For example, an asset with alternating segment duration sof 8s and 4s, is fine as long as there are an equal number of each leading so that the avererage duration is 6s. The segment duration is not allowed to vary more than 50% compared to that average duration in order for livesim2 to be able to generate MPDs with SegmentTemplate with \$Number\$.

The input audio segment do not need to follow the duration of the video segments, although it is beneficial if they are the same, as for example 1.92s segments for 50fps video and 48kHz AAC audio. If the audio segment duration is not identical to the corresponding video segment, the audio will be resegmented as to follow the video segments as far as possible. Every audio segment will start less than one audio frame after the video segment starts.

There are multiple ways to get content to the livesim2 server.

1. Use the bundled test content (only 8s or 12s long)
2. Fetch content that was used with [livesim1][1] from github at [livesim-content][livesim-content]
3. Use the dashfetcher tool to download a DASH asset
4. Use an existing VoD asset in isoff-live profile

livesim2 will scan all the segments of all representations and store metadata about each segments timing in memory.

To avoid doing this at every startup, livesim2 supports storing and reading such representation data from a file. The generation of such data is controlled via the writerepdata, writemissingrepdata and repdataroot configuration parameters.

If such files are detected at startup, they will be used instead of scanning the files, unless the writerepdata parameter is set, which forces a re-scan and overwrites the files. Use writemissingrepdata instead to generate only the files that are missing without overwriting (and still use any existing files).

Bundled test streams with the livesim2 server

A few very short (8s) test assets are bundled with the code. These makes it possible to start the server and get live output by running

> cd cmd/livesim2
> ./livesim2 --vodroot=app/testdata/assets

The log will list the available assets and the port where the server runs.

They can then be streamed via URLs like:

http://localhost:8888/livesim2/WAVE/vectors/cfhd_sets/12.5_25_50/t3/2022-10-17/stream.mpd
http://localhost:8888/livesim2/testpic_2s/Manifest_thumbs.mpd
http://localhost:8888/livesim2/testpic_8s/Manifest.mpd

The default pattern provides MPDs with SegmentTemplate using $Number$. To stream with SegmentTimeline with $Time$, one should add the parameter /segtimeline_1 between livesim2 and the start of the asset path. For SegmentTimeline with $Number$, use /segtimelinenr_1 instead. Other parameters are added in a similar way.

Adding longer assets somewhere under the vodroot results in longer loops. All sources are NTP synchronized (using the host machine clock) with a initial start time given by availabilityStartTime and wrap every sequence duration after that.

Edit lists

CMAF allows two types of edit lists.

1. Time-shift for audio priming in audio streams
2. Time-shift for video to initial zero-valued presentation time

There are two test assets included for these cases

1. WAVE/av/combined.mpod has shifted audio where the first two frames are used for priming. This is seen in the MPD which has a shorter initial segment. This is reflected in the livesim2 MPD where all segments are shifted by this amount
2. bbb_hevc_ac3_8s/manifest.mpd has an edit list for video which shifts all composition time offsets so that the first presentation time is zero. This shift is kept in the MPD and in the sidx boxes of the output from livesim2

livesim-content at Github

In the repo [livesim-content][livesim-content], the content that was used for the [livesim1 online][1-online] service is being gathered to make it easy to reproduce the same use cases.

All content and features are not yet (2023-08-08) in place, but should be so before end of October 2023.

To download and use that content, run

git clone https://github.com/Dash-Industry-Forum/livesim-content.git