How to play multi-track video files

Browsers do not support playback of multi-track mp4 files, they will only play the first video, the first audio track and no subtitle tracks. Some browsers support the audioTracks api, but it is also limited to play one audio track at the time and requires you to enable experimental flags in the browser. So what you normally need to do is to conform your video material into multiple files to be able to play all tracks. This is a process that usually is performed when ingesting the material, but that might not be feasible to do on large already existing archives for example.

info

Read about the known limitations in the reference documentation

Fortunately, Validate offers support for multi-track playback. To activate this feature, set one or both of the following settings:

Enable track extraction from video proxy

{
  "settings": {
    "trackExtraction": {
      "extractAudio": true,
      "extractSubtitles": true
    }
  }
}

Enabling these options enables the playback of multiple muxed audio and subtitle tracks. You also need to load a video file containing multiple muxed audio and/or subtitle tracks.

info

Multi-track playback only functions with the main video proxy; side-loading separate multi-track audio files is not possible. However, single-track audio files can be used with a multi-track video file.

For testing purposes, here is a sample video you can use:

Load a video proxy with muxed audio and/or subtitles

{
  "data": {
    "assets": [
      {
        "files": [
          {
            "type": "VIDEO",
            "url": "https://accurate-player-demo-assets.s3.eu-central-1.amazonaws.com/liden/ToS-multiaudio_fixed_w_subs.mp4"
          }
        ]
      }
    ]
  }
}

Putting this together we get this launch template:

Complete example

{
  "data": {
    "assets": [
      {
        "files": [
          {
            "type": "VIDEO",
            "url": "https://accurate-player-demo-assets.s3.eu-central-1.amazonaws.com/liden/ToS-multiaudio_fixed_w_subs.mp4"
          }
        ]
      }
    ]
  },
  "settings": {
    "trackExtraction": {
      "extractAudio": true,
      "extractSubtitles": true
    }
  }
}

Try it out

Custom track labels

By default, track labels are read from user data set on the tracks in the source file.

You can customize how the track labels are displayed in Validate by setting the audioFileDisplay and/or subtitleFileDisplay options:

{
  "settings": {
    "audioFileDisplay": "%metadata:label (%channels channels)",
    "subtitleFileDisplay": "%metadata:label (%language)"
  }
}

To set metadata on the individual tracks within the video file you can specify them under the container object like so:

{
  "data": {
    "assets": [
      {
        "files": [
          {
            "type": "VIDEO",
            "url": "https://accurate-player-demo-assets.s3.eu-central-1.amazonaws.com/liden/ToS-multiaudio_fixed_w_subs.mp4",
            "container": {
              "audioStreams": [
                {
                  "metadata": [
                    {
                      "key": "label",
                      "value": "Custom audio 1"
                    }
                  ]
                },
                {
                  "metadata": [
                    {
                      "key": "label",
                      "value": "Custom audio 2"
                    }
                  ]
                }
              ],
              "subtitleStreams": [
                {
                  "metadata": [
                    {
                      "key": "label",
                      "value": "Custom subtitle 1"
                    }
                  ]
                }
              ]
            }
          }
        ]
      }
    ]
  }
}

Now you'll see the custom labels in Validate on the tracks extracted from the video file:

Try it out

CORS Considerations

The multi-track functionality needs to be able to read the Content-Range header, when fetching the media, to be able to function properly. This header is not exposed by browsers by default when doing cross-origin requests.

You need to configure your media storage to set the Access-Control-Expose-Headers header to include Content-Range. If you are using AWS S3 you can do so by specifying this in the CORS configuration:

Example S3 CORS configuration

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "HEAD"],
    "AllowedOrigins": ["https://*.accurate.video"],
    "ExposeHeaders": ["Content-Range"],
    "MaxAgeSeconds": 3000
  }
]