Text cues

Reads the cues of text tracks, from the media and from the app, and tracks which cues are active at the current time. The timed metadata guide shows how to use it.

textCues has one entry per track being read, in textTrackList order. Each entry holds the track as textTrackList lists it, every one of its cues ordered by start time, and its activeCues. Cues are plain data with an id, startTime, endTime, text, and, when the cue carries data, a value and a type.

By default the feature reads every metadata and chapters track that isn’t disabled, except the thumbnails track. watchTextCues(filter) adds the tracks a filter matches until the function it returns is called.

Tracks from the media and tracks from the app are read the same way. The text track feature lists both in textTrackList, and each track’s source is 'media' or 'app'. Cue changes are kept out of the text track feature, so UI that only lists tracks doesn’t update as cues come and go.

Import

import { textCuesFeature } from '@videojs/html';

The audioFeatures, liveAudioFeatures, liveVideoFeatures, and videoFeatures feature bundles include this feature.

(filter: { kind?: TextTrackKind | readonly TextTrackKind[]; label?: string; language?: string } | ((track: MediaTextTrack) => boolean)) => (() => void)

Selector

import { createPlayer, UIElement, selectTextCues } from '@videojs/html';
import { videoFeatures } from '@videojs/html/video';

const { PlayerController } = createPlayer({ features: videoFeatures });

class CueCount extends UIElement {
  readonly #textCues = new PlayerController(this, selectTextCues);
}

Active cues

A cue is active from its startTime up to its endTime. A cue whose endTime equals its startTime is a point, and stays active for one second from where it sits, so a seek that lands far past a point never activates it. The rule is the same for tracks from the media and from the app. The one exception is a chapters track the app added: a chapter without an endTime runs to the next chapter, and the last one to the end of the media.

Active cues follow the media’s currentTime through timeupdate and seeked events, and a media track’s cuechange events. They aren’t frame-accurate. Updates come a few times a second, so fast playback can move past a point, or a short cue, between two of them. A cue that started since the last update counts as active for that update even when it’s already over, so playback never skips one. A seek doesn’t: the cues it jumps over stay inactive.

Cues delivered with a stream can be open-ended. Cue ends are clamped to the media duration once it’s finite.

Cue values

A cue’s value comes from the first of these that applies:

1. The cue’s own value. A DataCue, such as the ones Safari’s native HLS creates for in-band metadata, carries one, and so do the cues hls.js creates for ID3 and EXT-X-DATERANGE metadata. Its type says what value holds, such as org.id3. A type names the shape of value, not where it came from, so ID3 carried in an emsg box is org.id3 too.
2. On a metadata track, cue text that parses as a JSON object or array.
3. Otherwise, the cue has no value. Its text is always in text.

App cues keep the value and type they were added with.

A media publishes cues on its text tracks in the platform’s shapes. A VTTCueLike carries text, like the platform’s VTTCue. A DataCueLike carries a value and, when known, its type, like the platform’s DataCue. A TextCueLike is either one, and a single cue can be both, as the cues hls.js creates are. A custom media writes its cues in these shapes.

App tracks

addTextTrack() and removeTextTrack() come from the text track feature, and are described here. addTextTrack() adds a track to the player rather than to the media, so it works with any media, including embeds without text tracks of their own. It takes a track definition and its content, or one object describing both, and returns a handle to the track.

const player = document.querySelector('video-player')!;
const quiz = player.store.addTextTrack({ kind: 'metadata', label: 'quiz' });

quiz.addCue({ startTime: 42, value: { question: 'Ready?' } });
quiz.remove();

The track description takes these fields:

Each cue takes these fields:

The handle addTextTrack() returns has the track’s id in textTrackList, and removed, which turns true once the track is gone. addCue(cue) and removeCue(id) change its cues, and remove() removes it. removeTextTrack(id) removes a track the app added; tracks from the media stay.

Like a <track> element, an app track stays until it’s removed, through source changes and media swaps. Removing a track stops loading its src. A file given as src that fails to load leaves the track with the cues it already has.

App tracks start hidden. Browsers never draw metadata or chapters cues. An app captions or subtitles track is copied into the media, so the browser draws its cues when the track is selected, including in iOS fullscreen where the player’s UI isn’t shown, and the captions menu lists it. The track takes its copy’s mode, so it’s disabled until it’s selected and showing while it is.

When the media has no chapters track, the text track feature’s chaptersCues reads the first chapters track the app added, so the time slider can show its chapters. In the same way, when the media has no thumbnails track, thumbnailsTrack reads the first metadata track the app added with the label thumbnails, such as a storyboard file. Image URLs in its cues resolve against its src, as they do for a <track> element.

Exports