# Text cues

Cues and active cues from metadata, chapters, and app text tracks

Reads the cues of text tracks, from the media and from the app, and tracks which cues are active at the current time. The [timed metadata guide](https://videojs.org/docs/framework/html/guides/timed-metadata) shows how to use it.

`textCues` has one entry per track being read, in `textTrackList` order. Each entry holds the `track` as `textTrackList` lists it, every one of its `cues` ordered by start time, and its `activeCues`. Cues are plain data with an `id`, `startTime`, `endTime`, `text`, and, when the cue carries data, a `value` and a `type`.

By default the feature reads every `metadata` and `chapters` track that isn’t `disabled`, except the `thumbnails` track. `watchTextCues(filter)` adds the tracks a filter matches until the function it returns is called.

Tracks from the media and tracks from the app are read the same way. The [text track feature](https://videojs.org/docs/framework/html/reference/api/feature-text-tracks) lists both in `textTrackList`, and each track’s `source` is `'media'` or `'app'`. Cue changes are kept out of the text track feature, so UI that only lists tracks doesn’t update as cues come and go.

## Import

```ts
import { textCuesFeature } from '@videojs/html';
```

The `audioFeatures`, `liveAudioFeatures`, `liveVideoFeatures`, and `videoFeatures` [feature bundles](https://videojs.org/docs/framework/html/guides/presets) include this feature.

## API Reference

### State

| Property | Type | Description |
| --- | --- | --- |
| `textCues` | `MediaTrackCues[]` | Cues for the tracks being read: `metadata` and `chapters` tracks that aren't disabled, except the thumbnails track, plus any track a `watchTextCues()` filter matches. In `textTrackList` order. |

### Actions

| Action | Type | Description |
| --- | --- | --- |
| `watchTextCues` | `(filter: { kind?: TextTrackKind \| readonly TextTrackKind[]; label?: string; language?: string } \| ((track: MediaTextTrack) => boolean)) => (() => void)` | Read cues for more tracks, such as captions for an interactive transcript. Cues are tracked until the returned function is called. |

### Selector

Pass `selectTextCues` to [`PlayerController`](https://videojs.org/docs/framework/html/reference/api/player-controller) to subscribe to text cue state. Returns `undefined` if the text cues feature is not configured. Outside a custom element, read `player.store.textCues` in a `player.store.subscribe()` callback, or pick cues with [`getActiveTextCues`](https://videojs.org/docs/framework/html/reference/api/get-active-text-cues).

**cue-count.ts**

```ts
import { createPlayer, UIElement, selectTextCues } from '@videojs/html';
import { videoFeatures } from '@videojs/html/video';

const { PlayerController } = createPlayer({ features: videoFeatures });

class CueCount extends UIElement {
  readonly #textCues = new PlayerController(this, selectTextCues);
}
```

## Active cues

A cue is active from its `startTime` up to its `endTime`. A cue whose `endTime` equals its `startTime` is a point, and stays active for one second from where it sits, so a seek that lands far past a point never activates it. The rule is the same for tracks from the media and from the app. The one exception is a chapters track the app added: a chapter without an `endTime` runs to the next chapter, and the last one to the end of the media.

Active cues follow the media’s `currentTime` through `timeupdate` and `seeked` events, and a media track’s `cuechange` events. They aren’t frame-accurate. Updates come a few times a second, so fast playback can move past a point, or a short cue, between two of them. A cue that started since the last update counts as active for that update even when it’s already over, so playback never skips one. A seek doesn’t: the cues it jumps over stay inactive.

Cues delivered with a stream can be open-ended. Cue ends are clamped to the media duration once it’s finite.

## Cue values

A cue’s `value` comes from the first of these that applies:

1. The cue’s own `value`. A `DataCue`, such as the ones Safari’s native HLS creates for in-band metadata, carries one, and so do the cues hls.js creates for ID3 and `EXT-X-DATERANGE` metadata. Its `type` says what `value` holds, such as `org.id3`. A type names the shape of `value`, not where it came from, so ID3 carried in an `emsg` box is `org.id3` too.
2. On a `metadata` track, cue text that parses as a JSON object or array.
3. Otherwise, the cue has no `value`. Its text is always in `text`.

App cues keep the `value` and `type` they were added with.

A media publishes cues on its text tracks in the platform’s shapes. A `VTTCueLike` carries `text`, like the platform’s `VTTCue`. A `DataCueLike` carries a `value` and, when known, its `type`, like the platform’s `DataCue`. A `TextCueLike` is either one, and a single cue can be both, as the cues hls.js creates are. A custom media writes its cues in these shapes.

## App tracks

`addTextTrack()` and `removeTextTrack()` come from the [text track feature](https://videojs.org/docs/framework/html/reference/api/feature-text-tracks), and are described here. `addTextTrack()` adds a track to the player rather than to the media, so it works with any media, including embeds without text tracks of their own. It takes a [track definition](https://videojs.org/docs/framework/html/reference/api/define-text-track) and its content, or one object describing both, and returns a handle to the track.

```ts
const player = document.querySelector('video-player')!;
const quiz = player.store.addTextTrack({ kind: 'metadata', label: 'quiz' });

quiz.addCue({ startTime: 42, value: { question: 'Ready?' } });
quiz.remove();
```

The track description takes these fields:

| Field | Description |
| --- | --- |
| `kind` | The track kind: `metadata`, `chapters`, `captions`, `subtitles`, or `descriptions`. |
| `label` | The track label. Defaults to `''`. |
| `language` | The track language, as a BCP 47 tag. Defaults to `''`. |
| `cues` | The track’s cues. |
| `src` | A file to load cues from. The file is fetched once the player attaches to media, so only on the client. Without a `format`, it’s WebVTT, or JSON that is an array of cues or an object with a `cues` array. |
| `format` | A function that reads the `src` file’s text into cues, such as [`podcastChapters`](https://videojs.org/docs/framework/html/reference/api/chapters-track). It returns an array of cues or an object with a `cues` array, and can return a promise, so a parser for another format, such as SRT, loads only when a file needs it. |

Each cue takes these fields:

| Field | Description |
| --- | --- |
| `id` | The cue’s id. When it’s missing or empty, it defaults to an id unique within the track. |
| `startTime` | Where the cue starts, in seconds. |
| `endTime` | Where the cue ends, in seconds. Omit it for a point, or, in a chapters track, for a chapter that runs to the next one. |
| `text` | The cue’s text. Defaults to `''`. |
| `value` | Structured, JSON-serializable data. |
| `type` | What `value` holds. |

The handle `addTextTrack()` returns has the track’s `id` in `textTrackList`, and `removed`, which turns `true` once the track is gone. `addCue(cue)` and `removeCue(id)` change its cues, and `remove()` removes it. `removeTextTrack(id)` removes a track the app added; tracks from the media stay.

Like a `<track>` element, an app track stays until it’s removed, through source changes and media swaps. Removing a track stops loading its `src`. A file given as `src` that fails to load leaves the track with the cues it already has.

App tracks start `hidden`. Browsers never draw `metadata` or `chapters` cues. An app `captions` or `subtitles` track is copied into the media, so the browser draws its cues when the track is selected, including in iOS fullscreen where the player’s UI isn’t shown, and the captions menu lists it. The track takes its copy’s mode, so it’s `disabled` until it’s selected and `showing` while it is.

When the media has no chapters track, the text track feature’s `chaptersCues` reads the first chapters track the app added, so the [time slider](https://videojs.org/docs/framework/html/reference/components/time-slider) can show its chapters. In the same way, when the media has no thumbnails track, `thumbnailsTrack` reads the first `metadata` track the app added with the label `thumbnails`, such as a storyboard file. Image URLs in its cues resolve against its `src`, as they do for a `<track>` element.

## Exports

| Export | Description |
| --- | --- |
| `MediaTrackCues` | One `textCues` entry: a `track`, its `cues`, and its `activeCues`. |
| `MediaTextCue` | A cue as plain data. Its type parameter types `value`. |
| `TextCueLike` | A cue on a media’s text track: a `VTTCueLike`, a `DataCueLike`, or both. |
| `VTTCueLike` | A media cue that carries `text`, like the platform’s `VTTCue`. |
| `DataCueLike` | A media cue that carries a `value` and its `type`, like the platform’s `DataCue`. |
| `TextTrackFilter` | Picks tracks by `kind`, `label`, or `language`, or with a function. Each given field must match, and a list of kinds matches any kind in it. |
| `TextTrackSource` | Where a track came from: `'media'` or `'app'`. |
| `AddTextTrack` | The type of `addTextTrack()`. |
| `TextTrackInit` | A track description with its content. |
| `TextTrackContent` | What a track carries: `cues`, `src`, and `format`. |
| `TextCueInit` | A cue an app adds. |
| `TextCueFormat` | A function that reads a file’s text into cues. |
| `TextCueFormatResult` | What a `TextCueFormat` returns: an array of cues, or an object with a `cues` array. |
| `TextTrackHandle` | The handle `addTextTrack()` returns. |
| `InferCueValue` | The `value` type of a cue type. |
| `MediaTextCuesState` | The state this feature adds to the store. |

---

HTML documentation: https://videojs.org/docs/framework/html/llms.txt
All documentation: https://videojs.org/llms.txt