Sync content with timed metadata

Timed metadata is data tied to a span of playback: a quiz question, a product card, an ad break, or a chapter. Video.js reads it as text tracks and cues, the platform objects captions use, whether the cues come from the media or from your app.

Recommended approach

import { Container, defineTextTrack, type TextCueInit, useActiveTextCues, useCreateTextTrack } from '@videojs/react';
import { Video, VideoPlayer } from '@videojs/react/video';

interface Note {
  title: string;
  body: string;
}

const notesTrack = defineTextTrack<Note>({ kind: 'metadata', label: 'notes' });

const cues: TextCueInit<Note>[] = [
  { startTime: 1, endTime: 8, value: { title: 'Timed metadata', body: 'This note is a cue the app added.' } },
  { startTime: 12, endTime: 19, value: { title: 'Any data', body: 'Each cue carries a value you render.' } },
  { startTime: 23, endTime: 30, value: { title: 'Any media', body: 'The player holds the track, not the video.' } },
];

function Notes() {
  useCreateTextTrack(notesTrack, { cues });

  const [cue] = useActiveTextCues(notesTrack);
  if (!cue?.value) return null;

  return (
    <p className="react-timed-metadata-basic__note">
      <strong className="react-timed-metadata-basic__title">{cue.value.title}</strong>
      {cue.value.body}
    </p>
  );
}

export default function BasicUsage() {
  return (
    <VideoPlayer>
      <Container className="react-timed-metadata-basic__container">
        <Video src="https://stream.mux.com/BV3YZtogl89mg9VcNBhhnHm02Y34zI1nlMuMQfAbl3dM/highest.mp4" autoPlay muted playsInline loop />
        <Notes />
      </Container>
    </VideoPlayer>
  );
}

.react-timed-metadata-basic__container {
  position: relative;
}

.react-timed-metadata-basic__container video {
  width: 100%;
  aspect-ratio: 16 / 9;
}

.react-timed-metadata-basic__note {
  position: absolute;
  top: 12px;
  left: 12px;
  max-width: calc(100% - 24px);
  padding: 10px 14px;
  margin: 0;
  font-size: 0.875rem;
  line-height: 1.4;
  color: black;
  background: rgba(255, 255, 255, 0.85);
  border-radius: 12px;
  backdrop-filter: blur(10px);
}

.react-timed-metadata-basic__title {
  display: block;
}

In your app, the cues usually come from your API or CMS. Fetch them and pass them as cues, or pass the file’s URL as src and the player loads it.

How it works

A text track has a kind, label, and language, and a list of cues. Each cue spans startTime to endTime, in seconds, and carries text, structured data in value, or both. value and type are named after the platform’s DataCue. A cue without an endTime is a point, active for one second, except in a chapters track your app adds, where it runs to the next chapter.

addTextTrack() adds the track to the player, not the media, so it works with any media, including YouTube and other embeds. The track sits next to the media’s tracks in textTrackList with source: 'app'.

The text cues feature, included in the video, audio, and live presets, reads cues from the media’s metadata and chapters tracks and from the tracks your app adds, and tracks which cues are active at the current time. Tracks from either place are read the same way.

defineTextTrack<Note>() types each cue’s value, and works as a filter: reading through it picks the tracks with the same kind, label, and language. A plain filter, such as { label: 'notes' }, works too, but its cues’ value is unknown.

Browsers never draw metadata or chapters cues, so the overlay, its styles, and its accessibility are up to your UI.

Availability and constraints

• Active cues follow currentTime, which updates up to about four times a second. Don’t rely on them for frame-accurate timing. A cue that playback passes between two updates still shows up as active once; a cue a seek jumps over doesn’t.
• cues are plain, JSON-serializable data, so a server-rendered page can pass them in. A src is fetched only once the player runs on the client, and a cross-origin file needs CORS headers.
• Browsers load a <track> element’s cues only while the track is showing or hidden. default makes a <track kind="metadata"> hidden; without it, the track stays disabled and its cues never load.
• value isn’t checked at runtime unless the definition has a schema. See Check values with a schema.

Common variations

Read cues from a WebVTT file

Add a <track kind="metadata"> to the media. Cue text that parses as a JSON object or array becomes the cue’s value; any other text stays in text.

WEBVTT

00:00:01.000 --> 00:00:08.000
{ "title": "Timed metadata", "body": "This note came from a WebVTT file." }

<Video src="video.mp4" playsInline>
  <track kind="metadata" label="notes" src="/notes.vtt" default />
</Video>

A definition with the same kind and label reads it, so the rendering code from the recommended approach works unchanged. To use the file with media that has no <track> support, such as an embed, pass it as src when you add the track instead.

Read metadata from an HLS stream

When hls.js plays a stream, in HlsJsVideo and media built on it such as MuxVideo, it publishes ID3 frames and EXT-X-DATERANGE tags as cues on a metadata track. Read them with hlsMetadataTrack. Each cue’s type tells an ID3 frame from a date range attribute, and narrows its value.

import { hlsMetadataTrack, useActiveTextCues } from '@videojs/react';

function AdBadge() {
  const cues = useActiveTextCues(hlsMetadataTrack);
  const inAd = cues.some((cue) => cue.type === 'com.apple.quicktime.HLS' && cue.value?.key === 'X-AD-ID');

  return inAd ? <span className="ad-badge">Ad</span> : null;
}

Where the browser plays HLS natively, as Safari does, in-band metadata arrives as DataCues on the browser’s own metadata tracks. Read those with a { kind: 'metadata' } filter.

Mark cues on the time slider

<TimeSlider.Root>
  <TimeSlider.Track>
    <TimeSlider.Fill />
  </TimeSlider.Track>
  <TimeSlider.Markers filter={notesTrack} />
  <TimeSlider.Thumb />
</TimeSlider.Root>

A point is a marker with no width, a range spans its cue, and a marker has data-active while its cue is active. Markers are unstyled: position each one with its --media-slider-marker-* custom properties. See the markers reference for the full styling contract.

Check values with a schema

Cues from a file or a stream can carry anything. Pass a schema to defineTextTrack() to check each value where it’s read. Any Standard Schema works, such as one from zod or valibot, and the value type comes from the schema.

import { z } from 'zod';

const NoteSchema = z.object({ title: z.string(), body: z.string() });

const notesTrack = defineTextTrack({ kind: 'metadata', label: 'notes', schema: NoteSchema });

Readers skip cues whose value fails the schema, and warn in development. The schema must be synchronous.

Add chapters

Add a chapters track the same way. A chapter’s title is its cue text, and a chapter without an endTime runs to the next one. When the media has no chapters track of its own, the time slider shows these chapters.

import { type TextCueInit, useCreateTextTrack } from '@videojs/react';

const chapters: TextCueInit[] = [
  { startTime: 0, text: 'Introduction' },
  { startTime: 95, text: 'Setup' },
  { startTime: 410, text: 'Wrap-up' },
];

function Chapters() {
  useCreateTextTrack({ kind: 'chapters', cues: chapters });

  return null;
}

To load a Podcasting 2.0, Apple HLS, or JSON-LD chapters file, pass its converter as format, such as podcastChapters. Read the current chapter, with any artwork the file gives it, through chaptersTrack.

import { chaptersTrack, podcastChapters, useActiveTextCues, useCreateTextTrack } from '@videojs/react';

function CurrentChapter() {
  useCreateTextTrack({ kind: 'chapters', src: '/episodes/42/chapters.json', format: podcastChapters });

  const [chapter] = useActiveTextCues(chaptersTrack);

  return chapter ? <p className="current-chapter">{chapter.text}</p> : null;
}

Show an interactive transcript

A transcript reads captions cues, which the text cues feature skips unless something asks for them.

import { useActiveTextCues, useTextCues } from '@videojs/react';
import { usePlayer } from '@videojs/react/video';

const captions = { kind: 'captions', language: 'en' } as const;

function Transcript() {
  const store = usePlayer();
  const cues = useTextCues(captions);
  const [active] = useActiveTextCues(captions);

  return (
    <ol className="transcript">
      {cues.map((cue) => (
        <li key={cue.id} aria-current={cue.id === active?.id}>
          <button type="button" onClick={() => store.seek(cue.startTime)}>
            {cue.text}
          </button>
        </li>
      ))}
    </ol>
  );
}

The browser loads a <track> element’s cues only while captions are on. To show the transcript either way, add the captions track from your app instead of a <track> element. The player copies it into the media, so the browser still draws the captions when a viewer turns them on.

useCreateTextTrack({ kind: 'captions', label: 'English', language: 'en', src: '/captions.vtt' });

Troubleshooting

No cue is ever active

Confirm that:

• The track is one the text cues feature reads: a metadata or chapters track that isn’t disabled. Give a <track kind="metadata"> the default attribute. For other kinds, read through the hooks or call watchTextCues().
• Your filter or definition matches the track’s kind, label, and language exactly. getTextTracks() lists the tracks the player has.
• Cue times are in seconds.

A cue’s value is undefined

Only metadata cue text that parses as a JSON object or array becomes value; other text stays in text. With a schema, cues whose value fails it are skipped, with a warning in development.

An app track outlives its video

Cues from a src never load

The file failed to load, often because of a missing CORS header or a wrong URL. The track keeps any cues it already had, and a warning shows in development.

Related pages