Searchable video archive

Find the moment in hours of recordings by what was said

Example design

01Video readyStream's webhook tells the Worker a recording has finished encoding. The Worker asks Stream to generate captions.
02Fetch captionsA queued job checks the caption status, and once the captions are ready the Worker fetches the WebVTT file.
03Split into passagesCues are grouped into passages of a few sentences, overlapping slightly, each with its start time.
04Embed and indexEach passage is embedded with a Workers AI text embedding model and stored in Vectorize with the video ID, start time and access group.
05SearchA viewer types a question in plain language. The Worker embeds it and asks Vectorize for the closest passages the viewer is allowed to see.
06ResultsEach result shows the passage, a thumbnail from Stream taken at that second, and a link that starts the video there.
07Play from the momentThe Stream player opens at the start time of the passage.
ifA name is misheardSpeech-to-text gets names and jargon wrong, and an exact search for a misheard name misses. When someone corrects the caption, the passage is re-embedded, so search and the captions agree again.

The problem

Most organisations have years of recorded meetings, lectures and webinars that nobody can search. File names say when something was recorded and nothing about what was said, and transcribing the archive used to be a project in itself. In this design the captions Stream generates become the search index. A result links to the second in the recording where something was said.

What it uses

Stream: Hosting, generated captions, thumbnails at a given time, and the player
Workers AI: Text embeddings for passages and queries
Vectorize: The passage index, with metadata for access filtering
D1: Full-text search for exact names and codes
Queues: Caption checks and re-indexing

The decisions, and why

TranscriptStream generated captions: The captions viewers see are also the search index, so there is one transcript to correct.
Passage sizeA few cues, with overlap: A single cue is too short to match a question, and a whole recording is too long to point at a moment.
Search typeSemantic and exact: Embeddings find "budget sign-off" when someone said "we approved the spend". Names and product codes need exact matching, which D1 full-text search handles.
AccessFiltered at query time: Each passage carries its access group in Vectorize metadata, so a search only returns recordings the viewer can already watch.

Limits to plan around

Generated captions cover 12 languages. Recordings in other languages need a different transcription step.
Search quality depends on caption quality, which is lowest for crosstalk, heavy accents and specialist vocabulary.

What you get

Indexing pipeline from webhook to Vectorize
Search API with access filtering
Search page with thumbnails and timed links
Re-indexing when a caption is corrected
Tests for access filtering
Runbook for backfilling an existing archive

Other projects

DRM solutions

Three corporate use cases, each with a Widevine demo that runs in your browser and checks what your device supports.

Talk about your video project

Tell us what you are building, or what is not working in the platform you have.

[email protected]

Registered office

VI Data Services Ltd
128 City Road, London, EC1V 2NX, United Kingdom

Telephone

020 8058 7616

Company number

17420568
Registered in England and Wales