Fire TV track · Amazon Developer Hackathon
A fan saves last night's replay for lunch, and scrubbing for a goal shows the score. Flipbook reads the trickplay file the player already downloaded — one small JPEG every 10 s — marks where the score box changes and holds, and lands 30 s before the last frame with the old score. It never decodes the video.
PICTURE · 2:20 · 320×180 thumbnail
READY>> next change · << previous
The trickplay file · 517 thumbnailsengine view
npm run trace
no backendno event feedno video decode
jumps landed before the play on the development match — our labels, not ground truth
bench · goalJumpsBeforePlayfalse markers: Flipbook vs a plain frame differ at its best threshold for the same two goals
bench · ablationBaselinepass 1 on the Vega virtual device: Amazon's own 428-frame BIF, 203 decoded, detector v1 build
VVD log · pass1 doneThe trickplay file we built from the CC0 match
Here is the whole development match as one contact sheet: 517 thumbnails, one every 10 s, every score box masked. We built the BIF from the CC0 broadcast with make-bif; the app on the virtual device fetched the same 6,575,564 bytes. The amber rule replays the scan's recorded read order — 66 sprocket samples first, then 11 bisect reads only where the box differed, then pass 2 through everything else.
Rows run four minutes each, 0:00 to 86:00. Frames 0–2 (the opening) and 515–516 (the credits) carry no score graphic; pass 2 counts them as skipped. In the app, blind mode shows none of this — only how far the scan has read. Switch the hero to Highlights to see the two changes circled here.
What the detector sees
These are the actual pixels inside the calibrated box around the first change, unmasked. Flipbook never reads the digits: it shrinks the box to a 64×5 luma grid, takes the median of three present frames on each side, and fires only if the top-3 cell distance clears τ = 20. The marker sits on the anchor — the last frame that still shows the old score — and goes 30 s before it.
Vega virtual device · Apple M-series · Vega SDK 0.24
These are captures from the Vega Virtual Device playing Amazon's own webinar trickplay file — the title the sample app itself streams, our "witness". Beside them, the device's log from the run with the detector's first build (5be6a4a), verbatim apart from the date, with elided lines marked.
→ 3:49 (−30 s lead-in); the seek bar's thumbnail area stays covered.15:45:57.572236 [flipbook] scan start frames=428 rect=0.050,0.050,0.900,0.90015:45:57.838317 [flipbook] ref frame=344 area=12,163,0,24015:45:58.010412 [flipbook] marker anchor=19926ms change=69741ms d=105.4 pass=115:45:58.134384 [flipbook] marker anchor=229147ms change=239110ms d=26.2 pass=115:45:58.217397 [flipbook] marker anchor=259036ms change=268999ms d=20.8 pass=1··· 31 more pass-1 marker lines and 1 line “box saved for witness-amazon-tv-webinar” ···15:46:01.656692 [flipbook] pass1 done n=203/428 markers=34 ms=4085··· 2 pass-2 marker lines ···15:46:06.401225 [flipbook] pass2 done added=2 lost=0 absent=0 ms=474515:46:06.401320 [flipbook] thumb mode=overlay video bytes decoded=015:46:42.838350 [flipbook] seek from=50572ms currentTime=199.1 anchor=229147ms15:46:44.342362 [flipbook] after seek currentTime=199.115:47:07.621282 [flipbook] seek from=226280ms currentTime=229.0 anchor=259036ms15:47:09.122096 [flipbook] after seek currentTime=229.0
displayAboveThumb); on this title the calibrated box covers most of the frame.
How it works
No backend, no feed, no model. Amazon's video sample extended with a TypeScript detector with no UI dependencies, and four Vega platform surfaces.
A BIF is a contact sheet of the whole match — one JPEG roughly every 10 s, downloaded so the seek bar can preview. The sample's BIF service fetches it; one added call hands Flipbook the same bytes, and Flipbook's own parser reads them as a corpus. Amazon's tv-webinar-hd.bif has 428 frames at 240×180 and a timestamp-multiplier field of 0xFFFFD8F0, so that parser sanity-checks the header and falls back to a 10 s interval.
jpeg-js decodes each thumbnail on the device; only the calibrated rectangle becomes a 64×5 luma grid. Cells that jitter — the pitch showing through — are masked out.
Pass 1 reads every 8th frame and bisects only where the box differs: on the development match both ticks were up after 77 of 517 decodes. Pass 2 still reads all 517 and can only add.
Medians of three present frames on each side: a one-frame flash cannot fire, and in synthetic tests neither does a replay wipe or a dropped graphic. The anchor walks back over any graphic-absent run to the last old-score frame.
useTVEventHandler takes the skip keys and sets currentTime to anchor − 30 s. On the VVD Flipbook's target was 199.1 s (anchor 229.1 s − 30 s), and 1.5 s later the player still reported 199.1 s — no second 30 s seek stacked on top.
The KUI SeekBar's displayAboveThumb draws film-base black over the score box on every scrub thumbnail; before calibration, thumbnails stay off. Blind mode is the default: no ticks, no counts, only how far the scan has read.
VideoPlayer (W3C media) — playback; the jump sets currentTimeSeekBar — markers for ticks, displayAboveThumb for the maskuseTVEventHandler — skip keys and calibration; TVFocusGuideView — seek-bar focusEvery number on this page is in the repo
The development match was labelled by the detector's author to tune thresholds, so these numbers show the pipeline works, not how well it generalises. The held-out result waits for a human's labels.
evidence/bench/dev-winchester-320-trace.json · D, same computation as scripts/visibility.ts
Two spikes clear τ = 20: 32.0 at 17:00 and 31.8 at 49:30. Outside those two three-frame spikes, the noisiest frame reaches 12.0 (18:50) — a 2.64× margin at 320 px, 2.04× at 240×180 letterboxed.
bench · ablationBaseline: consecutive frames, every cell, no hold
To keep both goals the differ fires 237 false markers; raise its threshold to 40 and it misses both and still fires 149. Without the hold and the jitter mask, Flipbook itself would mark 350 times.
bench · score.jumps, ablationAnchor (development labels)
PREREG.json, detector-params sha256 6f21e417…; the bench refuses matches A and B without a committed truth file.From the README's honest limits
“Virtual device only. No Fire TV hardware was available; every timing is ‘Vega virtual device on Apple M-series’.”
“The held-out headline is pending. … Until then no such number is claimed.”
“The numbers above come from one development match with two labelled changes, labelled by the detector's author from 2 s stills. The 95 % confidence interval on 2/2 is [0.158, 1].”
Questions a judge would ask
Yes, by construction: the scan code only ever opens the BIF's thumbnail JPEGs, decoded with jpeg-js. The bench and the device log print it — videoBytesDecoded: 0, video bytes decoded=0 — as a constant that records this, not as a measurement.
On the Vega Virtual Device (Vega SDK 0.24.12112) on an Apple M-series Mac. No Fire TV hardware was available, so every timing says so. Some on-device captures are still owed — a calibrated scan of the development match among them — and the README lists them.
On the development match — two goals, labelled by the detector's author — it found 2 of 2 with 0 false markers, and both jumps landed before the play (95 % CI [0.158, 1]). That shows the pipeline works, not that it generalises.
The held-out number (goal jumps landing before the play on match B) will exist only after a human labels matches A and B. The parameters are frozen in PREREG.json first.
Trickplay thumbnails are tiny — Amazon's are 240×180 — so Flipbook detects that the box changed and held, not what it says. No OCR, no ML, no LLM: arithmetic on a calibrated rectangle.
A little, and the README says so: a jump tells you a change lies ahead, "No later change" tells you none does, and a goal the detector missed is skipped without any sign.
The three matches are Belmont Media Center high-school broadcasts released CC0 1.0 on the Internet Archive. They show students and school marks; the marks are the schools', not ours, and CC0 does not cover trademarks. We mute the original commentary in everything we publish and use no close-ups. On this page the contact sheet and every seek-bar preview mask the score box; the stage's stills (the present, at the playhead) and the six crops show it on purpose.