CLEARCUT / benchmark · open in console

Detection benchmark

Every film below was built by us with clearance landmines at known timestamps, so every number here is checkable: the ground-truth JSON and the film sit side by side in the repo (spike/films/*), and spike/scripts/run_evals.py regenerates this page.

Headline

By category

Flag categoryPlantedDetectedRecall

By film

Method

A planted item counts as detected when a prediction of a compatible type overlaps its true window by ≥ 2 s. Predictions that match nothing are counted as spurious; those falling inside declared negative zones (title cards, empty rooms, credits) are the strict false positives. Music cues pass when both boundaries are within ±3 s of truth. Detection is a visual pass over the film plus an audio pass where the ruler proposes and the model labels: ffmpeg silencedetect finds the sound windows deterministically, Gemini says which contain music (and may split a window), and a loudness backstop rejects any cue whose window is silent. An earlier prompt-only audio pass scored 4/5 cues on this set — it heard the backlot piano but placed it at the wrong time; the ruler fixed that. Free-tier models; no fine-tuning; no prompt contains the answers.