Identify the error.
Establish a comparison.

In speech, we examine recognition, speaker assignment and dialogue separately. In vision, we examine image and scene interpretation, continuity and the effects of a repair. Each task needs its own source material, baseline and acceptance criteria.
Our initial speech programme compared adapted checkpoints with unchanged foundation models. Broader visual-model and video-production comparisons remain ongoing or proposed work.
The tradeoff: a tightly defined test makes an error easier to explain, but its result may not extend to another language, scene or recording condition.
Read the evaluation methods ↗
