Skip to content

[Docs] Document video caption-quality evaluation with Summarize-then-Align #2125

Description

@lbliii

Context

Child of #2118. #1980 added eval/video/ tooling for caption-quality evaluation with CosmosEmbed1. Documentation exists only in eval/video/README.md, outside the published Fern site.

Scope

  • Benchmark-dataset construction: sampling, embedding, K-means selection, and expected artifacts
  • Caption scoring: summarization, CosmosEmbed1 encoding, cosine similarity, caching, and CSV outputs
  • Model/data prerequisites and expected directory layouts
  • Interpretation and limitations of the baseline scores

Acceptance criteria

  • A published Fern evaluation guide is added and linked from video captioning docs
  • Both scripts have tested commands and complete argument explanations
  • Input/output schemas and directory layouts are shown
  • Hardware/model-download expectations and reproducibility limits are explicit
  • The distinction between evaluation tooling and the main curation pipeline is clear
  • Fern checks pass

Related PR: #1980

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions