Skip to content

Quality gate + mandatory fields #10

Description

@monneyboi

export-csv currently emits every extracted observation, with no filtering. We need to decide:

  • Which fields are mandatory for a row to be exported? Today the non-blank core in practice is dataset, source_url, snapshot_id, organisation_name, person_name, position_name (with its fallback). Every other field is blank when the page doesn't state it. Should we filter incomplete rows?
  • Should zavod consume every extraction, or only observations accepted by a Kolkhoz quality gate? A gate would likely be based on provenance signals rather than scoring — e.g. whether the value was extracted from plaintext vs. an image, whether it came with a page locator, whether the extracted string appears verbatim in the source document, etc.

Carries over from #4.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions