Skip to content

Isolate worksheet-local property mappings to prevent cross-sheet metadata corruption and spurious header ingestion - #55

Open
Montana wants to merge 1 commit into
dhs-gov:mainfrom
Montana:fix-worksheet-column-mapping
Open

Montana wants to merge 1 commit into
dhs-gov:mainfrom
Montana:fix-worksheet-column-mapping

Conversation

@Montana

@Montana Montana commented Sep 20, 2026

Copy link
Copy Markdown

Hi Homeland Security,

This PR fixes a worksheet-boundary state management defect in TemplateReader.inventory_records() that can corrupt imported metadata or terminate Excel ingestion with a KeyError.

The property mapping is currently initialized outside the worksheet loop. After processing the first inventory worksheet, subsequent worksheets inherit its column-to-property associations and bypass header initialization.

This produces three observable failures:

  • Subsequent worksheet headers are imported as inventory records.
  • Reordered columns silently associate values with incorrect DCAT properties, including assigning dataset titles to identifiers.
  • Additional populated columns absent from the original mapping raise a KeyError.

The change moves props = None inside the worksheet loop. Each inventory worksheet independently initializes its mapping from its own header comments and skips that header before processing records. Existing record construction and blank-row handling remain unchanged.

The accompanying parametrized regression tests exercise eight combinations: identical, reordered, expanded, and reduced column layouts, each with either a populated or header-only first inventory worksheet. The fixtures also include blank rows and a non-inventory instructions sheet.

Cheers,
Michael

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant