Skip to content

feat: recommend trip_headsign - #2192

Merged
davidgamez merged 2 commits into
MobilityData:masterfrom
mackenziereading19:1801-recommend-trip-headsign
Sep 17, 2026
Merged

davidgamez merged 2 commits into
MobilityData:masterfrom
mackenziereading19:1801-recommend-trip-headsign

Conversation

@mackenziereading19

Copy link
Copy Markdown
Contributor

Summary

Fixes #1801.

Marks trip_headsign as a recommended GTFS field by adding @Recommended to GtfsTripSchema.tripHeadsign().

The generated GtfsTripTableDescriptor then classifies trip_headsign as FieldLevelEnum.RECOMMENDED, so the existing parser emits MissingRecommendedFieldNotice when the field is present but empty.

Tests

Added focused loader regression coverage for:

  • empty trip_headsign -> MissingRecommendedFieldNotice
  • populated trip_headsign -> no missing-recommended notice

Local verification:

  • GtfsTripTableLoaderTest: 2 passed
  • TripHeadsignValidatorTest: 8 passed
  • FeedMetadataTest: passed
  • generated descriptor confirms FieldLevelEnum.RECOMMENDED
  • :main:spotlessCheck: passed

Implementation commit: 16357dcc680754633ca882d042a71109c497e00d

@emmambd
emmambd requested a review from skalexch September 17, 2026 14:22

@skalexch skalexch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on a couple feeds and works well! All good on the spec and tests side.

@davidgamez
davidgamez self-requested a review September 17, 2026 18:01
@davidgamez

Copy link
Copy Markdown
Member

📝 Acceptance Test Report

📋 Summary

❌ The rule acceptance test has failed for commit 6c03b76
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (0 out of 1001 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Errors (0 out of 1001 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (224 out of 1001 datasets, ~22%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
mdb-1000 missing_recommended_field
mdb-1003 missing_recommended_field
mdb-1004 missing_recommended_field
mdb-1027 missing_recommended_field
mdb-103 missing_recommended_field
mdb-1038 missing_recommended_field
mdb-106 missing_recommended_field
mdb-1067 missing_recommended_field
mdb-107 missing_recommended_field
mdb-1076 missing_recommended_field
mdb-1082 missing_recommended_field
mdb-1085 missing_recommended_field
mdb-1091 missing_recommended_field
mdb-1098 missing_recommended_field
mdb-1103 missing_recommended_field
mdb-1111 missing_recommended_field
mdb-1127 missing_recommended_field
mdb-1129 missing_recommended_field
mdb-1131 missing_recommended_field
mdb-1132 missing_recommended_field
mdb-116 missing_recommended_field
mdb-1172 missing_recommended_field
mdb-1173 missing_recommended_field
mdb-119 missing_recommended_field
mdb-1227 missing_recommended_field
mdb-1258 missing_recommended_field
mdb-1260 missing_recommended_field
mdb-1262 missing_recommended_field
mdb-1265 missing_recommended_field
mdb-1291 missing_recommended_field
mdb-13 missing_recommended_field
mdb-1309 missing_recommended_field
mdb-15 missing_recommended_field
mdb-16 missing_recommended_field
mdb-1813 missing_recommended_field
mdb-1830 missing_recommended_field
mdb-1837 missing_recommended_field
mdb-1870 missing_recommended_field
mdb-1871 missing_recommended_field
mdb-1902 missing_recommended_field
mdb-1924 missing_recommended_field
mdb-1961 missing_recommended_field
mdb-1962 missing_recommended_field
mdb-1970 missing_recommended_field
mdb-2 missing_recommended_field
mdb-2002 missing_recommended_field
mdb-2012 missing_recommended_field
mdb-2014 missing_recommended_field
mdb-2020 missing_recommended_field
mdb-2021 missing_recommended_field
mdb-2022 missing_recommended_field
mdb-2023 missing_recommended_field
mdb-2033 missing_recommended_field
mdb-2034 missing_recommended_field
mdb-2036 missing_recommended_field
mdb-2077 missing_recommended_field
mdb-2079 missing_recommended_field
mdb-2132 missing_recommended_field
mdb-2164 missing_recommended_field
mdb-2165 missing_recommended_field
mdb-2172 missing_recommended_field
mdb-2174 missing_recommended_field
mdb-2180 missing_recommended_field
mdb-2186 missing_recommended_field
mdb-2187 missing_recommended_field
mdb-2190 missing_recommended_field
mdb-220 missing_recommended_field
mdb-2200 missing_recommended_field
mdb-2202 missing_recommended_field
mdb-2210 missing_recommended_field
mdb-2231 missing_recommended_field
mdb-2237 missing_recommended_field
mdb-2243 missing_recommended_field
mdb-2249 missing_recommended_field
mdb-2250 missing_recommended_field
mdb-226 missing_recommended_field
mdb-227 missing_recommended_field
mdb-2275 missing_recommended_field
mdb-2276 missing_recommended_field
mdb-2312 missing_recommended_field
mdb-2317 missing_recommended_field
mdb-2324 missing_recommended_field
mdb-236 missing_recommended_field
mdb-2364 missing_recommended_field
mdb-2371 missing_recommended_field
mdb-2372 missing_recommended_field
mdb-2373 missing_recommended_field
mdb-2381 missing_recommended_field
mdb-2393 missing_recommended_field
mdb-2407 missing_recommended_field
mdb-2446 missing_recommended_field
mdb-2447 missing_recommended_field
mdb-25 missing_recommended_field
mdb-2519 missing_recommended_field
mdb-2611 missing_recommended_field
mdb-2615 missing_recommended_field
mdb-2648 missing_recommended_field
mdb-2651 missing_recommended_field
mdb-2668 missing_recommended_field
mdb-2688 missing_recommended_field
mdb-2690 missing_recommended_field
mdb-2721 missing_recommended_field
mdb-2724 missing_recommended_field
mdb-2725 missing_recommended_field
mdb-2726 missing_recommended_field
mdb-2727 missing_recommended_field
mdb-2728 missing_recommended_field
mdb-2729 missing_recommended_field
mdb-2731 missing_recommended_field
mdb-2732 missing_recommended_field
mdb-2733 missing_recommended_field
mdb-2734 missing_recommended_field
mdb-2735 missing_recommended_field
mdb-2736 missing_recommended_field
mdb-2737 missing_recommended_field
mdb-2748 missing_recommended_field
mdb-2756 missing_recommended_field
mdb-2757 missing_recommended_field
mdb-2758 missing_recommended_field
mdb-2759 missing_recommended_field
mdb-2760 missing_recommended_field
mdb-2761 missing_recommended_field
mdb-2773 missing_recommended_field
mdb-2775 missing_recommended_field
mdb-2776 missing_recommended_field
mdb-2779 missing_recommended_field
mdb-2790 missing_recommended_field
mdb-2791 missing_recommended_field
mdb-2796 missing_recommended_field
mdb-28 missing_recommended_field
mdb-2801 missing_recommended_field
mdb-2808 missing_recommended_field
mdb-2811 missing_recommended_field
mdb-2812 missing_recommended_field
mdb-2814 missing_recommended_field
mdb-2817 missing_recommended_field
mdb-2831 missing_recommended_field
mdb-2853 missing_recommended_field
mdb-2867 missing_recommended_field
mdb-2881 missing_recommended_field
mdb-2882 missing_recommended_field
mdb-2894 missing_recommended_field
mdb-2910 missing_recommended_field
mdb-2920 missing_recommended_field
mdb-2926 missing_recommended_field
mdb-2928 missing_recommended_field
mdb-2929 missing_recommended_field
mdb-2940 missing_recommended_field
mdb-2941 missing_recommended_field
mdb-2942 missing_recommended_field
mdb-2943 missing_recommended_field
mdb-2944 missing_recommended_field
mdb-2945 missing_recommended_field
mdb-2946 missing_recommended_field
mdb-2947 missing_recommended_field
mdb-296 missing_recommended_field
mdb-298 missing_recommended_field
mdb-2994 missing_recommended_field
mdb-2995 missing_recommended_field
mdb-2996 missing_recommended_field
mdb-305 missing_recommended_field
mdb-314 missing_recommended_field
mdb-35 missing_recommended_field
mdb-383 missing_recommended_field
mdb-389 missing_recommended_field
mdb-48 missing_recommended_field
mdb-496 missing_recommended_field
mdb-522 missing_recommended_field
mdb-536 missing_recommended_field
mdb-538 missing_recommended_field
mdb-553 missing_recommended_field
mdb-570 missing_recommended_field
mdb-572 missing_recommended_field
mdb-578 missing_recommended_field
mdb-58 missing_recommended_field
mdb-583 missing_recommended_field
mdb-584 missing_recommended_field
mdb-588 missing_recommended_field
mdb-606 missing_recommended_field
mdb-609 missing_recommended_field
mdb-61 missing_recommended_field
mdb-62 missing_recommended_field
mdb-646 missing_recommended_field
mdb-665 missing_recommended_field
mdb-68 missing_recommended_field
mdb-681 missing_recommended_field
mdb-69 missing_recommended_field
mdb-690 missing_recommended_field
mdb-696 missing_recommended_field
mdb-70 missing_recommended_field
mdb-720 missing_recommended_field
mdb-769 missing_recommended_field
mdb-770 missing_recommended_field
mdb-771 missing_recommended_field
mdb-772 missing_recommended_field
mdb-774 missing_recommended_field
mdb-780 missing_recommended_field
mdb-783 missing_recommended_field
mdb-79 missing_recommended_field
mdb-799 missing_recommended_field
mdb-806 missing_recommended_field
mdb-808 missing_recommended_field
mdb-809 missing_recommended_field
mdb-814 missing_recommended_field
mdb-85 missing_recommended_field
mdb-850 missing_recommended_field
mdb-854 missing_recommended_field
mdb-86 missing_recommended_field
mdb-861 missing_recommended_field
mdb-867 missing_recommended_field
mdb-868 missing_recommended_field
mdb-869 missing_recommended_field
mdb-875 missing_recommended_field
mdb-877 missing_recommended_field
mdb-886 missing_recommended_field
mdb-893 missing_recommended_field
mdb-900 missing_recommended_field
mdb-906 missing_recommended_field
mdb-92 missing_recommended_field
mdb-924 missing_recommended_field
mdb-928 missing_recommended_field
mdb-93 missing_recommended_field
mdb-930 missing_recommended_field
mdb-992 missing_recommended_field
Dropped Warnings (0 out of 1001 datasets, ~0%) ✅

No changes were detected due to the code change.

New Info Notices (0 out of 1001 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Info Notices (0 out of 1001 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

2 out of 1003 sources (~0 %) are corrupted.
Dataset Ref Report Exists Ref Report Readable Latest Report Exists Latest Report Readable
mdb-1114
mdb-1123
🔍 System errors for mdb-1114 (reference)
[
  {
    "code": "i_o_error",
    "severity": "ERROR",
    "totalNotices": 1,
    "sampleNotices": [
      {
        "exception": "java.util.zip.ZipException",
        "message": "Archive is not a ZIP archive"
      }
    ]
  }
]
🔍 System errors for mdb-1114 (latest)
[
  {
    "code": "i_o_error",
    "severity": "ERROR",
    "totalNotices": 1,
    "sampleNotices": [
      {
        "exception": "java.util.zip.ZipException",
        "message": "Archive is not a ZIP archive"
      }
    ]
  }
]
🔍 System errors for mdb-1123 (reference)
[
  {
    "code": "i_o_error",
    "severity": "ERROR",
    "totalNotices": 1,
    "sampleNotices": [
      {
        "exception": "java.util.zip.ZipException",
        "message": "Archive is not a ZIP archive"
      }
    ]
  }
]
🔍 System errors for mdb-1123 (latest)
[
  {
    "code": "i_o_error",
    "severity": "ERROR",
    "totalNotices": 1,
    "sampleNotices": [
      {
        "exception": "java.util.zip.ZipException",
        "message": "Archive is not a ZIP archive"
      }
    ]
  }
]

💾 Out of Memory Check

No datasets experienced an OutOfMemoryError.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 5.73 5.99 ⬆️+0.26
Median -- 1.55 1.90 ⬆️+0.36
Standard Deviation -- 26.90 26.45 ⬇️-0.45
Minimum in References Reports mdb-1936 0.30 0.64 ⬆️+0.34
Maximum in Reference Reports mdb-2014 756.02 740.24 ⬇️-15.79
Minimum in Latest Reports mdb-1105 0.35 0.48 ⬆️+0.13
Maximum in Latest Reports mdb-2014 756.02 740.24 ⬇️-15.79
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 580.31 MiB 579.37 MiB ⬇️-964.90 KiB
Median -- 323.93 MiB 323.93 MiB ⬆️+1016 bytes
Standard Deviation -- 1.06 GiB 1.04 GiB ⬇️-12.48 MiB
Minimum in References Reports mdb-77 38.44 MiB 41.24 MiB ⬆️+2.80 MiB
Maximum in Reference Reports mdb-2393 10.17 GiB 9.63 GiB ⬇️-544.51 MiB
Minimum in Latest Reports mdb-651 54.02 MiB 40.19 MiB ⬇️-13.83 MiB
Maximum in Latest Reports mdb-2393 10.17 GiB 9.63 GiB ⬇️-544.51 MiB

@mackenziereading19

Copy link
Copy Markdown
Contributor Author

Thanks, I’ve reviewed the acceptance-test result.

The 224 new missing_recommended_field warnings appear to be the expected effect of marking trip_headsign as recommended: feeds where the field is present but empty now trigger the existing recommended-field notice.

The run shows:

  • 0 new errors
  • 0 dropped errors
  • 224 new warnings, all missing_recommended_field
  • 0 dropped warnings
  • 0 new/dropped info notices

The two corrupted sources also show the same ZIP errors in both reference and latest reports.

So I don’t currently see evidence of an unintended regression here; the failure appears to come from the acceptance workflow’s 1% warning threshold being exceeded by the intended validation change.

Happy to adjust anything if you’d prefer a different treatment.

@davidgamez

Copy link
Copy Markdown
Member

Thanks, I’ve reviewed the acceptance-test result.

The 224 new missing_recommended_field warnings appear to be the expected effect of marking trip_headsign as recommended: feeds where the field is present but empty now trigger the existing recommended-field notice.

The run shows:

  • 0 new errors
  • 0 dropped errors
  • 224 new warnings, all missing_recommended_field
  • 0 dropped warnings
  • 0 new/dropped info notices

The two corrupted sources also show the same ZIP errors in both reference and latest reports.

So I don’t currently see evidence of an unintended regression here; the failure appears to come from the acceptance workflow’s 1% warning threshold being exceeded by the intended validation change.

Happy to adjust anything if you’d prefer a different treatment.

Hi @mackenziereading19, Thanks for reviewing the acceptance tests. This is standard procedure to ensure the implementation is valid against real datasets. The failure lets us check cases that affect a large number of feeds. This case is expected.

@davidgamez davidgamez left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidgamez
davidgamez merged commit 0c0bfed into MobilityData:master Sep 17, 2026
102 of 103 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add trip_headsigns as recommended field

4 participants