Exclude unlisted pages from llms.txt

[birosrichard]

Description

Pages with unlisted: true in their frontmatter are intentionally hidden from Docusaurus navigation and search, but the plugin was including them in llms.txt and llms-full.txt regardless. This adds a check in shouldIncludeRoute to skip any route where route.props?.unlisted is truthy.

Type of Change

• Bug fix

Testing

• Added Jest test file

Additional Notes

Docusaurus sets unlisted: true on the route props for pages with the matching frontmatter. The fix follows the same pattern as the existing categoryGeneratedIndex check in shouldIncludeRoute.

[Devon-White]

After reviewing the PR, I found that the original route.props?.unlisted check wasn’t enough. Docusaurus 3.10.2 doesn’t include that flag on docs, blog, or page routes—it keeps it in the loaded plugin metadata.

We updated the fix to read from that metadata, save the flag in the cache, and filter unlisted content from both llms.txt and llms-full.txt. We also invalidate older caches so CLI regeneration can’t accidentally add unlisted content back.