Map itemprop meta tags to JSON-LD. - #124
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support for extracting and normalizing JSON-LD data and HTML microdata (using
itemscope,itemtype,itemid, anditemprop) into JSON-LD-style maps in thez_url_metadatamodule. The metadata extraction now returns ajson_ldproperty containing all decoded JSON-LD blocks and normalized microdata, making it easier to work with structured data from web pages. The changes include updates to the metadata structure, extraction logic, property access, and comprehensive tests for these features.JSON-LD and Microdata Extraction:
Added extraction and normalization of JSON-LD and HTML microdata into a new
json_ldproperty in the metadata structure, with support for both<script type="application/ld+json">blocks and itemscope-based microdata. Microdata is normalized to JSON-LD maps, with schema.org types using"https://schema.org"as@context. (F8036545L541R1028, [1] [2] [3] [4]Updated the
p/2property accessor and metadata type definitions to support the newjson_ldproperty. [1] [2] [3]HTML Attribute and Metadata Handling:
Added support for recognizing
itemid,itemprop,itemscope, anditemtypeas valid HTML attributes to enable correct parsing of microdata.Improved handling of meta tag property names by normalizing certain property names to lower case for consistency (e.g.,
datePublished→datepublished).Documentation and Testing:
Updated the
README.mdto document how to access the newjson_ldproperty and describe the normalization process for microdata.Added comprehensive tests for extracting and normalizing JSON-LD and microdata, including cases for YouTube video metadata, nested microdata, and JSON-LD scripts.
Dependencies:
jsxrecordas a dependency for decoding JSON-LD data.