fix(utils): keep urls from the other sitemaps when one sitemap is malformed

[jayhemnani9910]

Sitemap.load() returns an empty list when any one of the sitemaps it reads fails to parse. That includes a single child of a sitemap index. One unescaped & is enough:

// robots.txt lists sitemap-products.xml (valid) and sitemap-blog.xml (has a `<loc>` with `?tag=a&b`)
const robots = await RobotsTxtFile.find('https://example.com/');
await robots.parseUrlsFromSitemaps(); // [], the product URLs are lost too

parseSitemap checks for parser errors right after it sets up the pipeline, but the pipeline reports them later. The error is thrown from the for await over the parsed items instead, escapes parseSitemap, and Sitemap.#parse catches it and discards every URL it has collected. RobotsTxtFile.parseUrlsFromSitemaps() passes no logger, so there it fails silently.

Now an error while reading one sitemap logs Malformed sitemap content: <url> and the parser moves on to the next sitemap. Raw string sources already log the error in their pipeline callback, so they are not logged twice. SitemapRequestLoader already read each sitemap on its own; for a malformed one it now gets this warning from parseSitemap instead of its own Error loading sitemap contents log.

Reproduced on 4.0.0-rc.1 and 3.18.2.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.