Skip to content

AdaptivePlaywrightCrawler runs deferred cleanups differently from BasicCrawler #4248

Open

Description

@janbuchar

BasicCrawler runs registerDeferredCleanup() callbacks on the pipeline's cleanup stack: in reverse registration order, with a thrown error caught and logged at debug level (#createBaseContext).

AdaptivePlaywrightCrawler.crawlOne collects them into its own array and runs them in a bare finally with Promise.all, so:

  • they run concurrently instead of in order
  • a throwing cleanup propagates out of crawlOne and fails the attempt (and a rejection from one cleanup doesn't wait for the others)

Two options:

  • align with BasicCrawler: sequential, reverse order, catch + log.debug
  • or wire the per-attempt registerDeferredCleanup into the sub-crawler pipeline's onCleanup so there is one code path

Came up while documenting registerDeferredCleanup in the crawling-context guide; the guide describes only the BasicCrawler behavior.

Activity

  1. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Oct 9, 2026
  2. added this to the 4.0 milestone on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions