You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
AdaptivePlaywrightCrawler runs deferred cleanups differently from BasicCrawler #4248
BasicCrawler runs registerDeferredCleanup() callbacks on the pipeline's cleanup stack: in reverse registration order, with a thrown error caught and logged at debug level (#createBaseContext).
AdaptivePlaywrightCrawler.crawlOne collects them into its own array and runs them in a bare finally with Promise.all, so:
they run concurrently instead of in order
a throwing cleanup propagates out of crawlOne and fails the attempt (and a rejection from one cleanup doesn't wait for the others)
Two options:
align with BasicCrawler: sequential, reverse order, catch + log.debug
or wire the per-attempt registerDeferredCleanup into the sub-crawler pipeline's onCleanup so there is one code path
Came up while documenting registerDeferredCleanup in the crawling-context guide; the guide describes only the BasicCrawler behavior.
Activity