Skip to content

Commit 1d4ae41

Browse files
authored
test: fix flaky test_crawler_intermediate_statistics on Windows (#2265)
`test_crawler_intermediate_statistics` failed on Windows 3.12 ([run](https://github.com/apify/crawlee-python/actions/runs/36837667765/job/110288921245?pr=2263)) with `98582 µs >= 100000 µs`. The test sleeps 100 ms with `asyncio.sleep` and expects `crawler_runtime` to be at least that long, but `crawler_runtime` comes from `datetime.now()`. On Windows before Python 3.13 that clock ticks every ~15.6 ms, and the only slack in the test was the poll lag, which can be close to zero. The test now takes its own `datetime.now()` readings when it sees the crawler start and after the sleep, and asserts the runtime is at least the gap between them. Both sides use the same clock, and the check still fails if the runtime isn't updated while the crawler runs. *✍️ Drafted by Claude Code*
1 parent ce96578 commit 1d4ae41

1 file changed

Lines changed: 7 additions & 3 deletions

File tree

‎tests/unit/crawlers/_basic/test_basic_crawler.py‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
from collections import Counter
1313
from concurrent.futures import ProcessPoolExecutor
1414
from dataclasses import dataclass
15-
from datetime import timedelta
15+
from datetime import datetime, timedelta, timezone
1616
from itertools import product
1717
from typing import TYPE_CHECKING, Any, Literal, cast
1818
from unittest.mock import ANY, AsyncMock, Mock, call, patch
@@ -2446,11 +2446,15 @@ async def handler(_: BasicCrawlingContext) -> None:
24462446
# has enough time to reach the active state on slow CI runners under xdist load.
24472447
crawler_task = asyncio.create_task(crawler.run(['https://a.placeholder.com']))
24482448
assert await poll_until_condition(lambda: crawler.statistics.active, timeout=30)
2449+
active_seen_at = datetime.now(timezone.utc)
24492450

2450-
# Wait some time and check that runtime is updated.
2451+
# Wait some time and check that runtime is updated. The runtime is measured on the wall clock, which on Windows
2452+
# ticks at ~15.6 ms independently of the `asyncio.sleep` clock, so bound it by a wall-clock interval inside the run.
24512453
await asyncio.sleep(check_time.total_seconds())
2454+
min_runtime = datetime.now(timezone.utc) - active_seen_at
24522455
crawler.statistics.calculate()
2453-
assert crawler.statistics.state.crawler_runtime >= check_time
2456+
assert min_runtime > timedelta(0)
2457+
assert crawler.statistics.state.crawler_runtime >= min_runtime
24542458

24552459
# Wait for crawler to finish
24562460
await crawler_task

0 commit comments

Comments
 (0)