Generate seed file presigned URL in crawl operator, update prior to expiry - #3600

Merged

tw4l merged 12 commits into

mainfrom

issue-3576-seed-file-presigned-url

Sep 29, 2026

Commits on Sep 29, 2026

Generate new seed file presigned URL on each configmap load
tw4l
committed
54c94ba
Temp: Add working TODOs to reference later
tw4l
committed
2eef184
Only update configmap and get new presigned if crawler restarts/resumes
tw4l
committed
00a43dc
Switch to snake_case
tw4l
committed
9930f98
Set status.update_seed_file_presigned after any crawl pod exited
tw4l
committed
bf78345
Pass seed file id for scheduled crawls as well
tw4l
committed
cb21aac
Regenerate seed file presigned url when close to expiring
tw4l
committed
148a6f4
Make get_public_file_out consistent with get_file_out
tw4l
committed
3305ac4
Change to one hour
tw4l
committed
c7fe3a8
Add more logging
tw4l
committed
c52596f
Remove temporary debug logging
tw4l
committed
d22cd53
Reduce presigned URL expiry buffer to 5 mins
tw4l
committed
56bfd48