refactor(tasks): move background tasks to Postgres (pg-boss); remove node-resque + resque-admin - #519
Draft
evantahler wants to merge 3 commits into
Draft
evantahler wants to merge 3 commits into
evantahler wants to merge 3 commits into
Conversation
…nOutStore) Introduce a TaskBackend + FanOutStore seam beneath the unchanged public api.actions.* task interface. Both node-resque+Redis and pg-boss+Postgres are implemented as adapters, selected from config.tasks.backend / fanOutStore. - classes/TaskBackend.ts, classes/FanOutStore.ts — the two abstractions - backends/NodeResqueBackend.ts + RedisFanOutStore.ts — default adapters (zero behavior change) - backends/PgBossBackend.ts + PgFanOutStore.ts — Postgres adapters (drop Redis entirely) - initializers/tasks.ts — selects adapters, owns the shared task runner; api.resque kept as a deprecated alias - contract tests parameterized across both backends and both fan-out stores - recurring tasks stay single-instance with no leader (node-resque QueueLock/DelayQueueLock; pg-boss short-policy queue) closes #415 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… + resque-admin Research validated pg-boss as the best embeddable Postgres queue, so commit fully to Postgres for background tasks and drop the Redis/node-resque path entirely. - remove NodeResqueBackend + RedisFanOutStore and the node-resque dependency - PgBossBackend + PgFanOutStore are now the only implementations; TaskBackend and FanOutStore stay abstract as a thin internal seam - drop the TASKS_BACKEND / TASKS_FANOUT_STORE selectors and the api.resque alias - remove the resque-only api.actions introspection (locks, delLock, timestamps, delayedAt, allDelayed, workingOn, cleanOldWorkers, delByFunction, delQueue) - delete the @keryxjs/resque-admin plugin workspace (obsolete without node-resque) - framework + example default to Postgres; docs, env examples, and CLAUDE.md updated Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
evantahler
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move background tasks to Postgres (pg-boss); remove node-resque + resque-admin
Keryx's background-task system now runs entirely on Postgres via pg-boss — no Redis required for tasks. The public
api.actions.*task interface (enqueue,enqueueIn,enqueueAt,fanOut,fanOutStatus,taskDetails,stopRecurrentAction,Action.task) is unchanged; only the engine underneath changed.closes #415What changed
PgBossBackend(queue) andPgFanOutStore(fan-out state) behind thin abstract seams (TaskBackend/FanOutStore). Fan-out state lives in framework-managedkeryx_fanout/keryx_fanout_eventstables, created at boot. pg-boss owns its own schema (keryx_tasks).node-resquedependency,NodeResqueBackend,RedisFanOutStore, theTASKS_BACKEND/TASKS_FANOUT_STOREselectors, the deprecatedapi.resquealias, and the resque-onlyapi.actionsintrospection (locks,delLock,timestamps,delayedAt,allDelayed,workingOn,cleanOldWorkers,delByFunction,delQueue).@keryxjs/resque-adminplugin workspace (a Redis/resque dashboard, obsolete without node-resque).docs/guide/tasks.md+config.md, both.env.examples,README.md, andCLAUDE.mdupdated to the Postgres story.Recurring tasks stay single-instance without a leader
The one hard requirement with no obvious Postgres answer. Recurring actions are routed to a dedicated pg-boss queue created with the
shortpolicy, whose unique index permits exactly one pending job per action cluster-wide and frees the slot the moment the job starts — so the job's own re-enqueue succeeds while duplicates are rejected. Verified by a test: 5 concurrent enqueues → 1 pending copy.Why pg-boss (research)
A fact-checked comparison of embeddable Postgres queues narrowed the field to pg-boss and graphile-worker (pgmq has no official TS client + no managed retries/DLQ; Hatchet is a gRPC client to a separate engine). pg-boss wins on maturity (post-1.0, ~754k weekly downloads), verified
SKIP LOCKED+ dead-letter-with-redrive, and an integration that's Bun-smoke-tested and green. graphile-worker's edges (native TS, verified leaderless cron) are neutralized here since we solve recurring dedup ourselves.Verification
🤖 Generated with Claude Code