Skip to content

feat(ai): multi-language + force regeneration in generate modal, and per-article AI overview board - #2804

Merged
Innei merged 26 commits into
masterfrom
feat/admin-ai-generate-multilang
Aug 10, 2026
Merged

feat(ai): multi-language + force regeneration in generate modal, and per-article AI overview board#2804
Innei merged 26 commits into
masterfrom
feat/admin-ai-generate-multilang

Conversation

@Innei

@Innei Innei commented Aug 10, 2026

Copy link
Copy Markdown
Member

本分支载两件相关的 AI 模块工作:生成弹窗的多语言与强制重生,以及单文章 AI 概览看板。

1. 生成弹窗:多语言输入 + 强制重新生成

四种生成(摘要 / 精读 / 翻译 / 朗读)共用的 GeneratePromptModal

  • 语言改为逗号分隔(半角 , 与全角 皆可),输入框下方实时渲染解析后的 chips 与计数,去重、折叠大小写、保持首次出现顺序。上限 8 种。
  • 一次提交派发覆盖多种语言的任务。后端 task payload 本就收数组,此前的单语言限制纯粹来自前端。
  • 新增**「强制重新生成」**勾选,四种生成一律可用;精读无语言参数,弹窗只剩它。
  • 语言留空是合法提交,语义为「按 AI 设置中的目标语言」;摘要与翻译的默认值从 /options/ai 拉取。

后端补齐 force 语义(TTS 本就有,摘要 / 精读 / 翻译此前没有):

  • 三个 DTO 与 payload 加 forcecomputeAITaskDedupKey 加 force 位,否则一个在跑的普通任务会把随后的强制任务当重复吞掉。
  • AiInFlightServicebypassResultCache:force 时不读 resultKey,并在确认拿到 lockKey 之后才删它——若在拿锁前删,删除与拿锁之间的窗口会让并发的无关请求看到 result / error / lock 三者皆空而误判「无结果」抛错。
  • buildSummaryKey / buildInsightsKey / buildTranslationKey 三处 in-flight key 并入 force。此前它们只由 articleId / lang / 内容哈希决定,与含 force 的去重键脱节:force 请求撞上仍在跑的普通请求会退化为 follower,静默复用其增量输出,用户以为强制生效实则未生效。现在两者各走独立的 Redis 锁 / 流 / 结果键,重复的 force 之间仍合流。
  • 翻译 force 时不把 existing 传给 translateContentStream,整篇重译;existing 仍读取,用于 sourceModifiedAt 回退与 create / update 事件判定。force 亦透传进 PartialFailed 的自动重试 payload,否则部分失败后的重试会悄悄退回增量。
  • 语言上限 MAX_LANGS_PER_TASK = 8 提为共用常量,两个 DTO 加 .max(),前端遂退化为 UX 提示而非唯一防线。

顺带修掉一个潜伏 bug:admin 旧的 createSummaryTask 发的是 lang 字段,而它从来不在 CreateSummaryTaskSchema 里,被全局 Zod pipe 静默剥离——摘要弹窗的语言输入此前一直是空操作,永远回退配置。

设计文档见 docs/superpowers/specs/2026-08-10-admin-ai-generate-modal-multilang-design.md

2. 单文章 AI 概览看板

新增 /ai/overview/ai/overview/:id/ai 由指向 /ai/summary 改为指向此处),按文章聚合四种 AI 资产的覆盖矩阵、成本汇总与在途任务列表。后端新增 GET /ai/overview/groupedGET /ai/overview/article/:id

其中两处失败可见性的修复值得单说——此前生成失败是完全无声的:toast 报「已创建任务」,转圈一闪而逝,用户无从得知发生了什么。

  • 服务端 activeTasks 现含近 10 分钟内的失败任务及其原因(worker 常把原因记在 error 级日志而非 task.error,故两处都读);矩阵新增红色 failed 格态,点击即重试;任务列表对失败行显示原因、重试与详情链接。
  • 轮询修复两条:其一,refetchInterval 在窗口失焦时被 TanStack 暂停,点击后那一次抓取恰好捕到 pending 便再无第二次,看板永远停在那一帧——加 refetchIntervalInBackground: true,进度视图本就该在用户切走时仍然诚实。其二,乐观 pending 项靠条件自清而条件未必成立,可能熬过所有退场条件成为永久幻影——今删之,服务端已同时上报在途与近期失败,客户端不必自造状态;另加 20 秒派发宽限窗,保证轮询活到队列有话可说。

测试

  • core:AI 相关 spec 全绿(in-flight、dedup key、summary / insights / translation service、overview 三件)。
  • admin:src/features/ai 39 项全绿。
  • 两 app tsc --noEmit 皆 0 error。

apps/admin 的 lint 脚本依赖 oxlint,本机未安装,admin 侧以 tsc --noEmit 代偿。

已知遗留

  1. parseLangInput 不归一语言码:zh-CN, zh 在摘要 / 翻译会生成两条重复记录并双倍计费,TTS 则折叠为一种。同一输入框,三种能力三种结果。
  2. 精读 force 重生后,派生译文因正文哈希未变而不失效,与新正文脱节。
  3. AI 设置的目标语言为空 + 弹窗留空提交 = 记为成功的空任务,仅一条 warn 日志。新的「留空」提示会放大命中率。
  4. AiInFlightService 非 bypass 路径下 parseResult 抛错后的 delsetNX 窗口是与本次所修同类的竞态。

Innei added 15 commits August 10, 2026 16:16
…ation tasks

Thread force through the summary/insights/translation task payloads and
generation pipelines, mirroring the existing TTS force behavior: dedup keys
split force vs incremental so a forced task cannot be swallowed by an
in-flight regular one, and AiInFlightService.runWithStream gains
bypassResultCache to skip (and evict) a cached result on force. Translation
additionally drops block-level incremental reuse on force while still
reading the existing row for its other bookkeeping uses.
…load

buildTranslationRetryTask rebuilt retryPayload without the original task's
force flag, so a force task that partially failed would silently retry as
incremental and hit the in-flight result cache instead of regenerating.
…ate modal

parseLangInput normalizes comma-separated (incl. full-width comma) lang
input into a deduped, ordered list. GeneratePromptModal now renders live
lang chips with a count, caps input at 8 languages, and adds a "force
regenerate" checkbox shown across all four generate flows. Call sites
are updated in a follow-up task.
tooMany was computed unconditionally from the parsed lang input, but langs
are only actually submitted when promptForLang is true. A caller passing
defaultLangs with >8 entries alongside promptForLang: false silently
disabled the submit button with no visible error.
… commit

An unrelated in-progress rename (apps/admin/src/views/(intelligence)/ai/page.tsx
-> ai/overview/page.tsx) from a concurrent session was staged in the shared
index and got swept into the prior commit. This restores the tracked path to
its pre-existing location without touching any working-tree file contents,
so that session's uncommitted work is unaffected.
Connects GeneratePromptModal's {langs, force} result to all four AI
generate flows (summary, translation, tts, insights) plus the quick
actions menu and the article detail generate button. Adds
useAiDefaultLangs to prefill the modal from the AI config's
summary/translation target-language settings. createSummaryTask now
takes targetLanguages (array) instead of a single lang, matching the
backend DTO; createInsightsTask and createTranslationTask gain
force. Inline retranslate/regenerate row actions always pass
force: true.
Adds a reverse view of the AI surfaces: given one article, which assets
exist, what is missing, and what the generation has cost.

Server: GET /ai/overview/grouped lists every article (posts, notes,
pages) newest first with a compact per-capability language projection;
GET /ai/overview/article/:id returns the assets, a per-resource-type
cost roll-up summed across every generation ever recorded, and the AI
tasks currently in flight for that article.

Admin: /ai/overview holds a coverage matrix of capability x language.
An empty cell dispatches the generation task, a filled one scrolls to
its asset row, and a queued one shows a spinner so a second click
cannot duplicate the work. Languages beyond the configured targets can
be added as ad-hoc columns.
runWithStream's bypassResultCache path deleted resultKey before
attempting the lock, opening a window where an unrelated concurrent
follower for the same key could observe resultKey, errorKey, and
lockKey all empty and throw. Defer the delete until this instance
actually holds lockKey, and skip reading the cached result entirely
on bypass instead of racing a delete against the read.

Also give summary/translation task DTOs the same targetLanguages cap
TTS already enforces at runtime, sourced from one shared
MAX_LANGS_PER_TASK constant instead of a second literal.
The insights quick-action passes promptForLang: false yet calls the
same runWithLangPrompt helper, discarding its first arg as _langs.
Rename only — the helper drives every generate-prompt flow, not just
the language-prompting ones.
buildSummaryKey/buildInsightsKey/buildTranslationKey hashed only
articleId/lang/content, so a force request racing an in-flight plain
request for the same key lost the lock, became a follower, and
silently replayed the plain leader's incremental output — force
appeared to succeed but never regenerated anything. Fold force into
each hash so force and plain requests never share a lock/stream/result
key; repeated force requests still converge on the same key.
Records the design for the multi-language generate prompt modal and
core force-regeneration semantics, updated with the actual in-flight
key behavior (force and plain requests now use independent Redis
lock/stream/result keys instead of the originally-accepted "force
follows plain" degradation).
…ions

Extends the per-article AI overview board with an active-task list, an add-language control and a grouped overview section; core side gains active-task derivation for the overview payload.
…settles

Clicking a coverage cell left a spinner that never resolved. Two causes,
both found by driving the board in a browser against a dev server whose
AI feature was off, so every task failed within a millisecond:

- refetchInterval is suspended while the window is unfocused, so the one
  fetch that caught the task mid-queue was also the last one. The board
  then displayed that frozen snapshot forever. refetchIntervalInBackground
  keeps a progress view honest when the user switches away.
- An optimistic pending entry bridged the gap before the queue registers a
  task, retired by conditions that did not always hold. It could outlive
  every one of them and become a permanent phantom. The server now reports
  live tasks and recent failures alike, so the client no longer invents
  state; a dispatch grace window keeps polling alive until the queue has
  something to say.

Polling now runs while anything is live or within 20s of a dispatch, and
stops once the queue is quiet.
@safedep

safedep Bot commented Aug 10, 2026

Copy link
Copy Markdown

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

View complete scan results →

This report is generated by SafeDep Github App

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 14c6d0e4ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

articleId,
lang,
textHash: md5(text),
force: Boolean(force),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Serialize forced and ordinary generations

Including force in the in-flight key lets forced and ordinary requests for the same article/language become independent leaders. If both run concurrently, each generates and upserts the same asset, so whichever finishes last wins and can overwrite the result the user explicitly forced while also charging for both calls. The same split is applied to insights and translations; forced requests should bypass cached results without permitting a second writer for the same resource.

Useful? React with 👍 / 👎.

Comment thread apps/admin/src/features/ai/utils/ai.ts Outdated
Comment on lines +13 to +16
const lang = segment.trim().toLowerCase()
if (!lang || seen.has(lang)) continue
seen.add(lang)
result.push(lang)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Canonicalize languages before deduplicating them

When the new multi-language field contains aliases such as zh-CN, zh, this code treats them as distinct. Summary and translation task handlers consume these raw strings rather than canonicalizing them, so they issue two provider calls and can persist two rows for the same logical language, producing duplicate assets and charges. Normalize each segment with the shared language-code parser before applying seen and sending the list.

Useful? React with 👍 / 👎.

Comment on lines +102 to +105
const base = detail.assets.insights.find((row) => !row.isTranslation)
if (!base || base.lang === lang)
return createInsightsTask({ force, refId })
return createInsightsTranslationTask({ refId, targetLang: lang })

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Generate the requested insights translation after bootstrapping

When no base insights row exists and the user clicks a target-language cell, this branch creates only the source-language insights task and discards the requested lang. If auto-translation is disabled or that language is not in insightsTargetLanguages, the task completes without ever creating the selected translation and the cell returns to a gap. The action needs to chain the target translation after base generation rather than treating the base task as fulfillment of any language.

Useful? React with 👍 / 👎.

const base = detail.assets.insights.find((row) => !row.isTranslation)
if (!base || base.lang === lang)
return createInsightsTask({ force, refId })
return createInsightsTranslationTask({ refId, targetLang: lang })

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Forward force when regenerating translated insights

For an existing translated-insights asset, the overview's Regenerate action reaches this branch with force=true, but the value is dropped because the translation endpoint and payload receive no force flag. In particular, while the in-flight result cache is still alive, the request simply returns the cached row instead of invoking the model, so the advertised regeneration does nothing. Add force support to the insights-translation request and its in-flight path.

Useful? React with 👍 / 👎.

Comment on lines +175 to +176
const { data } = await this.taskQueueService.getTasks({
scope: 'ai',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include batch child tasks in overview activity

This call leaves includeSubTasks at its default false. Translation batch/all parent tasks finish after enqueueing child ai:translation tasks with a groupId, so while those children are actually generating, getTasks filters them out and the overview shows neither their pending state nor their failures. Pass includeSubTasks: true so the existing toActiveGenerations translation handling can associate those child tasks with the article.

Useful? React with 👍 / 👎.

Innei added 7 commits August 10, 2026 18:33
…codes

Folding `force` into the summary/insights/translation in-flight key hash
(8433347) fixed force silently joining a plain leader, but split the
mutex too: a force and a plain request for the same content now each ran
the model and upserted the same row, with whichever finished last
clobbering the other and doubling the bill.

Revert the key change and arbitrate on lock ownership instead. The lock
value now encodes the holder's mode (`force:`/`plain:`). A force request
that loses the race to a plain leader polls until the lock frees up (or
lockTtlSec elapses, degrading to a follower) instead of racing it with a
second writer; a lock already held by another force is joined immediately,
since two force runs converging on one leader is the desired outcome.
Plain-request behavior on a lost race is unchanged.

Also fold resolved target languages through parseLanguageCode before
dedup in the summary and translation task handlers, matching what TTS
already does — zh-CN and zh no longer produce two separate generations
for the same input.
…input

parseLangInput treated zh-CN and zh (or en_US and EN) as distinct
languages, so the multilang chips could lie about how many generations
would actually run. Normalize underscores to hyphens and drop a 2-letter
primary tag's region suffix before dedup — matching the backend's
authoritative parseLanguageCode without duplicating its alias table here.
waitForForceLock only inspected the lock holder once, when the initial
NX attempt failed. If a plain leader released the lock mid-wait and a
different force request won it via the ordinary leader path, this
instance kept polling blind until lockTtlSec instead of noticing the
lock was now force-held and joining immediately.

Re-check the holder after each failed retry so a mid-wait handoff to
another force is picked up right away, saving a redundant model call
and up to lockTtlSec of unnecessary waiting. Behavior when the holder
stays plain (or the lock is released outright) is unchanged.
Acquiring the lock only cleared resultKey before starting a fresh run.
streamKey (done frames live 600s) and errorKey (30s) from a previous run
on the same key survived, so a follower or converging force joining this
leader reads the stream from '0-0' and can hit the old `done`/`error`
entry first — resolving to the previous run's result or throwing a stale
error instead of waiting for the new one.

Delete all three together, still gated behind the lock so a concurrent
plain follower never observes them all empty at once.
Two related language-normalization bugs, fixed together since they touch
the same call sites.

parseLanguageCode's fallback for anything it doesn't recognize is
DEFAULT_SUMMARY_LANG ('zh'). Folding summary/translation target languages
through it meant a free-typed token like "english" (the admin generate
modal takes arbitrary input) collapsed onto 'zh' and silently overwrote
an actual zh row instead of just being its own odd entry. Add
normalizeTargetLang (ai-language.util.ts): known codes/aliases still fold
via normalizeLanguageCode, but anything unrecognized passes through as
trim+lowercase instead of defaulting. Use it in both the summary task
handler and executeTranslationTask.

Since executeTranslationTask now generates against normalized language
codes, buildTranslationRetryTask's PartialFailed diff broke: it compared
raw payload.targetLanguages (e.g. 'zh-CN') against already-normalized
result.translations[].lang ('zh'), so a successful zh-CN run never
matched and got retried (with force inherited, at extra cost). Normalize
payload.targetLanguages the same way before diffing.
normalizeTargetLang trimmed a blank/whitespace-only token and then, since
normalizeLanguageCode returns undefined for it, fell back to the trimmed
(still empty) string — so an empty target language silently became a
language named ''. The public task DTOs don't reject it either
(CreateSummaryTaskSchema / CreateTranslationTaskSchema only checked
z.string(), no non-empty constraint), so any direct API caller (not just
the admin modal, which already filters client-side) could push
targetLanguages: [''] through and generate against it.

Two-sided fix: normalizeTargetLang now returns undefined for a blank
token instead of '', and both call sites (summary handler, executeTranslationTask)
filter it out — dropping it rather than defaulting it to DEFAULT_SUMMARY_LANG,
which would reintroduce the same silent-overwrite problem the unrecognized-token
fix just closed. The DTOs add `.trim().min(1)` per element so a blank
entry 400s at the boundary instead of reaching the task handler at all.
…n dedup key

computeAITaskDedupKey canonicalized languages for Tts (parseLanguageCode)
but not for Summary/Translation, which just sorted+joined the raw
targetLanguages. Now that generation itself normalizes (zh-CN and zh
produce the same result), two requests differing only in region suffix
enqueue as two distinct tasks at the queue layer — the loser runs for
nothing.

Add canonicalTargetLangs, using the same normalizeTargetLang the
handlers generate against (not parseLanguageCode — that would be a third
normalization scheme for the same data), and use it for the Summary and
Translation branches. Tts keeps its existing parseLanguageCode-based
canonicalization unchanged.
@Innei

Innei commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

已处理 Codex 的两条本分支意见,另修掉修复过程中自查出的四条。

已修

P1 强制与普通生成并行成两个 writer1d53ffcb8c380adf19

此前为了让 force 不再静默跟随普通请求的 leader,把 force 并进了 in-flight key,结果把互斥也一并拆了——摘要 / 精读 / 翻译除 in-flight 锁外并无任何 per-(refId, lang) 互斥(只有 TTS 有 withTtsLangLock)。现改为:

  • 撤回 key 里的 force,恢复单一 in-flight 键,单 writer 得以保证。
  • 锁值编码持锁者模式(force: / plain:)。force 抢锁失败时按持锁者身份决定去留:持锁者也是 force 则合流(省一次模型调用),是 plain 则等其释放后自己做 leader,等待上限为 lockTtlSec,超时降级为 follower 而非抛错。每轮轮询重查持锁者身份,中途被另一个 force 接手即提前合流。
  • 「force 不得复用普通请求产出的缓存结果」这条不变量由 bypassResultCache 保持不变。

P2 语言码不归一1d53ffcb8f472f4642357e3128360b973471

  • 后端摘要与翻译 handler 归一并去重,与 TTS 既有写法一致。新增 normalizeTargetLang:认得的折(zh-CNzh),认不得的原样放行(english 保持 english)——不用 parseLanguageCode,因为它会把未知词元撞成 DEFAULT_SUMMARY_LANG,那意味着手打 english 会用英文内容覆盖掉合法的 zh 行。
  • 空串 / 纯空白语言码在 DTO 层拒掉,归一层返回 undefined 并由 handler 滤除。
  • Summary / Translation 的任务去重键也过同一归一函数,否则 zh-CNzh 两个请求生成结果相同却不去重,白跑两遍。
  • 前端 parseLangInput 折掉区域后缀,chips 不再撒谎;不复制后端的别名表,后端才是权威。

需要说明的取舍:zh-TW 在别名表里折向 zh,而 getLanguageName 对表外码回落原串,故此前 zh-TW 确实作为独立 prompt 目标生效过。归一后繁体这一路会并入 zh。这是有意为之——系统其余部分(TTS、LANGUAGE_CODE_TO_NAME)本就只认二字母码;若日后要繁体,应作为一个正式的区域变体特性来做,而非靠别名表漏网。

修复过程中自查出并一并修掉的

  • force leader 复用残留 streamKey / errorKeyf4720bd6c)。bypass 分支原先只删 resultKey,而上一轮的 done 帧在 streamKey 存活 600s。合流的 follower 从 '0-0' 起读会在旧 done 处提前返回上一轮的 resultId。现持锁后三键齐删。
  • 归一后翻译重试的差集失配74c9303fc)。buildTranslationRetryTasktargetLangs 取自未归一 payload,而 successLangs 已归一,zh-CN 会被误判为失败并重译。现两端同用一个归一函数。

未处理

Codex 另三条落在同分支的 AI 概览代码上(useOverviewActions.ts 两条、ai-overview.service.tsincludeSubTasks),非本次改动引入,留给该功能的作者。

另有两条已知、本次有意未修:waitForForceLock 的等待不感知 abort(任务取消后孤儿 waiter 最长 60s 内仍可能跑一次模型调用);expire / del 锁操作无持有者比对(既有问题,锁值现已带 instanceId,改 CAS 成本很低)。

验证

apps/coretest/src/modules/ai 759 通过 / 6 skip;apps/adminsrc/features/ai 41 通过;两 app tsc --noEmit 皆无输出。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60b9734716

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +94 to +96
await redis.del(resultKey)
await redis.del(streamKey)
await redis.del(errorKey)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the completed stream while forced work waits

When a forced request waits behind an ordinary leader, it can acquire the shared lock immediately after that leader writes done, then delete both its result and stream here while existing ordinary followers are still consuming them. A slow follower can consequently miss the original completion or continue reading tokens from the newly created forced stream, producing a mixed response. The new common-lock arbitration should retain each completed generation's stream until its readers have finished rather than erasing it when the next leader starts.

Useful? React with 👍 / 👎.

Comment on lines +236 to +238
summary: aiConfig.summaryTargetLanguages ?? [],
insights: aiConfig.insightsTargetLanguages ?? [],
translation: aiConfig.translationTargetLanguages ?? [],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Canonicalize configured languages before computing coverage

Summary and translation generation now canonicalize configured targets, so a setting such as zh-CN or jp is persisted as zh or ja, but this method returns the raw settings to the coverage calculation. The overview therefore compares zh-CN against an existing zh, permanently reports a gap, and lets the operator repeatedly enqueue work that cannot satisfy the displayed expectation. Normalize and deduplicate the summary and translation configuration with the same helper used by their task handlers.

Useful? React with 👍 / 👎.

Comment on lines +183 to +185
page: 1,
size: ACTIVE_TASK_SCAN_SIZE,
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Search past the first page for article tasks

In installations with more than 100 matching AI tasks, getTasks returns only the newest 100 before toActiveGenerations filters by refId, so an older pending/running task for the opened article disappears from the overview. The matrix then shows an actionable gap and can enqueue duplicate work even though that article's generation is still active. The lookup needs to paginate until the article is found or query by article rather than imposing this global first-page cap.

Useful? React with 👍 / 👎.

Comment on lines +208 to +213
onRetry={(task) =>
dispatchGeneration(
task.capability,
task.langs[0] ?? detail.coverage.sourceLang ?? 'zh',
true,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve every target when retrying a failed task

For a failed or partially failed multi-language task, the retry control dispatches only task.langs[0], leaving all remaining requested languages unretried. It is worse for tasks whose payload omitted languages to use configuration: langs is empty, so retry substitutes the article source language (or zh) instead of rerunning the configured targets. Retry should recreate the task with its complete original language payload and preserve the unspecified/config-driven case.

Useful? React with 👍 / 👎.

Innei added 2 commits August 10, 2026 21:44
A forced regeneration used to delete the shared stream the moment it took
the lock, so a follower still draining the finished run lost its tail and
spliced the new run's tokens into the same response. Streams are now keyed
by run (`:stream:<runId>`, resolved from the lock value), so a completed
generation stays readable until its TTL retires it and force only clears
the result/error cache. Run-scoped keys also retire the stale-frame hazard
the delete existed for.

Also settles the follower result promise a lock-race test left dangling:
its idle timeout rejected ~1s later and failed the whole shard.
…verview board

- coverage compares canonicalized configured targets, so a `zh-CN` setting
  no longer reports a gap the stored `zh` row can never close
- the article's active tasks include batch children and are looked up by
  refId, so a busy queue can no longer push a live task off the first page
- insights translation accepts force end to end (payload, DTO, dedup key,
  in-flight bypass)
- an insights task carries the language requested from a cell with no base
  row and chains its translation once the base exists
- retry re-dispatches every language the task ran on, and keeps the
  "use configured targets" case as an empty list instead of guessing one
- the language alias table moves to @mx-space/ai so admin folds exactly as
  the server does

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfb99d1c3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +165 to +166
const deadline = Date.now() + lockTtlSec * 1000
while (Date.now() < deadline) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep forced requests waiting for the ordinary leader

When an ordinary generation runs longer than AI_STREAM_LOCK_TTL (60 seconds), its heartbeat keeps renewing the lock, but this deadline expires and returns false, causing the forced request to follow the still-running ordinary generation and return its result without regenerating. Fresh evidence in the current revision is this finite wait deadline despite the leader's indefinitely renewed heartbeat, so generations exceeding one minute still reproduce the previously reported force-bypass failure.

Useful? React with 👍 / 👎.

streamMaxLen: AI_STREAM_MAXLEN,
readBlockMs: AI_STREAM_READ_BLOCK_MS,
idleTimeoutMs: AI_STREAM_IDLE_TIMEOUT_MS,
bypassResultCache: force,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh derived translations after forcing insights

When insights are force-regenerated without changing the article text, this bypass produces new insights content but continues to store md5(text) as its source hash. Existing translated insights therefore still match that hash, so deleteTranslationsWithDifferentHash retains them and the generated-event handler skips retranslation; the base insights and its translations can consequently describe different model outputs. Force regeneration should invalidate or requeue derived translations, or version them from the generated insights content rather than only the article text.

Useful? React with 👍 / 👎.

Comment on lines +69 to +70

export function resolveCell(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Canonicalize manually added overview languages

When an operator adds an accepted alias such as jp, jpn, cn, or kr, this helper leaves the alias in extraColumns, while the generation handlers canonicalize it to ja, zh, or ko. After generation the persisted asset appears under the canonical column, but the manually added alias column remains an actionable gap and can repeatedly enqueue work that will never fill it. Use the shared target-language normalizer here before storing the column.

Useful? React with 👍 / 👎.

…hen-translate pipeline

- ai_summaries gains is_translation/source_summary_id/source_lang, lang backfill,
  dedup and UNIQUE(ref_id, lang) via migration 0032
- new ai-multilang MultilangAdapter + MultilangGenerationService: reuse or
  generate the source-language base, invalidate stale translations, then
  translate remaining targets concurrently
- insights drops chained queue subtasks for inline concurrent translation and
  resolves its source lang from meta.lang instead of the nonexistent article.lang
- summary drops the per-language loop, adds SummaryTranslation task type,
  POST /ai/summaries/task/translate and SUMMARY_GENERATED
- insights task DTO gains the 8-language cap and blank-token rejection;
  overview and admin dispatch follow the base/translation split

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85ce422ea5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// design.
const summarisable = input.type !== CollectionRefTypes.Page

const sourceLang = input.metaLang ?? input.translationSourceLangs[0] ?? null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back to the generation source language in coverage

When an article has no meta.lang and no existing translation row, this leaves sourceLang null even though the summary, insights, translation, and TTS handlers all resolve the same missing value to DEFAULT_SUMMARY_LANG (zh). The overview consequently offers zh as an article-translation gap and omits the source-language TTS gap, so actions and gap counts disagree with what generation will actually do. Use the same source-language resolver as the generation paths.

Useful? React with 👍 / 👎.

Comment on lines +203 to +205
let usage: GenerationUsage = emptyUsage()
if (runtime.generateTextStream) {
for await (const chunk of runtime.generateTextStream({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Capture usage from streamed derived translations

With the current Pi runtime, generateTextStream is present but yields only text chunks, so this branch never updates usage from emptyUsage(). Every streamed summary translation is therefore persisted as a zero-token, zero-cost generation and contributes no cost to its task, making the newly added overview totals incorrect; the insights adapter repeats the same pattern. Use a stream API that exposes the final usage event or otherwise collect usage before recording metrics.

Useful? React with 👍 / 👎.

Comment on lines +70 to +74
result.push({
...base,
capability: 'summary',
langs: asStringArray(payload.targetLanguages),
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark the base language active for multilang tasks

When a summary task explicitly targets, for example, en for a zh article, the backend first generates or refreshes the zh base, but this reports only en as active. While that base work is running, the overview leaves the zh summary cell actionable and can enqueue another task with a different dedup key; insights tasks have the same mismatch. Include the source/base language in the active coverage, or represent these tasks as covering the entire capability while their base phase runs.

Useful? React with 👍 / 👎.

@Innei
Innei merged commit d8343e5 into master Aug 10, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant