feat(generation): provide real-time progress streaming and chunk tracking during synthesis (#760) - #1024
Conversation
📝 WalkthroughWalkthroughThe PR adds end-to-end generation progress tracking. The backend reports model, chunk, completion, cancellation, and error states through SSE. The frontend stores this data and displays progress in the floating generator, history table, and story content. ChangesGeneration progress flow
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant GenerationService
participant GenerationProgressManager
participant SSEEndpoint
participant useGenerationProgress
participant GenerationUI
GenerationService->>GenerationProgressManager: report model and chunk progress
GenerationProgressManager-->>SSEEndpoint: publish progress event
SSEEndpoint-->>useGenerationProgress: send SSE payload
useGenerationProgress->>GenerationUI: update generation store
GenerationUI-->>GenerationUI: render progress panel and status
GenerationService->>GenerationProgressManager: mark complete or error
GenerationProgressManager-->>SSEEndpoint: publish terminal event
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 8
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/StoriesTab/StoryContent.tsx`:
- Around line 391-399: The generating badge in StoryContent’s inline render
currently reads active progress via getState(), so progress updates do not
trigger rerenders. Subscribe to activeGenerationId and generationProgress using
Zustand selectors, derive the active progress record from those selected values,
and use it for the progress text while preserving the existing pendingCount
fallback.
In `@app/src/stores/generationStore.ts`:
- Around line 44-56: The removePendingGeneration updater should select another
ID from the remaining pendingGenerationIds when the removed id matches
activeGenerationId, rather than always setting activeGenerationId to null.
Preserve null only when no pending generations remain, and keep the existing
generationProgress cleanup unchanged.
In `@backend/routes/generations.py`:
- Around line 304-313: Update the terminal-event handling in the generation
progress stream to re-query or refresh DBGeneration after detecting completed or
failed status, since the existing gen instance may have stale metadata. Populate
the terminal payload with current duration, error, and source values before
yielding it, and ensure this refreshed terminal event is sent before returning
rather than bypassing the final database fallback.
In `@backend/services/generation.py`:
- Around line 102-116: The _on_chunk_progress callback labels chunk progress as
sentences; update its progress message to use chunk terminology, including the
formatted count, while leaving the progress calculation and update behavior
unchanged.
In `@backend/tests/test_generation_progress.py`:
- Around line 37-45: The lifecycle test reuses gen_id for both completed and
failed states, allowing a terminal-state reversal. In the test covering
mark_complete and mark_error, use a separate generation ID for the failure path,
and assert the original gen_id still has status "completed" after the failure
scenario.
In `@backend/utils/chunked_tts.py`:
- Around line 130-134: Update the punctuation scan in the visible helper so
`re.finditer` matches `。!?` only when the position is outside bracket tags,
skipping punctuation between `[` and its matching `]`. Preserve the existing
earliest-valid-position behavior and `-1` result when no eligible punctuation
remains.
In `@backend/utils/generation_progress.py`:
- Around line 40-51: The listener delivery logic must not drop terminal
completed or failed progress events when a queue is full. Update the queue
handling in the progress notification method to make room for terminal events,
such as removing obsolete queued progress before enqueueing them, while
preserving existing behavior for non-terminal updates and ensuring subscribe()
can observe completion.
- Around line 219-224: Synchronize initialization in
get_generation_progress_manager so concurrent callers cannot both create
separate GenerationProgressManager instances. Add a shared lock around the None
check and manager construction while preserving the existing singleton return
behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 43adf2d2-e851-496c-841f-8f4d4daed861
📒 Files selected for processing (10)
app/src/components/Generation/FloatingGenerateBox.tsxapp/src/components/History/HistoryTable.tsxapp/src/components/StoriesTab/StoryContent.tsxapp/src/lib/hooks/useGenerationProgress.tsapp/src/stores/generationStore.tsbackend/routes/generations.pybackend/services/generation.pybackend/tests/test_generation_progress.pybackend/utils/chunked_tts.pybackend/utils/generation_progress.py
| {(() => { | ||
| const activeId = Array.from(useGenerationStore.getState().pendingGenerationIds)[0]; | ||
| const prog = activeId ? useGenerationStore.getState().generationProgress.get(activeId) : undefined; | ||
| if (prog?.progress !== undefined) { | ||
| const chunkInfo = prog.currentChunk && prog.totalChunks ? ` (${prog.currentChunk}/${prog.totalChunks})` : ''; | ||
| return `Generating... ${Math.round(prog.progress)}%${chunkInfo}`; | ||
| } | ||
| return t('storyContent.generatingCount', { count: pendingCount }); | ||
| })()} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Subscribe to the active progress record.
Lines 392-393 use useGenerationStore.getState(). Progress-map updates do not change pendingCount, so this component does not rerender for SSE progress events. The badge can remain at its initial fallback text until an unrelated render occurs.
Subscribe to activeGenerationId and generationProgress with Zustand selectors. Use that selected record in this render. This also aligns the badge with FloatingGenerateBox.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/src/components/StoriesTab/StoryContent.tsx` around lines 391 - 399, The
generating badge in StoryContent’s inline render currently reads active progress
via getState(), so progress updates do not trigger rerenders. Subscribe to
activeGenerationId and generationProgress using Zustand selectors, derive the
active progress record from those selected values, and use it for the progress
text while preserving the existing pendingCount fallback.
| removePendingGeneration: (id) => | ||
| set((state) => { | ||
| const next = new Set(state.pendingGenerationIds); | ||
| next.delete(id); | ||
| return { pendingGenerationIds: next, isGenerating: next.size > 0 }; | ||
| const nextProgress = new Map(state.generationProgress); | ||
| nextProgress.delete(id); | ||
| return { | ||
| pendingGenerationIds: next, | ||
| isGenerating: next.size > 0, | ||
| generationProgress: nextProgress, | ||
| activeGenerationId: state.activeGenerationId === id ? null : state.activeGenerationId, | ||
| }; | ||
| }), |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Select a remaining active generation.
When the active generation ends while another generation is pending, Line 54 sets activeGenerationId to null. The floating progress panel then shows generic progress at 0% instead of the remaining generation state.
Select a remaining pending ID when the removed ID was active.
Proposed fix
removePendingGeneration: (id) =>
set((state) => {
const next = new Set(state.pendingGenerationIds);
next.delete(id);
const nextProgress = new Map(state.generationProgress);
nextProgress.delete(id);
+ const nextActiveGenerationId =
+ state.activeGenerationId === id ? Array.from(next).pop() ?? null : state.activeGenerationId;
return {
pendingGenerationIds: next,
isGenerating: next.size > 0,
generationProgress: nextProgress,
- activeGenerationId: state.activeGenerationId === id ? null : state.activeGenerationId,
+ activeGenerationId: nextActiveGenerationId,
};
}),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| removePendingGeneration: (id) => | |
| set((state) => { | |
| const next = new Set(state.pendingGenerationIds); | |
| next.delete(id); | |
| return { pendingGenerationIds: next, isGenerating: next.size > 0 }; | |
| const nextProgress = new Map(state.generationProgress); | |
| nextProgress.delete(id); | |
| return { | |
| pendingGenerationIds: next, | |
| isGenerating: next.size > 0, | |
| generationProgress: nextProgress, | |
| activeGenerationId: state.activeGenerationId === id ? null : state.activeGenerationId, | |
| }; | |
| }), | |
| removePendingGeneration: (id) => | |
| set((state) => { | |
| const next = new Set(state.pendingGenerationIds); | |
| next.delete(id); | |
| const nextProgress = new Map(state.generationProgress); | |
| nextProgress.delete(id); | |
| const nextActiveGenerationId = | |
| state.activeGenerationId === id ? Array.from(next).pop() ?? null : state.activeGenerationId; | |
| return { | |
| pendingGenerationIds: next, | |
| isGenerating: next.size > 0, | |
| generationProgress: nextProgress, | |
| activeGenerationId: nextActiveGenerationId, | |
| }; | |
| }), |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/src/stores/generationStore.ts` around lines 44 - 56, The
removePendingGeneration updater should select another ID from the remaining
pendingGenerationIds when the removed id matches activeGenerationId, rather than
always setting activeGenerationId to null. Preserve null only when no pending
generations remain, and keep the existing generationProgress cleanup unchanged.
| # Stream real-time progress events from GenerationProgressManager | ||
| async for event_str in progress_mgr.subscribe(generation_id): | ||
| if event_str.startswith("data: "): | ||
| try: | ||
| prog_data = json.loads(event_str[6:]) | ||
| prog_data["source"] = gen.source | ||
| prog_data["duration"] = gen.duration | ||
| yield f"data: {json.dumps(prog_data)}\n\n" | ||
| if prog_data.get("status") in ("completed", "failed"): | ||
| return |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Refresh terminal metadata before sending the terminal event.
gen was loaded before generation completed. run_generation() writes duration through a separate database session, so gen.duration remains stale here. The return on Line 313 also bypasses the final database fallback.
Refresh or re-query DBGeneration when prog_data is terminal, then add the current duration, error, and source values before yielding the event.
🧰 Tools
🪛 ast-grep (0.45.0)
[info] 310-310: use jsonify instead of json.dumps for JSON output
Context: json.dumps(prog_data)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/generations.py` around lines 304 - 313, Update the
terminal-event handling in the generation progress stream to re-query or refresh
DBGeneration after detecting completed or failed status, since the existing gen
instance may have stale metadata. Populate the terminal payload with current
duration, error, and source values before yielding it, and ensure this refreshed
terminal event is sent before returning rather than bypassing the final database
fallback.
| def _on_chunk_progress(current_chunk: int, total_chunks: int, chunk_text: str): | ||
| if total_chunks > 0: | ||
| # 10% to 90% allocated to chunk synthesis | ||
| pct = 10.0 + ((current_chunk - 1) / total_chunks) * 80.0 | ||
| else: | ||
| pct = 50.0 | ||
| msg = f"Sentence {current_chunk} of {total_chunks}" if total_chunks > 1 else None | ||
| progress_mgr.update_progress( | ||
| generation_id, | ||
| progress=pct, | ||
| current_chunk=current_chunk, | ||
| total_chunks=total_chunks, | ||
| status="generating", | ||
| message=msg, | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Label these values as chunks, not sentences.
split_text_into_chunks() uses clause, whitespace, and hard-cut fallbacks for oversized sentences. The current message can therefore report a partial sentence as Sentence N of M.
Proposed fix
- msg = f"Sentence {current_chunk} of {total_chunks}" if total_chunks > 1 else None
+ msg = f"Chunk {current_chunk} of {total_chunks}" if total_chunks > 1 else None📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def _on_chunk_progress(current_chunk: int, total_chunks: int, chunk_text: str): | |
| if total_chunks > 0: | |
| # 10% to 90% allocated to chunk synthesis | |
| pct = 10.0 + ((current_chunk - 1) / total_chunks) * 80.0 | |
| else: | |
| pct = 50.0 | |
| msg = f"Sentence {current_chunk} of {total_chunks}" if total_chunks > 1 else None | |
| progress_mgr.update_progress( | |
| generation_id, | |
| progress=pct, | |
| current_chunk=current_chunk, | |
| total_chunks=total_chunks, | |
| status="generating", | |
| message=msg, | |
| ) | |
| def _on_chunk_progress(current_chunk: int, total_chunks: int, chunk_text: str): | |
| if total_chunks > 0: | |
| # 10% to 90% allocated to chunk synthesis | |
| pct = 10.0 + ((current_chunk - 1) / total_chunks) * 80.0 | |
| else: | |
| pct = 50.0 | |
| msg = f"Chunk {current_chunk} of {total_chunks}" if total_chunks > 1 else None | |
| progress_mgr.update_progress( | |
| generation_id, | |
| progress=pct, | |
| current_chunk=current_chunk, | |
| total_chunks=total_chunks, | |
| status="generating", | |
| message=msg, | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/services/generation.py` around lines 102 - 116, The
_on_chunk_progress callback labels chunk progress as sentences; update its
progress message to use chunk terminology, including the formatted count, while
leaving the progress calculation and update behavior unchanged.
| mgr.mark_complete(gen_id) | ||
| completed_data = mgr.get_progress(gen_id) | ||
| assert completed_data["status"] == "completed" | ||
| assert completed_data["progress"] == 100.0 | ||
|
|
||
| mgr.mark_error(gen_id, "Test error message") | ||
| error_data = mgr.get_progress(gen_id) | ||
| assert error_data["status"] == "failed" | ||
| assert error_data["error"] == "Test error message" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Keep completed and failed lifecycle tests separate.
Line 42 marks gen_id as failed after line 37 marks it completed. This test permits a terminal state reversal. Use a separate generation ID for the failure path. Assert that gen_id remains completed.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_generation_progress.py` around lines 37 - 45, The
lifecycle test reuses gen_id for both completed and failed states, allowing a
terminal-state reversal. In the test covering mark_complete and mark_error, use
a separate generation ID for the failure path, and assert the original gen_id
still has status "completed" after the failure scenario.
| for m in re.finditer(r"[\u3002\uff01\uff1f]", text): | ||
| pos = m.start() | ||
| if best == -1 or pos < best: | ||
| best = pos | ||
| return best |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Skip CJK punctuation inside bracket tags.
This loop accepts 。!? inside [ ... ]. For example, [laugh。] splits into separate chunks and breaks the documented atomic-tag behavior.
Proposed fix
for m in re.finditer(r"[\u3002\uff01\uff1f]", text):
pos = m.start()
+ if _inside_bracket_tag(text, pos):
+ continue
if best == -1 or pos < best:
best = pos📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| for m in re.finditer(r"[\u3002\uff01\uff1f]", text): | |
| pos = m.start() | |
| if best == -1 or pos < best: | |
| best = pos | |
| return best | |
| for m in re.finditer(r"[\u3002\uff01\uff1f]", text): | |
| pos = m.start() | |
| if _inside_bracket_tag(text, pos): | |
| continue | |
| if best == -1 or pos < best: | |
| best = pos | |
| return best |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/utils/chunked_tts.py` around lines 130 - 134, Update the punctuation
scan in the visible helper so `re.finditer` matches `。!?` only when the position
is outside bracket tags, skipping punctuation between `[` and its matching `]`.
Preserve the existing earliest-valid-position behavior and `-1` result when no
eligible punctuation remains.
| for queue in listeners: | ||
| try: | ||
| try: | ||
| asyncio.get_running_loop() | ||
| queue.put_nowait(progress_data.copy()) | ||
| except RuntimeError: | ||
| if self._main_loop and self._main_loop.is_running(): | ||
| self._main_loop.call_soon_threadsafe( | ||
| lambda q=queue, d=progress_data.copy(): q.put_nowait(d) if not q.full() else None | ||
| ) | ||
| except asyncio.QueueFull: | ||
| logger.warning("Queue full for generation %s, dropping progress update", generation_id) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Do not discard terminal progress events.
A full listener queue drops completed and failed events. subscribe() then sends heartbeats indefinitely, and the route cannot reach its final database fallback.
Keep terminal events deliverable. For example, remove obsolete queued progress before enqueueing the terminal event.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/utils/generation_progress.py` around lines 40 - 51, The listener
delivery logic must not drop terminal completed or failed progress events when a
queue is full. Update the queue handling in the progress notification method to
make room for terminal events, such as removing obsolete queued progress before
enqueueing them, while preserving existing behavior for non-terminal updates and
ensuring subscribe() can observe completion.
| def get_generation_progress_manager() -> GenerationProgressManager: | ||
| """Get or create global generation progress manager.""" | ||
| global _generation_progress_manager | ||
| if _generation_progress_manager is None: | ||
| _generation_progress_manager = GenerationProgressManager() | ||
| return _generation_progress_manager |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Synchronize global manager initialization.
Two worker threads can both observe _generation_progress_manager is None and create different managers. If the SSE route receives one instance and generation updates use the other, the client receives no progress events.
Proposed fix
_generation_progress_manager: Optional[GenerationProgressManager] = None
+_generation_progress_manager_lock = threading.Lock()
def get_generation_progress_manager() -> GenerationProgressManager:
"""Get or create global generation progress manager."""
global _generation_progress_manager
- if _generation_progress_manager is None:
- _generation_progress_manager = GenerationProgressManager()
+ with _generation_progress_manager_lock:
+ if _generation_progress_manager is None:
+ _generation_progress_manager = GenerationProgressManager()
return _generation_progress_manager📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def get_generation_progress_manager() -> GenerationProgressManager: | |
| """Get or create global generation progress manager.""" | |
| global _generation_progress_manager | |
| if _generation_progress_manager is None: | |
| _generation_progress_manager = GenerationProgressManager() | |
| return _generation_progress_manager | |
| def get_generation_progress_manager() -> GenerationProgressManager: | |
| """Get or create global generation progress manager.""" | |
| global _generation_progress_manager | |
| with _generation_progress_manager_lock: | |
| if _generation_progress_manager is None: | |
| _generation_progress_manager = GenerationProgressManager() | |
| return _generation_progress_manager |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/utils/generation_progress.py` around lines 219 - 224, Synchronize
initialization in get_generation_progress_manager so concurrent callers cannot
both create separate GenerationProgressManager instances. Add a shared lock
around the None check and manager construction while preserving the existing
singleton return behavior.
|
Closes #760 |
Summary
This PR implements real-time progress information during voice audio generation across the backend inference pipeline and frontend UI components.
Previously, during audio generation, the UI displayed a static "Generating..." spinner without progress percentage or sentence chunk visibility. This change introduces end-to-end progress streaming via Server-Sent Events (SSE) and displays animated progress indicators across all generation components.
Key Changes
Backend (
backend/)GenerationProgressManager(backend/utils/generation_progress.py): Added a thread-safe progress manager tracking completion percentage (0-100%), sentence chunk indices (current_chunk/total_chunks), execution state (loading_model,generating,completed,failed), and structured log outputs.backend/utils/chunked_tts.py): Added_find_first_sentence_end()and updatedsplit_text_into_chunks()to chunk multi-sentence prompts per sentence boundary. Added aprogress_callbackparameter togenerate_chunked().backend/services/generation.py): HookedGenerationProgressManagerintorun_generation()to emit updates across model loading (0–10%), chunk synthesis (10–90%), post-processing (90–95%), and completion (100%).backend/routes/generations.py): EnhancedGET /generate/{generation_id}/statusSSE endpoint to stream real-time progress payloads while preserving backward compatibility for existing consumers.Frontend (
app/src/)app/src/stores/generationStore.ts): AddedgenerationProgressmap storing{ progress, currentChunk, totalChunks, status, message }per generation ID.app/src/lib/hooks/useGenerationProgress.ts): Updated SSE event listener to consume real-time progress payloads and sync with the Zustand store.FloatingGenerateBox,HistoryTable,StoryContent):FloatingGenerateBox.tsx: Added an animated progress bar and header displayingGenerating... 45% (Sentence 3/7)orLoading model... 5%.HistoryTable.tsx: Displaying live percentage and sentence chunk badges on pending history rows.StoryContent.tsx: Displaying live progress status pills in the story header.Verification & Testing
Automated Tests Passed
backend/tests/test_generation_progress.py:test_generation_progress_manager_basic: PASSEDtest_generation_progress_sse_subscription: PASSEDtest_generate_chunked_progress_callback: PASSEDtest_split_text_into_chunks_sentences: PASSEDtest_progress.py,test_generation_download.py): PASSEDnpm run typecheckinapp/): 0 ERRORSManual Testing Performed
0% -> 100%progress bar animation.Sentence 1/N -> Sentence N/Nprogress steps seamlessly.const x = { status: 200 },def foo() -> float:), version numbers (v1.2.3), markdown fences, and symbols ($USD,@mention,#hashtag,100%) parse sentence boundaries cleanly without breaking code blocks.Loading model... 0%before transitioning immediately toSentence 1/N.Summary by CodeRabbit
New Features
Bug Fixes