Skip to content

perf(studio): serve bounded JPEG timeline thumbnails - #4560

Merged
jrusso1020 merged 1 commit into
mainfrom
fix/studio-image-thumbnails
Sep 27, 2026
Merged

jrusso1020 merged 1 commit into
mainfrom
fix/studio-image-thumbnails

Conversation

@jrusso1020

Copy link
Copy Markdown
Collaborator

What

Serve bounded JPEG images to Studio timeline strips instead of displaying full-resolution originals. A 2560×1920 source becomes a 180×135 timeline thumbnail; the composition and exported video continue to use the original.

Why

ImageThumbnail currently displays original photos in tiny tiles and accounts for each cached image as 240×135 regardless of its actual dimensions. This change requests small JPEG responses and accounts for their real decoded dimensions.

Three local runs per variant on the same frozen photo-heavy project, Chrome 147 at 1280×720:

Median Before After
Largest frame gap during 10 seconds of Studio playback 208 ms 58 ms
Frame gaps over 50 ms 3 1
RAF callbacks / 10 seconds 579 593
Loading benchmark readiness threshold 2342 ms 2418 ms
Summed Chrome process RSS 2205 MiB 2239 MiB

The 52 loaded timeline sources' decoded-pixel estimate falls from 956 MiB to 4.4 MiB. This is not a measured RAM reduction: RSS did not improve. Loading is 76 ms slower in these samples, and no startup speedup is claimed. Results support improved frame pacing in this workload, not a whole-film FPS guarantee. The readiness threshold is at least 147 mounted timeline image elements decoded, not time-to-first-frame. Composition photo downloads are unchanged.

How

  • Same-origin preview JPEGs use a fixed 240×135 image-thumbnail endpoint; no arbitrary remote URLs or requested output sizes.
  • Existing project/path containment, JPEG magic/format checks, 64 MiB input and 40 MP decode limits, two concurrent jobs, bounded queue, and 8 MiB/128-entry in-memory cache.
  • Cache identities include source file metadata; private HTTP responses revalidate. EXIF orientation is respected.
  • Other formats and external sources retain their behavior. Older servers or failed conversions fall back to originals with honest size accounting.
  • Uses the same Sharp version already present in the CLI workspace.

A client-side resizing prototype was rejected because it added approximately 1.3 seconds to loading. Details and limitations are recorded in docs/plans/2026-09-26-image-thumbnails.md.

Related work

Refs #4553 (unwanted preview reloads). This PR is independent, based on main after #4546; it does not include the watcher changes.

Test plan

  • 32 client/component/scheduler tests and 13 server/coordinator tests
  • Source invalidation, concurrent requests, cancellation/fallback, unknown projects, traversal, symlink escape, corrupt/mislabelled/oversized images, pixel limits, EXIF orientation, and no upscaling
  • Studio and studio-server TypeScript checks
  • Studio and studio-server builds
  • Changed-file lint, format, and Fallow audit (duplication warnings only)
  • Actual Studio playback clock and frame pacing; all mounted image URLs decode
  • Synthetic public media captures below; private project media was not uploaded

Before

Actual ImageThumbnail component, synthetic 2560×1920 JPEG at timeline tile size.

Original JPEG timeline strip

After

Same component and display size, using the bounded 180×135 JPEG response.

Bounded JPEG timeline strip

@mintlify

mintlify Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
hyperframes 🟢 Ready View Preview Sep 27, 2026, 1:40 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 65fc575: complete production diff, route/coordinator/path helpers, client decoder and scheduler integration. No blocking finding.

The route accepts only contained local JPEGs, checks size/magic/format, caps decode pixels and fixed output dimensions, honors orientation, closes its descriptor, and uses bounded active/queued work and LRU cache. Original composition sources/export paths are untouched. Client mapping is restricted to same-origin preview JPEGs; unsupported/failed endpoints fall back once to originals with actual decoded-size accounting, and cancellation does not retry.

Independent exact-head focused tests:45/45 passed (native Sharp route8, coordinator5, decoder9, scheduler17, component6), using existing compatible Sharp0.35.3 on Linux. Current CI is complete with no failures. No browser frame-pacing/RSS benchmark, export render, packaged or cross-platform native startup was independently rerun.

Non-blocking coverage note: cache eviction and the32-key admission boundary are source-audited but not directly pinned by these route tests. At full capacity, even a duplicate key may get429 before coordinator dedupe; client fallback preserves output but loses the bounded-thumbnail optimization for that request.

Verdict: APPROVE
Reasoning: The bounded thumbnail path preserves original assets and has working negative-path, fallback and cancellation coverage. No correctness blocker found; performance claims remain scoped to the author's workload evidence. No merge/release performed.

— Magi

@jrusso1020
jrusso1020 added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit 91b21ae Sep 27, 2026
66 checks passed
@jrusso1020
jrusso1020 deleted the fix/studio-image-thumbnails branch September 27, 2026 02:25

This branch was successfully deployed

1 active deployment
staging - docs — 65fc575e Deployed Sep 27, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants