Skip to content

fix: harden artifact fetching and extraction in the build pipeline - #58

Merged
oto-macenauer-absa merged 1 commit into
masterfrom
fix/build-pipeline-hardening
Aug 14, 2026
Merged

fix: harden artifact fetching and extraction in the build pipeline#58
oto-macenauer-absa merged 1 commit into
masterfrom
fix/build-pipeline-hardening

Conversation

@oto-macenauer-absa

Copy link
Copy Markdown
Collaborator

What

Three related weaknesses in the path that turns another repository's release tarball into apps/{slug}/. Grouped because they live in the same two files and the fix for the third (a shared module) is what makes the first two apply to both onboarding paths at once.

Changes

scripts/artifacts.js — new, shared by both fetch paths

extractTarball(), stageArtifact() and copyDir(). scripts/fetch-apps.js (GitHub releases) and scripts/build-vite.js (prebuilt/localPath) both use it, so they can no longer drift on the safety checks the way they had already drifted on portability.

Token no longer crosses a shell — #43

ghApi() built a curl invocation as one shell string with GITHUB_TOKEN inlined:

execSync(`curl -fsSL -H "Authorization: Bearer ${GITHUB_TOKEN}" … "${API_BASE}${path}"`)

That put the token in the argv of a sh -c process — readable by any local process — and execSync includes the failed command verbatim in the Error it throws, so a transient 404 printed it into build logs. It also interpolated apps.json values (repo, version) with no escaping.

Both the API calls and the asset download now use fetch with the token as a request header. downloadAsset streams the response to disk, so curl is gone entirely and the token never appears in any process's argv. repo is validated against owner/name and version against tag-safe characters before either reaches a URL or the gh api fallback.

Incidental: fetch following the API asset redirect drops the Authorization header across origins, which is the behaviour the old comment was working around by hand.

Archives are validated before extraction — #44

Extraction was tar -xzf straight into a working directory. A tarball with ../ members, absolute paths, or symlinks writes outside the extraction root — and since the artifact comes from a different repository's release, that is a supply-chain path to file-write on whoever builds the knowledge base.

extractTarball() now refuses, before writing anything:

  • absolute members and members carrying a drive letter
  • any member with a .. path segment
  • declared symlink/hardlink members

The link check reads the archive listing, not the extracted result. That matters: Windows silently drops symlink members when the process lacks the create-symlink privilege, so a post-extraction check alone passes on a developer machine and only fails on Linux CI. A post-extraction lstat walk runs as well, as defence in depth. copyDir skips symlinks too.

The build runs on Windows again — #56

fetch-apps.js shelled out to cp -r (does not exist on Windows outside a POSIX shell) and to tar with absolute paths — the exact drive-letter parse build-vite.js already carried a six-line comment about working around. So the primary build path was the non-portable one and only the test path was portable. Both now go through scripts/artifacts.js, which keeps the documented workaround in one place.

Also removed: an unused writeFileSync import and a dynamic await import('fs') inside the fetch loop that only existed because copyFileSync was missing from the static import.

Verification

npm run build:headless                                → 8 apps integrated
npm test                                              → 49 passed  (was 43; +6 new)
npx playwright test --config=playwright.config.ci.js  → 15 passed

New tests/artifact-safety.spec.js covers the guards directly: a well-formed bundle extracts, and traversal / absolute / symlink / empty archives are each refused with a message naming the origin. The fixtures are written byte-by-byte as ustar rather than produced with tar, because a traversal member is precisely what tar refuses to create, and because it keeps the test identical on GNU tar and the bsdtar in Windows System32.

Not covered by CI: the GitHub fetch path itself never runs in CI — the suites are hermetic by design and build from tests/fixtures/. The shared extraction and copy helpers that path now uses are directly tested; the fetch calls around them are not.

Closes #43
Closes #44
Closes #56

Three related weaknesses in how sub-app artifacts reach apps/, all in the
path that consumes other repositories' release tarballs.

The GitHub API helper built a curl command as a single shell string with
GITHUB_TOKEN inlined, so the token became process argv (readable locally)
and was echoed back in the error execSync throws on a failed request. The
same string interpolated apps.json values with no escaping. Both API calls
and asset downloads now use fetch with the token as a request header, and
registry repo/version values are validated before use.

Archives were extracted with no validation, so a compromised doc repo
could write outside the staging directory or smuggle in a symlink that the
later HTML crawl would follow. Extraction now rejects absolute, drive-letter
and ".."-escaping members before writing anything, and refuses declared
link members — read from the archive listing rather than from the extracted
result, since Windows silently drops symlink members it lacks the privilege
to create.

fetch-apps.js also shelled out to cp -r and to tar with absolute paths,
neither of which works on Windows, while build-vite.js already carried a
documented workaround for exactly that. Both now share scripts/artifacts.js.

Closes #43
Closes #44
Closes #56

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QqFK6yffibtCBTF8xZ4hXW
@oto-macenauer-absa
oto-macenauer-absa merged commit 87a8b16 into master Aug 14, 2026
6 checks passed
@oto-macenauer-absa
oto-macenauer-absa deleted the fix/build-pipeline-hardening branch August 14, 2026 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant