fix: harden artifact fetching and extraction in the build pipeline - #58
Merged
Merged
Conversation
Three related weaknesses in how sub-app artifacts reach apps/, all in the path that consumes other repositories' release tarballs. The GitHub API helper built a curl command as a single shell string with GITHUB_TOKEN inlined, so the token became process argv (readable locally) and was echoed back in the error execSync throws on a failed request. The same string interpolated apps.json values with no escaping. Both API calls and asset downloads now use fetch with the token as a request header, and registry repo/version values are validated before use. Archives were extracted with no validation, so a compromised doc repo could write outside the staging directory or smuggle in a symlink that the later HTML crawl would follow. Extraction now rejects absolute, drive-letter and ".."-escaping members before writing anything, and refuses declared link members — read from the archive listing rather than from the extracted result, since Windows silently drops symlink members it lacks the privilege to create. fetch-apps.js also shelled out to cp -r and to tar with absolute paths, neither of which works on Windows, while build-vite.js already carried a documented workaround for exactly that. Both now share scripts/artifacts.js. Closes #43 Closes #44 Closes #56 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqFK6yffibtCBTF8xZ4hXW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three related weaknesses in the path that turns another repository's release tarball into
apps/{slug}/. Grouped because they live in the same two files and the fix for the third (a shared module) is what makes the first two apply to both onboarding paths at once.Changes
scripts/artifacts.js— new, shared by both fetch pathsextractTarball(),stageArtifact()andcopyDir().scripts/fetch-apps.js(GitHub releases) andscripts/build-vite.js(prebuilt/localPath) both use it, so they can no longer drift on the safety checks the way they had already drifted on portability.Token no longer crosses a shell — #43
ghApi()built acurlinvocation as one shell string withGITHUB_TOKENinlined:That put the token in the argv of a
sh -cprocess — readable by any local process — andexecSyncincludes the failed command verbatim in theErrorit throws, so a transient 404 printed it into build logs. It also interpolatedapps.jsonvalues (repo,version) with no escaping.Both the API calls and the asset download now use
fetchwith the token as a request header.downloadAssetstreams the response to disk, socurlis gone entirely and the token never appears in any process's argv.repois validated againstowner/nameandversionagainst tag-safe characters before either reaches a URL or thegh apifallback.Incidental:
fetchfollowing the API asset redirect drops theAuthorizationheader across origins, which is the behaviour the old comment was working around by hand.Archives are validated before extraction — #44
Extraction was
tar -xzfstraight into a working directory. A tarball with../members, absolute paths, or symlinks writes outside the extraction root — and since the artifact comes from a different repository's release, that is a supply-chain path to file-write on whoever builds the knowledge base.extractTarball()now refuses, before writing anything:..path segmentThe link check reads the archive listing, not the extracted result. That matters: Windows silently drops symlink members when the process lacks the create-symlink privilege, so a post-extraction check alone passes on a developer machine and only fails on Linux CI. A post-extraction
lstatwalk runs as well, as defence in depth.copyDirskips symlinks too.The build runs on Windows again — #56
fetch-apps.jsshelled out tocp -r(does not exist on Windows outside a POSIX shell) and totarwith absolute paths — the exact drive-letter parsebuild-vite.jsalready carried a six-line comment about working around. So the primary build path was the non-portable one and only the test path was portable. Both now go throughscripts/artifacts.js, which keeps the documented workaround in one place.Also removed: an unused
writeFileSyncimport and a dynamicawait import('fs')inside the fetch loop that only existed becausecopyFileSyncwas missing from the static import.Verification
New
tests/artifact-safety.spec.jscovers the guards directly: a well-formed bundle extracts, and traversal / absolute / symlink / empty archives are each refused with a message naming the origin. The fixtures are written byte-by-byte as ustar rather than produced withtar, because a traversal member is precisely whattarrefuses to create, and because it keeps the test identical on GNU tar and the bsdtar in Windows System32.Not covered by CI: the GitHub fetch path itself never runs in CI — the suites are hermetic by design and build from
tests/fixtures/. The shared extraction and copy helpers that path now uses are directly tested; thefetchcalls around them are not.Closes #43
Closes #44
Closes #56