Support building natively on Android (bionic) hosts - #238
Open
hpr wants to merge 46 commits into
Open
Conversation
With the Swift Android toolchain (e.g. under Termux), xtool can build and run on-device. Changes are additive platform guards: - Package.swift: enable the Subprocess, AsyncHTTPClient, OpenAPIAsyncHTTPClient, WebSocketKit, and XADI dependencies on Android, matching Linux. - XKit: use the AsyncHTTPClient-based HTTPClient on Android (the URLSession-based client relies on Darwin-only URLSessionWebSocketTask); bionic's FILE is an incomplete type, so stdoutSafe is an OpaquePointer there. - XToolSupport/XUtils: import the Android overlay where needed (SIG_IGN, flock, errno); bionic marks signal() warn_unused_result.
kabiroberai
reviewed
Aug 13, 2026
kabiroberai
left a comment
kabiroberai
left a comment
Member
There was a problem hiding this comment.
Nice! Wanna add a CI job as well? Should be able to cross compile on Linux with the Swift SDK.
Per review: cross-compile the package for Android (bionic) from a Linux host using the official Swift SDK artifact bundle for Android. Pins the 6.3.2 toolchain to match the bundle, installs the Android NDK to populate the SDK's ndk-sysroot (the bundle's setup-android-sdk.sh hardlinks it in), and builds the xtool product for aarch64-unknown-linux-android.
- Android/build-native-libs.sh: cross-build OpenSSL and the libimobiledevice stack (libplist, -glue, libusbmuxd, libtatsu, libimobiledevice; same set as the Linux Docker image) for aarch64-android, installing into the Swift SDK's ndk-sysroot so the zsign/SwiftyMobileDevice systemLibrary targets compile and link. libxadi is not needed: XADIProvider is os(Linux)-only. - CI: run the script after SDK install (adds autoconf/automake/ libtool). - SDKBuilder: validate a sample of known XIP-hardlink-victim files after copying (libc++ __config/vector/string, Swift shims HeapObject.h/KeyPath.h). When linkat(2) fails during extraction (e.g. SELinux hosts) these vanish silently and the SDK produces confusing 'file not found' compile errors long after install; fail at install time instead.
- unxip upstream's zlib/getopt shims (included when the manifest compiles on a Linux host) clash with the NDK's own modules during Android cross builds; mirror to the fork whose manifest models the shims with .when(platforms:) until saagarjha/unxip#41 lands. - build-native-libs.sh: cross-build xz (unxip links liblzma) and emit pkg-config files into the SDK so SwiftPM's systemLibrary targets resolve cflags/libs against the sysroot instead of the host.
clang resolves ld.lld from its own program dir before PATH, so the shim could never intercept. The toolset's linker options provably reach the final link (its -z max-page-size=16384 appears in the response file), so append -Wl,-m,aarch64elf there instead.
The link failure was not a missing emulation flag: host -L/usr/lib/x86_64-linux-gnu (from host pkg-config .pc files) let lld resolve -lm to Ubuntu's libm.so linker script, whose OUTPUT_FORMAT(elf64-x86-64) silently overrides -m in lld, dropping the link to x86-64. Point PKG_CONFIG_LIBDIR at the .pc files generated for the cross sysroot instead, and map SwiftPM's hardcoded -lstdc++ to the NDK's libc++ via a libstdc++.so script.
build-native-libs.sh receives the SDK's swift-android dir as $SDK and wrote .pc files to $SDK/swift-android/pkgconfig, but the build step exports PKG_CONFIG_PATH=$ANDROID_SWIFT_SDK/swift-android/pkgconfig. The exported dir didn't exist (job14: ls failed, killing the step under bash -e), so SwiftPM fell through to the host's .pc files and -L/usr/lib/x86_64-linux-gnu kept leaking into the link. Write to $SDK/pkgconfig instead; guard the diagnostic ls with || true.
libtatsu's pkg-config file is versioned (libtatsu-1.0.pc) but its libtool target is not: it installs libtatsu.a, so -ltatsu-1.0 can never resolve. Use -ltatsu, matching upstream's own libtatsu-1.0.pc.
NDK r27's prebuilt libc.a et al. ship debug sections compressed with ELFCOMPRESS_ZSTD, and the swift.org Ubuntu toolchain's lld is built without zstd support, failing the final xtool link. Strip debug info from every static archive in the SDK's ndk-sysroot using the NDK's own llvm-strip (verified on-device: stripped archives link cleanly).
Job17 still hit ELFCOMPRESS_ZSTD errors on the sysroot libc.a despite the strip sweep, so instrument it: count stripped archives and fail the step if readelf still reports compressed sections in libc.a.
setup-android-sdk.sh's default SWIFT_ANDROID_NDK_LINK=1 symlinks ndk-sysroot/usr/lib/<triple> into the NDK, and find's default -P never traverses symlinked dirs, so the strip sweep matched 0 archives (job18: 'stripped debug sections from 0 archives; compressed sections left: 2592'). Use find -L. Reproduced the exact layout on-device: -P finds 0, -L finds and strips through the symlink, lld links with no zstd errors.
The NDK ships some *.a files that are GNU ld scripts rather than object
archives (e.g. i686-linux-android/23/libc++.a); llvm-strip errors on
them ('not recognized as a valid object file') and set -e aborted the
sweep before libc.a was stripped. Strip per-file with failures ignored;
the readelf verification on the target-triple libc.a remains the gate.
The Android link is green; remove the temporary ld.lld probe, the response-file capture wrapper, and the failure-handler bisect. Keep the real fixes (pkg-config scoping, native lib build) and end the job with a file(1) check that the product is an aarch64 Android ELF, matching the other jobs' smoke-test convention (the binary can't run on x86_64).
Author
@kabiroberai Added CI build green at https://github.com/hpr/xtool/actions/runs/32047498390, but it depends on xtool-org/xtool-core#2 and saagarjha/unxip#41. |
The NDK's per-API sysroot lib dirs (on the clang driver link path) ship a legacy libstdc++.so stub, so SwiftPM's -lstdc++ resolves without help and --as-needed drops it; verified by CI run 32412113040 building the full dependency graph (incl. zsign's C++) with the mapping absent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This makes xtool build and run natively on Android with the Swift Android toolchain (tested on aarch64 Termux). All changes are additive platform guards; no behavior change on existing platforms.
Subprocess,AsyncHTTPClient,OpenAPIAsyncHTTPClient,WebSocketKit, andXADIdependencies on Android, matching Linux (Package.resolvedupdated accordingly).HTTPClienton Android too — the URLSession-based client needs Darwin-onlyURLSessionWebSocketTask. Also, bionic'sFILEis an incomplete type, sostdoutSafeis anOpaquePointerthere.Androidoverlay where needed (SIG_IGN,flock,errno); bionic markssignal()warn_unused_result.Companion PRs for the dependencies, required for an end-to-end Android build:
One remaining quirk when building on-device: SwiftPM doesn't pass swift-subprocess'
_SubprocessCShimsmodule map to dependents, so I had to build with-Xswiftc -Xcc -Xswiftc -fmodule-map-file=.../_SubprocessCShims/include/module.modulemap. This looks like a SwiftPM/Android toolchain issue rather than something xtool should work around, but I'm happy to add a note to the docs if you'd like.