build: portable wheels, buildable sdist, pyproject.toml instead of setup.py

[Hoeze]

tl;dr

• The bundled libsodium was compiled with -march=native, so the published wheels only run on CPUs like the CI runner's. This PR removes --enable-opt.
• With SODIUM_INSTALL=system, the module could end up without a link to libsodium on Linux. This is the -Wl,--no-as-needed problem from docs/setup.rst. The extension now declares libsodium itself.
• The sdist couldn't build a wheel, because MANIFEST.in pruned Makefiles that libsodium's configure needs.
• setup.py is replaced by pyproject.toml. The libsodium build step moves unchanged into build_libsodium.py.
• The testsuite now installs crypt4gh from the sdist, so CI covers what pip installs wherever no wheel matches.

Each point is a separate commit, so the first three can be taken without the others.

1. No -march=native in the bundled libsodium

--enable-opt adds -march=native -mtune=native (libsodium's configure.ac: "faster but not portable"). The release wheels are built by cibuildwheel on GitHub runners, so the wheels carry the runner's instruction set.

In the 1.8.6 cp312 manylinux x86_64 wheel, BMI2 instructions appear in blake2b_compress_ref, SHA512_Transform and fe25519_mul. AVX2 instructions appear in ge25519_* and crypto_scalarmult_curve25519_ref10. These are generic code paths, not runtime-dispatched ones. So the key exchange needs at least a Haswell-class CPU, and each wheel's requirement depends on which runner built it.

Without --enable-opt, libsodium still picks its SSSE3/AVX2/AVX-512 implementations at runtime.

2. Link libsodium with SODIUM_INSTALL=system

In system mode the extension did not declare libsodium, so users had to add -lsodium to LDFLAGS. setuptools puts LDFLAGS before the object files on the link line. A linker with --as-needed then drops libsodium, because nothing needs it yet at that point. --as-needed is the default on Ubuntu and is part of conda's LDFLAGS. The module then fails at import with undefined symbol: crypto_sign_ed25519_pk_to_curve25519.

Declaring libraries=['sodium'] in both modes puts -lsodium after the object files, so -Wl,--no-as-needed is no longer needed. The docs drop that advice. The Ubuntu CI job now uses the same LDFLAGS as macOS, so it exercises this.

3. Buildable sdist

MANIFEST.in prunes libsodium-stable/test, but configure lists test/Makefile and test/default/Makefile in AC_CONFIG_FILES. So building a wheel from the sdist failed with:

config.status: error: cannot find input file: 'test/default/Makefile.in'

cibuildwheel builds from the checkout, so CI never noticed. It affects pip install wherever no wheel matches, and python -m build, which builds the wheel from the sdist. MANIFEST.in now keeps the four Makefile.am/Makefile.in files of test/ and test/default/, and still prunes the tests themselves.

4. pyproject.toml instead of setup.py

• Metadata moves to [project], with the SPDX license expression Apache-2.0 (PEP 639). Hence setuptools>=77.
• The extension is declared in [tool.setuptools.ext-modules]. setuptools still labels that table experimental and prints a warning. Without setup.py, it's the only way to declare an extension with setuptools.
• BuildLibsodium moves to build_libsodium.py, which setuptools loads through [tool.setuptools.cmdclass]. Its logic is unchanged apart from commits 1 and 2.
• setuptools is now a build requirement instead of a line in requirements.txt.
• The release workflow builds the sdist with python -m build --sdist. PEP 517 sdist builds can't pass --owner=root --group=root, so the tarball entries now carry the runner's user. pip ignores them.
• The clean command is gone. It needed setup.py, and it didn't work anyway: CleanLibsodium didn't derive from Command. To clean, remove libsodium-build/ and run make distclean in libsodium-stable/.

Stack

This PR is part 2 of 3. A PR from a fork can only target master, so this PR also contains part 1. Please review only the last five commits. Once part 1 is merged with a merge commit, as usual in this repo, the diff shrinks to those five.

Part 3, refactor: replace libsodium with cryptography (#59), builds on this PR.

Stacked on #57.