ci: publish metaflow before metaflow-stubs on release

[anxkhn]

PR Type

• Bug fix
• New feature
• Core Runtime change (higher bar -- see CONTRIBUTING.md)
• Docs / tooling
• Refactoring

On release, publish.yml uploaded metaflow-stubs to PyPI before metaflow.
Because metaflow-stubs hard-pins metaflow==<same version>, publishing the
stubs first leaves them briefly (and, if the metaflow upload then fails,
permanently) uninstallable. This swaps the two publish steps so metaflow
publishes first.

Issue

Fixes #

Reproduction

Runtime: release workflow (GitHub Actions deploy job in .github/workflows/publish.yml)

This is a release-pipeline ordering defect, so the "reproduction" is the ordering
plus the version pin rather than a runnable local command; a live failure only
surfaces during an actual PyPI release.

Where evidence shows up: the deploy job's publish steps / the resulting PyPI index state.

The dependent package pins the dependency exactly:

# stubs/setup.py
version = <read from ../metaflow/version.py>          # same version as the metaflow package
install_requires=[f"metaflow=={version}"]             # stubs/setup.py:45

Before this change the deploy job published in this order:

# .github/workflows/publish.yml (before)
- name: Publish metaflow-stubs package     # uploaded FIRST (packages-dir: ./stubs/dist)
- name: Publish metaflow package           # uploaded SECOND (root dist)

So immediately after the first step, metaflow-stubs==X is on PyPI while
metaflow==X is not, and pip install metaflow-stubs==X cannot resolve its
metaflow==X dependency. If the second step (publishing metaflow) fails for any
reason (a PyPI hiccup, a metadata rejection, a re-run where the file already
exists), the state is permanent: PyPI does not allow re-uploading the same file,
and gh-action-pypi-publish defaults skip-existing: false, so a retried release
also dies on the already-published stubs.

After this change:

# .github/workflows/publish.yml (after)
- name: Publish metaflow package           # uploaded FIRST (root dist)
- name: Publish metaflow-stubs package     # uploaded SECOND (packages-dir: ./stubs/dist)

The depended-upon package is on the index before the package that pins it, so the
metaflow-stubs dependency is always resolvable.

Root Cause

The two PyPI publish steps in the deploy job were ordered dependent-first: the
metaflow-stubs upload ran before the metaflow upload, even though
metaflow-stubs's install_requires hard-pins the exact metaflow version
(stubs/setup.py:45), which is not on the index until the later step completes.

Why This Fix Is Correct

Standard practice is to publish a depended-upon distribution before any
distribution that pins it, so the dependency is resolvable the moment the
dependent package appears. Swapping the two steps restores that invariant with the
smallest possible change: only the order of the two existing
pypa/gh-action-pypi-publish steps changes. The packages-dir: ./stubs/dist
setting stays attached to the metaflow-stubs step, and the action version pins
are untouched.

Failure Modes Considered

1. Second upload fails after the first succeeds. Before: metaflow-stubs==X is
   orphaned with an unresolvable pin, and because PyPI blocks re-uploading the same
   file (skip-existing defaults to false), the release cannot be cleanly retried.
   After: if the second upload (metaflow-stubs) fails, metaflow==X is already
   published and installable on its own; only the stubs are missing, which is the
   less harmful ordering.
2. YAML step reordering misattaching with: packages-dir. Verified the
   packages-dir: ./stubs/dist block stays under the metaflow-stubs step and the
   metaflow step keeps no packages-dir (it uploads the root-built dist),
   confirmed by parsing the job with yaml.safe_load.

Tests

• Unit tests added/updated
• Reproduction script provided (required for Core Runtime)
• CI passes
• If tests are impractical: explain why below and provide manual evidence above

No unit test applies to a release-workflow ordering change (the repo has no test
that executes publish.yml, and the effect only manifests during a real PyPI
release). Validation: pre-commit run --files .github/workflows/publish.yml passes
(check-yaml, end-of-file, trailing-whitespace, mixed-line-ending, detect-private-key
all Passed), and the post-swap step order was confirmed by parsing the deploy job.

Non-Goals

• Not adding skip-existing: true to the publish steps. That would make re-runs
  idempotent and is arguably a good follow-up, but it changes publish semantics and
  is out of scope for this ordering fix. Kept the diff to the two-line reorder.
• Not touching the metaflow-stubs version pin in stubs/setup.py, the action SHA
  pins, or any other step.

AI Tool Usage

• No AI tools were used in this contribution
• AI tools were used (describe below)

An AI coding assistant helped locate the ordering defect and draft this PR text. I
reviewed and understand every line: the change is a two-line reorder of the two
existing pypa/gh-action-pypi-publish steps in .github/workflows/publish.yml so
the metaflow package publishes before the metaflow-stubs package that pins it.
I validated it with pre-commit and by re-parsing the resulting workflow.

[greptile-apps]

Greptile Summary

This PR changes the release publish order for the Python packages.

• Publishes metaflow before metaflow-stubs.
• Keeps the stubs upload scoped to ./stubs/dist.
• Leaves the PyPI publish action version and build steps unchanged.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.
• The root package upload still uses the default dist directory.
• The stubs package upload remains scoped to ./stubs/dist.

Important Files Changed

Filename	Overview
.github/workflows/publish.yml	The deploy job now uploads the root package before the stubs package while preserving the stubs-specific packages-dir setting.

_{Reviews (1): Last reviewed commit: "ci: publish metaflow before metaflow-stu..." | Re-trigger Greptile}

[anxkhn]

@saikonen when you have a chance, could you please take a look at this pr and let me know if any update is needed?

[Shriprasad-P]

Review

PR: ci: publish metaflow before metaflow-stubs on release

Touched: .github/workflows/publish.yml

• CI/tooling change — confirm the pipeline still passes on this branch.
• Size looks manageable (+2/-2).

Commenting as a drive-by reviewer after reading the diff. Happy to look again if maintainers want a deeper pass on a specific file.