Skip to content

[HDX-12246] migrate: support customer-required clusters (6.4) - #74

Open
barottomartin wants to merge 1 commit into
devfrom
HDX-12246/migrate-customer-support
Open

[HDX-12246] migrate: support customer-required clusters (6.4)#74
barottomartin wants to merge 1 commit into
devfrom
HDX-12246/migrate-customer-support

Conversation

@barottomartin

Copy link
Copy Markdown

What

Retrofits the global hdxcli migrate command for turbine-api 6.4's multi-org breaking changes.

Since turbine-api 6.3 (HDX-11681), project creation requires an existing Customer on the target cluster, and tables may only reference storages/credentials that are members of the project's customer (HDX-11643). migrate previously forwarded the source project body verbatim (carrying the source cluster's customer UUID), which fails against 6.4 targets — exactly the break flagged on the ticket.

Changes

  • New migrate/customer.py
    • Detects whether the target requires a customer from the projects OPTIONS metadata, so pre-6.3 targets are untouched (no version checks).
    • Resolves the target customer from --target-customer (name or UUID) → an existing target project's immutable customer → an interactive picker; offers to create the customer when missing, mapping 401/403 to an actionable insufficient-permissions message.
    • Registers the migrated table's non-default storages and autoingest credentials with the customer (add_storage/add_credential), always behind a confirmation, running before any resource is created so a decline or 403 never leaves a half-migrated project.
  • resources.py — strip the source customer/org, inject the resolved customer on project create, run the membership pre-flight.
  • commands.py — add --target-customer / -tc and document it.
  • Bumps hdxcli to 1.0.84.

Scope

Per the ticket comment, this covers only the global migrate command. The config-only project migrate (and the shared migrate_resource_config engine) have the same break and are left as a follow-up; hdxcli project create is also affected and out of scope here.

Testing

  • 21 unit tests in tests/test_migrate_customer.py (resolution precedence, OPTIONS gating, collectors, default-candidate rule). Full suite green.
  • Live, cross-version:
    • 6.4 AIO (v6.3.0-rc.1-193): flag resolution, interactive picker, customer creation offer, existing-project inheritance, conflicting-flag warning, and the storage-membership pre-flight (decline aborts before any creation; accept runs add_storage and the table create passes validation).
    • Local docker on turbine v6.0.17 (genuine pre-customer release): old→new upgrade (v6.0 source → 6.4 target, customer resolved + injected) and the new→old regression guard (6.4 source → v6.0 target: "Target customer → Not required", all customer/membership logic skipped, project created as legacy).

Notes

  • v1 nested URLs still work on 6.4 (no redirects/deprecation), so no URL changes were needed.
  • No PyPI publish is triggered by merge (repo has no publish workflow); the version bump is unspent until a manual tag + upload.

🤖 Generated with Claude Code

Since turbine-api 6.3 (HDX-11681), project creation requires an existing
Customer on the target cluster, and tables may only reference storages and
credentials that are members of the project's customer (HDX-11643). The
`migrate` command previously forwarded the source project body verbatim,
which fails against 6.4 targets.

Add a customer-resolution and membership pre-flight step to the global
`migrate` command:

- New `migrate/customer.py`:
  - Detects whether the target requires a customer from the projects
    OPTIONS metadata, so pre-6.3 targets are untouched (no version checks).
  - Resolves the target customer from `--target-customer` (name or UUID),
    an existing target project's immutable customer, or an interactive
    picker; offers to create the customer when missing, mapping 401/403 to
    an actionable insufficient-permissions message.
  - Registers the migrated table's non-default storages and autoingest
    credentials with the customer (add_storage / add_credential), always
    behind a confirmation, running before any resource is created so a
    decline or 403 never leaves a half-migrated project.
- `resources.py`: strip the source customer/org, inject the resolved
  customer on project create, and run the membership pre-flight.
- `commands.py`: add `--target-customer` / `-tc` and document it.

Verified live across versions: 6.4 AIO (flag, picker, creation offer,
inheritance, conflicting-flag warning, storage membership accept/decline)
and a local v6.0.17 pre-customer cluster (old->new upgrade injects the
customer; new->old skips all customer logic).

Bumps hdxcli to 1.0.84.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant