Skip to content

feat: Validate ARM request destinations against the configured endpoint - #5786

Merged
Matthew Christopher (matthchr) merged 31 commits into
mainfrom
theunrepentantgeek-cuddly-carnival
Oct 22, 2026
Merged

Matthew Christopher (matthchr) merged 31 commits into
mainfrom
theunrepentantgeek-cuddly-carnival

Conversation

@theunrepentantgeek

Copy link
Copy Markdown
Member

🤖 TheUnrepentantRobot says:

What this PR does

This PR makes the configured Resource Manager endpoint the common destination boundary for ARM requests, including long-running operation polling, final-result requests and redirects.

  • Adds shared HTTPS-origin validation using the hostname and effective port. Hostname normalisation follows Go's HTTP transport, including internationalised names; an omitted port is equivalent to port 443.
  • Applies destination checks through the ARM pipeline, including resource-provider registration and SDK clients constructed from the generic client's options.
  • Adds client-local redirect checks while preserving existing callbacks, client settings and connection pools.
  • Keeps data-plane clients on the original HTTP client, without inheriting ARM-specific destination restrictions.

Polling and result URLs can still use different paths on the configured origin. Rejected destinations flow through existing reconciliation error handling and LRO recovery. Each client uses its selected cloud endpoint, including credential-specific endpoints in multi-environment configurations.

The change also documents the endpoint requirements and adds regression coverage for polling forms, final results, redirects, cloud isolation and data-plane compatibility. There are no new controller settings or changes to resume-token formats, CRDs or YAML samples.

How does this PR make you feel?

More predictable ARM request routing, while keeping data-plane behaviour separate.

Checklist

  • this PR contains documentation
  • this PR contains tests
  • this PR contains YAML Samples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Canonicalise non-ASCII hosts using the transport's IDNA lookup profile before ASCII case folding. Preserve custom ASCII authorities and reject failed conversions. Cover authenticated resumed pollers using a real transport with network-blocked controls.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the verified IDNA correction and final review closure. Keep the ARM branch local until coordinated disclosure is cleared.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resource-provider registration constructs a separate authenticated pipeline from the ARM client options. The existing redirect regression covered only a normal GetByID request, while the existing registration test returned a successful response. Neither would detect registration retaining an unguarded HTTP client.

Add an offline regression that returns MissingSubscriptionRegistration for the initial resource PUT, then redirects the registration POST to another origin. Assert the exact two legitimate requests, zero redirected sends, a typed DestinationError, and diagnostics without query secrets or bearer-token text. Keep the existing successful-registration test unchanged.

Verified the regression's sensitivity by temporarily giving the registration pipeline the original unguarded client while guarding the main pipeline: the test failed on an off-origin send. Restoring the correct construction order made it pass. All interactions use a fake RoundTripper, with no external requests.

Validation: final uncached, non-short race regressions passed across HTTP, generic ARM, and ARM recovery packages (30 top-level tests; 112 including subtests), including the existing successful-registration test. The required format-code, generator:quick-checks, and controller:quick-checks tasks passed separately in order. All 460 recording hashes are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The existing data-plane isolation regression sent only an initial request outside ARM. That request also succeeds through the ARM redirect wrapper, which intentionally validates redirects rather than initial requests. The test therefore could not detect DataPlaneClientOptions accidentally returning the ARM-guarded client instead of the original client.

Add an offline SDK-pipeline regression that follows a redirect between two non-ARM HTTPS origins. Assert successful completion, the exact initial and redirected requests, and invocation of the original CheckRedirect callback with the expected destination and request history. Preserve the existing user-agent, retry, cloud, logging, and telemetry coverage.

Verified the regression's sensitivity by temporarily returning the ARM-guarded transport from DataPlaneClientOptions: the test failed with a DestinationError at the redirect. Restoring the original HTTP client made it pass. Both origins are handled by a fake RoundTripper, so the test performs no DNS or external network requests. No production behavior changes are included.

Validation: final uncached, non-short race regressions passed across HTTP, generic ARM, and ARM recovery packages (30 top-level tests; 112 including subtests). The required format-code, generator:quick-checks, and controller:quick-checks tasks passed separately in order. All 460 recording hashes are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
IDNA Lookup can remove every character from a non-ASCII hostname without returning an error. In particular, a hostname made entirely of U+00AD SOFT HYPHEN maps to an empty string. The validator checked for a non-empty hostname before conversion but not afterward, allowing NewOrigin and ARM client construction to accept an origin that would reject every subsequent request.

Reject an empty hostname immediately after successful IDNA conversion. This restores the invalid-endpoint-fails-construction contract without adding DNS-label restrictions, changing the ASCII fast path, or rejecting valid Unicode hostnames merely because they contain ignored characters. This is a configuration-validation fix, not an off-origin credential-escape fix.

Add regressions for a single ignored character, repeated ignored characters, and an all-ignored hostname with a custom port. Each requires a typed DestinationError during construction. Add a positive equivalence case for an ignored character embedded in a valid hostname so the fix cannot simply reject all such input.

Red/green verification: all three empty-host cases failed against the prior implementation with successful construction, then passed after the one-line guard. Final uncached, non-short race regressions passed across HTTP, generic ARM, and ARM recovery packages (30 top-level tests; 112 including subtests), including existing custom-authority and Unicode/punycode cases. The required format-code, generator:quick-checks, and controller:quick-checks tasks passed separately in order. All 460 recording hashes are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expand existing origin, redirect-client, poller-response and test-fixture literals so each field is independently visible. Apply the same layout to nested client/cloud configuration and adjacent poller-state maps, without changing values or request behaviour.

Keep assertion conversions, named table restructuring, format arguments and NewOrigin control flow in their own commits. Newly introduced table literals use the same multiline convention in the table commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lay out the destination-error Sprintf call with the format string and each value on separate lines. Use the same layout for the transport regression's multi-argument diagnostic so token, send, authentication and dial observations are easy to scan.

Preserve the exact format strings, argument order and emitted output; this change affects readability only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Return immediately when parsing a configured endpoint fails or origin canonicalisation rejects it. Leave the successful origin return at the end of NewOrigin instead of nesting it beneath an err == nil branch.

Preserve the typed, sanitised DestinationError for both failure paths and retain all existing HTTPS, authority, port and IDNA validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace testing.Fatal and compound boolean checks with Gomega assertions throughout the origin and redirect tests and the shared poller-token helper. Express error-text checks with MatchError matchers rather than calling Error directly, and use MatchError for callback and blocked-dial sentinels.

Retain typed DestinationError checks, authentication and send counts, callback preservation, copied-client settings, redirect limits and response-body assertions. Keep table data and table restructuring separate from this assertion-only change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Convert slice-based and scalar-loop test cases to map[string]struct tables across origin validation, redirect handling, poller forms and final results, SDK cloud isolation, transport controls and reconciliation recovery. Use descriptive map keys as subtest names and remove redundant name fields.

Keep every existing input, expected result and no-network control. Expose formerly unsplit endpoint loops as parallel named subtests, and lay out all new case fields individually. Document dummy userinfo credentials narrowly where the new keyed fixtures trigger gosec.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@theunrepentantgeek

Copy link
Copy Markdown
Member Author

🤖 TheUnrepentantRobot says:

Guide to configured ARM request destinations

The change carries one configured ARM origin through request policies and HTTP redirects, while keeping ARM-specific behaviour out of data-plane clients.

Start here: v2/internal/genericarmclient/generic_client.go:61 - this connects the selected endpoint to both destination-checking mechanisms.

1. An origin is a hostname and effective port, not a URL prefix

What changed: The shared Origin type validates HTTPS endpoints and compares normalised hostnames and numeric ports. Hostname normalisation follows Go's HTTP transport conventions, including IDNA conversion; paths and queries are outside the comparison.

Why it matters: The same definition works across configured clouds and operation URLs without tying acceptance to a particular resource path.

Grok first: v2/internal/httpclient/origin.go:63, v2/internal/genericarmclient/generic_client.go:61

Questions:

  1. How does a destination with a different path and explicit port 0443 compare with the same hostname configured without a port?
  2. Why does Unicode hostname conversion precede case folding, and what happens if that conversion produces an empty hostname?

2. Validation follows outgoing requests, not resume-token fields

What changed: A per-call policy checks each destination entering the ARM pipeline. Pollers still use the SDK's existing resume-token handling and the client's pipeline for polling and final-result retrieval.

Why it matters: One request-level rule applies across ordinary ARM calls and the different LRO forms without introducing a second interpretation of SDK token formats.

Grok first: v2/internal/genericarmclient/destination_policy.go:26, v2/internal/genericarmclient/generic_client.go:82, v2/internal/genericarmclient/poller.go:32

Questions:

  1. How does a resumed operation reach the same destination policy used by ordinary ARM calls?
  2. How does Resume update its stored poller and response when polling or final-result retrieval fails, compared with a successful resume?

3. Redirects are a separate HTTP-client control flow

What changed: A copied HTTP client checks redirected destinations around the existing redirect callback, without replacing the underlying transport. Its Do wrapper also gives destination-validation errors a consistent outward form.

Why it matters: Redirects happen within the HTTP client rather than re-entering the SDK pipeline, so they need the same origin rule at that layer.

Grok first: v2/internal/httpclient/redirects.go:19, v2/internal/httpclient/redirects.go:43

Questions:

  1. What happens if an existing redirect callback changes an initially acceptable destination to another origin and returns no error?
  2. How do a wrapped DestinationError and an unrelated HTTP error emerge differently from Client.Do?

4. ARM and data-plane clients inherit different options

What changed: ARM options carry the destination policy and redirect-checked client into derived SDK clients and the separately constructed resource-provider registration pipeline. Data-plane options retain the original HTTP client while continuing to share cloud, logging, retry, telemetry and user-agent settings.

Why it matters: The ARM boundary follows ARM client construction without changing destination behaviour for connections to other services.

Grok first: v2/internal/genericarmclient/generic_client.go:99, v2/internal/genericarmclient/generic_client.go:137, v2/internal/genericarmclient/policy_register_rp.go:34

Questions:

  1. What destination-handling behaviour does a client built from ClientOptions() inherit that one built from DataPlaneClientOptions() does not?
  2. Why does installing the redirect-checked transport before constructing the resource-provider registration policy matter for that policy's own requests?

How the rest follows: The tests exercise these same ideas through origin variants, polling and final-result URLs, redirect callbacks, independent cloud clients and data-plane compatibility. The reconciliation test captures existing LRO recovery after a destination error. Documentation explains the endpoint contract, the existing IDNA dependency becomes direct, and the formatting, Gomega and named-table edits make those behaviours easier to read.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive routing boundary needs human validation, and a reconciliation recovery issue remains unresolved.

1 open issue
What changed in this PR

Restricts Azure Resource Manager requests to the configured HTTPS origin while keeping data-plane traffic independent.

Changes:

  • Adds origin validation to ARM pipelines and redirects.
  • Preserves shared HTTP settings without applying ARM restrictions to data-plane clients.
  • Documents endpoint requirements and adds regression coverage.
File Description
v2/​internal/​reconcilers/​arm/​destination_failure_test.go Tests generic reconciliation recovery.
v2/​internal/​httpclient/​redirects.go Guards redirected destinations.
v2/​internal/​httpclient/​redirects_test.go Tests redirect restrictions and compatibility.
v2/​internal/​httpclient/​origin.go Validates HTTPS origins.
v2/​internal/​httpclient/​origin_test.go Tests origin normalization and rejection.
v2/​internal/​genericarmclient/​generic_client.go Integrates guards and separates data-plane transport.
v2/​internal/​genericarmclient/​destination_transport_test.go Checks transport hostname behavior.
v2/​internal/​genericarmclient/​destination_sdk_test.go Tests SDK enforcement and cloud isolation.
v2/​internal/​genericarmclient/​destination_poller_test.go Tests polling and final-result destinations.
v2/​internal/​genericarmclient/​destination_policy.go Rejects disallowed pipeline destinations.
v2/​internal/​genericarmclient/​destination_policy_test.go Tests early rejection and registration redirects.
v2/​internal/​genericarmclient/​data_plane_client_options_test.go Verifies data-plane redirect compatibility.
v2/​go.mod Makes hostname-normalization dependency direct.
docs/​hugo/​content/​guide/​aso-controller-settings-options.md Documents endpoint restrictions.

Review effort: Balanced

Comment thread v2/internal/genericarmclient/generic_client.go
@theunrepentantgeek Bevan Arps (theunrepentantgeek) changed the title Validate ARM request destinations against the configured endpoint feat: Validate ARM request destinations against the configured endpoint Oct 19, 2026
Comment thread docs/hugo/content/guide/aso-controller-settings-options.md Outdated
Comment thread v2/internal/genericarmclient/data_plane_client_options_test.go Outdated
Comment thread v2/internal/genericarmclient/destination_sdk_test.go
Comment thread v2/internal/httpclient/origin.go Outdated
if host == "" || strings.ContainsAny(host, "%\\/?#@ \t\r\n") {
return Origin{}, false
}
if strings.Contains(host, ":") && net.ParseIP(host) == nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 net.ParseIP(host) is only used as a validity check here; its canonical representation is discarded. A configured origin such as [2001:db8::1] will therefore reject [2001:0db8:0:0:0:0:0:1], even though both literals identify the same IPv6 address. Please normalize parsed IP hosts with ip.String() before storing/comparing them, and cover compressed versus expanded IPv6 forms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what it is trying to say here is that the above two IPs will not compare as equal in this code.

Here's a more detailed breakdown from my agent after I asked it to explain more:

This is a false rejection, not a security bypass. The validator can reject two URLs that Go ultimately sends to the same IP address.

Current behavior

originOf() extracts the hostname and checks IPv6 validity:

Go

host := authority.Hostname()

if strings.Contains(host, ":") && net.ParseIP(host) == nil {
    return Origin{}, false
}

But afterward it stores the original textual representation:

Go

return Origin{
    host: strings.ToLower(host),
    port: strconv.Itoa(n),
}, true

Origin.Validate() compares the structs directly:

Go

candidate == o

That means the host strings must be identical.

Why that fails for IPv6

IPv6 permits multiple textual representations of the same 128-bit address:

Plain text

2001:db8::1
2001:0db8:0:0:0:0:0:1
2001:0DB8:0000:0000:0000:0000:0000:0001

All three represent the same address. net.ParseIP() successfully parses each, and comparing the parsed addresses would show they are equal. However, lowercasing only fixes the third form’s uppercase letters; it does not remove leading zeros or expand/compress zero groups.

For example:

Go

allowed, _ := NewOrigin("https://[2001:db8::1]")

err := allowed.Validate(
    mustParseURL("https://[2001:0db8:0:0:0:0:0:1]/operations/123"))

Internally, the comparison becomes approximately:

Go

Origin{
    host: "2001:db8::1",
    port: "443",
} == Origin{
    host: "2001:0db8:0:0:0:0:0:1",
    port: "443",
}

That is false even though both URLs reach the same address.

When it could affect ASO

This requires a configured Resource Manager endpoint using an IPv6 literal rather than a DNS name, for example:

Plain text

https://[2001:db8::1]

It then requires ARM, a custom gateway, a redirect, or an LRO resume token to express that address differently. The request would be rejected with DestinationError, even though it stays on the same endpoint.

Possible consequences include:

  • LRO polling stops.

  • A final-result request fails.

  • A redirect is rejected.

  • A persisted resume token repeatedly fails after an endpoint-format change.

Public Azure endpoints such as management.azure.com are unaffected because they use DNS names. The concern is primarily custom clouds, private gateways, and testing infrastructure.

Why normalization is safe

Canonicalization would not permit another server or DNS alias. It only makes different encodings of the same IP address compare equal.

A suitable implementation could use net/netip, whose Addr.String() returns a canonical representation:

Go

if strings.Contains(host, ":") {
    addr, err := netip.ParseAddr(host)
    if err != nil || !addr.Is6() {
        return Origin{}, false
    }

    host = addr.String()
}

Both inputs would then normalize to:

Plain text

2001:db8::1

Tests should cover at least:

Go

"https://[2001:db8::1]"
"https://[2001:0db8:0:0:0:0:0:1]"
"https://[2001:0DB8:0000:0000:0000:0000:0000:0001]"

IPv4-mapped IPv6 addresses should also have an explicit test because different normalization APIs may either preserve the mapped IPv6 form or collapse it to dotted IPv4.

Priority assessment

Medium reflects the impact—requests can be permanently blocked—but the trigger is uncommon. If literal IPv6 custom ARM endpoints are not an expected supported configuration, Low would also be defensible. It is not a vulnerability because the current behavior is stricter than necessary; it is a compatibility and availability issue.

Comment thread v2/internal/genericarmclient/destination_policy.go
Comment thread v2/internal/genericarmclient/generic_client.go Outdated
Comment thread v2/internal/httpclient/origin.go Outdated
Comment thread v2/internal/httpclient/origin.go Outdated
Comment thread v2/internal/httpclient/redirects.go Outdated
Comment thread v2/internal/httpclient/redirects.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Clarified the one remaining comment but feel free to ignore it OR if you think it's worth fixing, fix in a followup PR

Comment thread v2/internal/httpclient/origin.go Outdated
if host == "" || strings.ContainsAny(host, "%\\/?#@ \t\r\n") {
return Origin{}, false
}
if strings.Contains(host, ":") && net.ParseIP(host) == nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what it is trying to say here is that the above two IPs will not compare as equal in this code.

Here's a more detailed breakdown from my agent after I asked it to explain more:

This is a false rejection, not a security bypass. The validator can reject two URLs that Go ultimately sends to the same IP address.

Current behavior

originOf() extracts the hostname and checks IPv6 validity:

Go

host := authority.Hostname()

if strings.Contains(host, ":") && net.ParseIP(host) == nil {
    return Origin{}, false
}

But afterward it stores the original textual representation:

Go

return Origin{
    host: strings.ToLower(host),
    port: strconv.Itoa(n),
}, true

Origin.Validate() compares the structs directly:

Go

candidate == o

That means the host strings must be identical.

Why that fails for IPv6

IPv6 permits multiple textual representations of the same 128-bit address:

Plain text

2001:db8::1
2001:0db8:0:0:0:0:0:1
2001:0DB8:0000:0000:0000:0000:0000:0001

All three represent the same address. net.ParseIP() successfully parses each, and comparing the parsed addresses would show they are equal. However, lowercasing only fixes the third form’s uppercase letters; it does not remove leading zeros or expand/compress zero groups.

For example:

Go

allowed, _ := NewOrigin("https://[2001:db8::1]")

err := allowed.Validate(
    mustParseURL("https://[2001:0db8:0:0:0:0:0:1]/operations/123"))

Internally, the comparison becomes approximately:

Go

Origin{
    host: "2001:db8::1",
    port: "443",
} == Origin{
    host: "2001:0db8:0:0:0:0:0:1",
    port: "443",
}

That is false even though both URLs reach the same address.

When it could affect ASO

This requires a configured Resource Manager endpoint using an IPv6 literal rather than a DNS name, for example:

Plain text

https://[2001:db8::1]

It then requires ARM, a custom gateway, a redirect, or an LRO resume token to express that address differently. The request would be rejected with DestinationError, even though it stays on the same endpoint.

Possible consequences include:

  • LRO polling stops.

  • A final-result request fails.

  • A redirect is rejected.

  • A persisted resume token repeatedly fails after an endpoint-format change.

Public Azure endpoints such as management.azure.com are unaffected because they use DNS names. The concern is primarily custom clouds, private gateways, and testing infrastructure.

Why normalization is safe

Canonicalization would not permit another server or DNS alias. It only makes different encodings of the same IP address compare equal.

A suitable implementation could use net/netip, whose Addr.String() returns a canonical representation:

Go

if strings.Contains(host, ":") {
    addr, err := netip.ParseAddr(host)
    if err != nil || !addr.Is6() {
        return Origin{}, false
    }

    host = addr.String()
}

Both inputs would then normalize to:

Plain text

2001:db8::1

Tests should cover at least:

Go

"https://[2001:db8::1]"
"https://[2001:0db8:0:0:0:0:0:1]"
"https://[2001:0DB8:0000:0000:0000:0000:0000:0001]"

IPv4-mapped IPv6 addresses should also have an explicit test because different normalization APIs may either preserve the mapped IPv6 form or collapse it to dotted IPv4.

Priority assessment

Medium reflects the impact—requests can be permanently blocked—but the trigger is uncommon. If literal IPv6 custom ARM endpoints are not an expected supported configuration, Low would also be defensible. It is not a vulnerability because the current behavior is stricter than necessary; it is a compatibility and availability issue.

Merged via the queue into main with commit 7482ad2 Oct 22, 2026
10 checks passed
@matthchr
Matthew Christopher (matthchr) deleted the theunrepentantgeek-cuddly-carnival branch October 22, 2026 22:03
@github-project-automation github-project-automation Bot moved this from In Progress to Recently Completed in Azure Service Operator Roadmap Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Recently Completed

Development

Successfully merging this pull request may close these issues.

3 participants