Skip to content

Add WorkerProxy extension artifact compatibility shim - #11963

Open
Shyju Krishnankutty (kshyju) wants to merge 1 commit into
devfrom
shkr/worker_proxy_extension_fetching
Open

Shyju Krishnankutty (kshyju) wants to merge 1 commit into
devfrom
shkr/worker_proxy_extension_fetching

Conversation

@kshyju

@kshyju Shyju Krishnankutty (kshyju) commented Sep 1, 2026

Copy link
Copy Markdown
Member

Summary

Adds an internal WorkerProxy compatibility shim that packs extensions.json and .azurefunctions from a supplied function app directory into a tar payload, for worker SDKs that do not ship extension artifacts.

  • Deterministic output. Entries are sorted by emitted entry name and stamped 0644 and the Unix epoch, so identical content yields identical bytes. Enumeration order is a filesystem property on Linux, and inherited metadata would let an extracted extension keep the execute bit or stay writable by another account.
  • Unusable inputs yield no artifact and log a warning, matching the host's existing MissingAzureFunctionsFolderValidator checks — a missing, wrong-kind, or empty input generally means the deployment package was assembled incorrectly. An empty or absent function app directory is reported apart from a publish output that carries content but no extensions.json, since the two point at different causes.
  • Unreadable inputs propagate as access failures instead of being reported as absent.
  • Symbolic links found while enumerating .azurefunctions are excluded, along with the contents of any linked directory, so the archive covers only files that belong to the deployment. The two root inputs are resolved the way the host resolves them, and are not themselves link-checked.
  • Honors cancellation.

Testing

18 test methods (22 cases) cover archive contents and ordering, byte-identical output for identical content stamped with differing permissions and timestamps, the fixed entry metadata, hidden-file inclusion, symbolic link exclusion, every unavailable-input case with the level it is logged at, access-failure propagation, and cancellation. Functions.WorkerProxy.Tests runs 47 cases in total.

CreateAsync_ExcludesSymbolicLinkedContent and CreateAsync_PropagatesAccessFailureInsteadOfReportingMissingInputs assert behavior the host OS has to support (symbolic link creation, permission enforcement) and report as skipped where it is unavailable rather than passing without exercising anything. CI runs the suite on Linux through Azure.Functions.Host.slnx.

Follow-up

The shim is registered but not hooked up yet. Capability selection and runtime/WorkerProxy gRPC integration will be added in a future PR.

Issue describing the changes in this PR

N/A

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

  • Backporting to the in-proc branch is not required
  • My changes do not require documentation changes
  • My changes should not be added to the release notes for the next release
  • My changes do not need to be backported to a previous version
  • My changes do not require diagnostic events changes
  • I have added all required tests (Unit tests, E2E tests)

Copilot AI lite review requested due to automatic review settings September 1, 2026 22:17
@kshyju Shyju Krishnankutty (kshyju) added the compute-separation Compute separation / external worker support label Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A few correctness/quality issues (notably inaccurate exception documentation and guard-clause diagnostics) should be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a WorkerProxy-internal “extension artifact” compatibility shim that can package extensions.json plus the .azurefunctions directory into a tar payload and compute a SHA-256 digest, and registers the shim in the WorkerProxy DI container (with unit tests covering archive ordering/digest and service registration).

Changes:

  • Introduces ExtensionArtifactShim + IExtensionArtifactShim and ExtensionArtifact model under Functions.WorkerProxy.
  • Registers the shim as a singleton in WorkerProxyApplication.
  • Adds unit tests validating payload/digest behavior, required-input handling, cancellation, and DI registration.
File summaries
File Description
test/Functions.WorkerProxy.Tests/ExtensionArtifactShimTests.cs Adds unit tests for archive payload ordering, digest correctness, required-input checks, and cancellation behavior.
test/Functions.WorkerProxy.Tests/ExtensionArtifactShimRegistrationTests.cs Verifies the shim is registered as a singleton and resolves as ExtensionArtifactShim.
src/Functions.WorkerProxy/WorkerProxyApplication.cs Registers IExtensionArtifactShim as a singleton in the WorkerProxy app container.
src/Functions.WorkerProxy/ExtensionArtifacts/IExtensionArtifactShim.cs Defines the internal shim interface contract for creating extension artifacts.
src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.Log.cs Adds source-generated logging for shim availability and archive creation events.
src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.cs Implements tar creation, deterministic ordering, digest computation, and required-input validation.
src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifact.cs Adds the artifact payload + digest model used by the shim.
Review details

Suppressed comments (1)

src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.Log.cs:19

  • This archive-prepared log is likely to be produced during normal operation and could be frequent depending on how often the shim is invoked. To avoid high-volume Information logs, consider using Debug (consistent with other WorkerProxy operational traces).
    [LoggerMessage(
        EventId = 2,
        Level = LogLevel.Information,
        Message = "Extension artifact archive prepared. Digest='{Digest}', Size={SizeBytes} bytes.")]
  • Files reviewed: 7/7 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.Log.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The shim currently archives any files under .azurefunctions without filtering symlinks/reparse points, which can allow reading and packaging content from outside the function app directory.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The shim currently guards against symlinks inside .azurefunctions but can still follow symlinked root inputs (extensions.json and/or the .azurefunctions directory itself), allowing out-of-app content into the payload/digest.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.cs:80

  • AttributesToSkip = ReparsePoint prevents traversing symlinks inside the .azurefunctions tree, but if the .azurefunctions directory itself is a symlink the shim will still enumerate and archive files outside the function app root. Guarding against a reparse-point root directory avoids this escape hatch and keeps the payload deterministic.
        var azureFunctionsDirectory =
            Path.Combine(functionAppDirectory, AzureFunctionsDirectoryName);
        if (!Directory.Exists(azureFunctionsDirectory))
        {
  • Files reviewed: 7/7 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The shim currently allows symlink/reparse-point roots for required inputs (extensions.json / .azurefunctions), which can cause out-of-deployment content to be archived and contradicts the stated symlink-exclusion guarantees.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/Functions.WorkerProxy/ExtensionArtifacts/ExtensionArtifactShim.cs:185

  • ProbeArtifactPath treats reparse points (e.g., symlinks) as usable inputs. Because EnumerationOptions only skips discovered entries (not the root), a symlinked extensions.json or .azurefunctions root would be followed and could archive/digest out-of-deployment content. Reject FileAttributes.ReparsePoint up front when probing required paths.
        // Attributes are readable for a directory standing where a file belongs, so the kind is
        // what separates a usable input from one that only fails later, on open or on walk.
        bool isDirectory = (attributes & FileAttributes.Directory) != 0;

        return isDirectory == expectDirectory
  • Files reviewed: 9/9 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread test/Functions.WorkerProxy.Tests/ExtensionArtifactShimTests.cs
@kshyju
Shyju Krishnankutty (kshyju) force-pushed the shkr/worker_proxy_extension_fetching branch 2 times, most recently from c26b8d9 to a122f56 Compare September 2, 2026 15:25
@kshyju
Shyju Krishnankutty (kshyju) marked this pull request as ready for review September 2, 2026 15:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The shim currently treats reparse points as usable roots, so a symlink/junction extensions.json or .azurefunctions root can cause out-of-app content to be archived, and a regression test for the .azurefunctions root-symlink case is needed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

test/Functions.WorkerProxy.Tests/ExtensionArtifactShimTests.cs:279

  • There is test coverage for symlinks within .azurefunctions, but not for the case where the .azurefunctions root itself is a symlink. Without a guard, the shim will enumerate and archive the linked directory’s contents (out-of-app), which is both a determinism and security issue. Add a regression test that asserts CreateAsync returns null when the .azurefunctions root is a symlink.
  • Files reviewed: 9/9 changed files
  • Comments generated: 1
  • Review effort level: Lite

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new xUnit Fact attributes can throw during test discovery (temp directory creation/cleanup and platform/filesystem exceptions), which can fail the entire test run instead of skipping as intended.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread test/Functions.WorkerProxy.Tests/RequiresPermissionEnforcementFactAttribute.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation matches the PR’s stated behavior and is backed by targeted, capability-aware tests that cover determinism, ordering, warnings, cancellation, and access-failure propagation.

Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

/// <summary>
/// Represents an extension artifact payload.
/// </summary>
internal sealed record ExtensionArtifact

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a type for this? Can it just be Stream instead? The benefit of a Stream over ReadOnlyMemory<byte> abstraction is that a stream doesn't have to be in memory at all. We could directly stream from the filesystem to the caller without ever loading the payload into the processes memory.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to see a pattern which doesn't need a special contract for the shim, but a shared contract that both the shim approach and the worker-provided approach satisfy.

Maybe a handler pattern? Or something where each provider is called in order until one is able to provide the value.

public interface IExtensionArtifactProvider
{
     // probably need a wrapper around `Stream` to identify handled vs not-handled
    Task<Stream> GetAsync(CancellationToken cancellationToken);
}

and then have the construction of the shim version get the function app directory supplied to it via DI'd options?

WorkerProxy registers an internal compatibility shim for worker SDKs that do not provide extension artifacts.

Given a caller-supplied function app directory, it validates extensions.json and .azurefunctions, then packs both into a tar payload. Entries are ordered by the name they are written under and are stamped with a fixed mode and timestamp, so identical content yields a byte-identical payload. Directory enumeration order is a filesystem property on Linux and cannot supply that on its own.

An input that is missing or of the wrong kind is reported as a warning and yields no artifact, matching how the host already reports a deployment whose publish output was zipped incorrectly. A function app directory that is empty, or that does not exist, is reported apart from a publish output that carries content but not extensions.json, because the two point at different failures. An input that cannot be inspected propagates instead of being reported as absent, so an unreadable deployment is never mistaken for one that carries no extensions. Artifact creation honors caller cancellation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5b618ea7-21fa-4d8f-be02-c5e6c4593bb6
@kshyju
Shyju Krishnankutty (kshyju) force-pushed the shkr/worker_proxy_extension_fetching branch from 1009814 to 0e65ba5 Compare September 15, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compute-separation Compute separation / external worker support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants