Consider strengthening host invariants

[littledan]

In the current draft, hosts have lots of flexibility:

• Hosts may ignore or reject unknown attributes
• Hosts may reject imports of the same specifier with mismatching attributes, or make a separate copy interpreted differently
• Hosts may expose attributes in import.meta, but not guaranteed

Based on the discussion in openjs-foundation/standards#91 , I think it's important that we consider making more specific requirements in hosts. This investigation would be based on details of concrete hosts and attributes.

[littledan]

cc @devsnek @jkrems

[ljharb]

In general, I think any place where we have no choice but to delegate to hosts is a potential (albeit, sometimes appropriate) interoperability hazard. I am strongly in favor of locking down as much as possible.

[hybrist]

For the investigation, the scenarios I'm most interested is code reuse across (versions of) hosts. Which I guess is another phrasing of the interoperability hazard @ljharb mentioned but I think it's important to stress that this doesn't require cross-environment interactions. The hazard exists even within a single host if the set of recognized attributes changes over time.

Given those potential hazards, my current assumption is that there will be two possible outcomes:

1. The spec locks down one specific behavior and hosts align with the spec.
2. The web hosts lock down one specific behavior and every other host that ever wants to run a JS file that also supports browsers has to match it. The other behaviors would be effectively impossible to choose for a host but they need to check the HTML spec to find out.

So while I would vastly prefer if we could lock this down as part of ECMAScript, the 2nd path does exist.

@littledan For this investigation, is there a specific format you had in mind? E.g. is there anything that's particularly helpful that could be posted in this issue without just adding noise?

[littledan]

There are a few different things here--in openjs-foundation/standards#91, we talked about caching behavior, but @jkrems discusses reuse across environments, part of which is the set of attributes and values. I want to cover these separately.

About alignment between environments in general: I want to respect the fact about the world that there are some JS environments that are working towards more web compatibility (e.g., Deno) and other environments that explicitly have web compatibility as a non-goal (e.g., XS/TC53). The difference already shows through in basic aspects, like whether the file extension is part of the module specifier or not.

To respect the whole JS ecosystem, just about anything we do with modules in TC39 will have to be careful about supporting both cases. Environments which want to align with the web will, to some extent, work on aligning with browsers outside of TC39, i.e., 2 (which is the same approach as import maps across environments). At the same time, this proposal still tries to gather up as much as possible and require it uniformly across JS environments.

Overall, I believe that the concerns here, when it comes to both caching and the set of attributes and values, are the same for in-band and out-of-band approaches. If we had an out-of-band format, we'd still want it to be meaningful across JS environments, for just the same reasons. And we'd still face the question of what the semantics are when linking different packages together.

Set of attributes and values

In general, this proposal does aim to define as much as possible in ECMA-262. That's why type: "json" and invariants for type are in this specification.

At the same time, this proposal also aims to permit types which only make sense on some environments and not on others. These include HTML, CSS and WebAssembly modules. I wouldn't want to require these in all JS environments. What we could do is require that, if a particular type is supported, then it has a particular interpretation (defined in another spec). I'd be happy to add this as follow-on normative PRs if we get consensus on them.

This proposal aims to not close us off from adding more attributes. In issues, people have proposed various other attributes, which we're not defining the first time around. Some of the attributes proposed are environment-specific. I feel like it's a bit early to cut these all off entirely.

For this investigation during Stage 2, the next steps would be to look more into the possible type values, and possible non-type attributes, in more detail, and see if there's more that we should include in this specification, in terms of requirements for all JS hosts, or evolution over time to add.

Caching behavior

A core question is, "should module attributes be part of the cache key?" or, in other words, "is the host allowed to 'clone' a module to import it with different attributes?"

Attributes which just "check" the module, such as type (or, theoretically, an integrity attribute, but we probably shouldn't do that with module attributes, and instead with something out of band) do not transform the module. But other proposed attributes would change the interpretation of the module (and this includes, exposing all attributes to the JS in the module through import.meta.attributes). I feel like this is an interesting area, and I'm not ready to close it off entirely.

If you have multiple modules in the module graph that import the same module specifier, but with different attributes, where the attribute changes how the module is interpreted (not just checked), then the system has three choices:

• Race and use the attribute that was requested first (this seems broken--the second usage is ignored)
• Reject the module graph and don't load (this seems bad for composition--using two unrelated packages together could break)
• Clone and make two copies of the module, for the different ways it's transformed (folks raised concerns about this in [TC39] Potentially blocking "Module Attributes" proposal openjs-foundation/standards#91)

The current spec text allows hosts to choose any of these three, and the choice may be based on what the attribute is. I'd be open to restricting it to just one or two of those choices (or saying, module attributes can never transform, only check), but I think this determination should be based on some study and agreed-on narrowing of use cases.

For this investigation during Stage 2, the next steps would be to study the possible 'transforming' module attributes, look into how modules are composed and used together across various JS environments, and see if, based on those details, we want to generally prohibit this "cloning".

[littledan]

Now that #66 has landed, I'm wondering if people have further concerns that should be expressed in the host invariants, or if this issue should be closed. I'll leave it open for a bit for people to express any further concerns.

[hybrist]

Positive confirmation: I think for me this is resolved with the recent updates.

[phoddie]

... and other environments that explicitly have web compatibility as a non-goal (e.g., XS/TC53)

I wanted to add a couple of clarifications about this comment. (I have no immediate concerns about the discussion in this issue.)

XS does support, and intends to continue to support, the full ECMA-262 module specification. The topic of module paths cause some confusion. XS allows a host to implement web style paths. Moddable does that for our desktop device simulator mcsim, for example. On constrained devices which don't even have a file system, paths are meaningless and have overhead. In hosts for those devices, path resolution is therefore omitted. Bare module specifiers are sufficient.

Interestingly, thee work on Compartments in Secure ECMAScript looks like it may give a way, using only JavaScript, to map paths to bare module specifiers, if that's what a project wants.

Regarding TC53, we will provide web compatible APIs where practical. The committee has been clear that there is no desire to create meaningless differences. However, as the priority is a feasible implementation on constrained devices, that is often not the case.

[bmeck]

This is coming up again in the Node Modules WG due to integration concerns, just a heads up. There is disagreement given the proposed change to allow same URL differing resources in a module map for HTML integration. We will be discussing this in the following meetings and hope to have some better feedback before plenary.

nodejs/modules#427