allow non-trivial expressions in flakes

[KiaraGrouwstra]

lifts flakes' restriction allowing the use of only trivial syntax, which has particularly affected its inputs.

Motivation

• lifting flakes' syntax restrictions to allow non-trivial nix expressions, particularly for the purpose of declaring inputs, came up as an alternative approach that came up in [RFC 0193] TOML Flakes rfcs#193.
• i looked into this in the context of figuring out if it would be possible to allow hybrid means of dependency management, e.g. managing dependencies outside flake inputs yet still allowing to override dependencies using the flakes interfaces (CLI flag --override-input, inputs attribute .follows) as well - sort of as a reverse flake-compat.
  • edit: it seems url = "file:/dev/null"; works

Context

• shifts responsibility on evaluation performance to flake authors.
• making use of the ability to use non-trivial expressions may or may not work with existing tooling presuming non-trivial expressions. so far the one example that comes to mind for me there would be flakehub's cli, which contains functionality to write to inputs to update them. (logically, one probably should not expect to set their entire inputs section using some function call while simultaneously using tools to write to inputs to update them.)
• i don't actually know if this change might be considered desirable - but maybe a draft PR people could try might help people experiment so as to get a better sense of that.
• an additional consideration here could be the stance of flake-implementing forks such as Determinate Nix and Lix, as introducing inconsistencies before the introduction of versioning and a central spec could lead to incompatibilities.
  • this might not be as much of an issue for unpublished flakes, but would be more relevant for libraries expected to be consumed by others, potentially running various nix flavors.
• also see previous discussion at What language is flake.nix written in? #4945.

[edolstra]

I don't think this is a good idea. It's very predictable that people are going to abuse this in horrible ways, such as doing IFD during the computation of the inputs. So commands like nix flake metadata or nix flake lock might not only do an unbounded (possibly non-terminating) amount of evaluation, but also do arbitrary builds.

[KiaraGrouwstra]

thanks for the feedback - i'll close then.

[Malix-Labs]

@edolstra what we are sure of is that allowing full nix for flakes input would allow for such incorrect behavior

But are we sure that the current subset of nix allowed for flakes inputs (basically just static strings) is the only safe option?

Would it not be possible to allow for more, safe expressions (still making it a subset of nix instead of full nix) without introducing such risks?

[roberth]

I'm not convinced that Flakes users need to be patronized in this way, or that IFD is particularly problematic here.
Arguably it's more controlled than the alternative, which is that users have to cobble together input modifications using scripts that perform text replacements in flake.nix, or things along those lines.

[Malix-Labs]

#15132 (comment) from @roberth :

Arguably it's more controlled than the alternative, which is that users have to cobble together input modifications using scripts that perform text replacements in flake.nix, or things along those lines.

This is a strong argument;
People really wanting to compute their flakes inputs are just doing it regardless, in an unsafer way.

[eveeifyeve]

I'm not convinced that Flakes users need to be patronized in this way, or that IFD is particularly problematic here. Arguably it's more controlled than the alternative, which is that users have to cobble together input modifications using scripts that perform text replacements in flake.nix, or things along those lines.

A good question how would stuff like the flake.lock would work in this case if it's unsafe and still will give you IFD? Like it has to import the flake.nix inside of the flake.lock to get the hashes?? IMO this is a really bad idea not for performance at evaluation time, but also for ergonomics that flakes wasn't originally designed for, so I agree with edolstra statement above.

[Malix-Labs]

Depends, is there a way to ensure that an expression evaluates finally to a static string not requiring IFD?

[Malix-Labs]

See #4945 (comment)

[Malix-Labs]

Fixes: #5663

[Malix-Labs]

Thinking about this, I think IFD must be considered impure
Otherwise anything might be possible and I don't think how --impure would have any real value or guarantees

But I think we can make the "prove that a flake input didn't rely on IFD" happen

What would it take?

[xokdvium]

In general, much more than just IFD is unsound. Doing any I/O when initially computing the inputs should probably be considered unsound, in part due to the janky 2-step "refetching" that flakes do for self attrs and the like.

That being said, there are currently quite well known workarounds to actually be able to evaluate arbitrary expressions in the inputs, so realistically whatever the intention was it doesn't really work (though that should be considered adversarial expressions and we don't really care about those).

A more interesting possibility would be to have the inputs be evaluated in a scopedImport with a minimal set of primops exposed and with no filesystem and I/O access.

[Malix-Labs]

That sounds also reasonable

At least way better than not allowing any non-trivial input

[Malix-Labs]