Drop libidn support?

[rockdaboot]

For the release v0.22.0 we had to work around an issue with libidn after upgrading to the latest PSL data.

The PSL now contains the Balinese domain ᬩᬮᬶ.id, which contains characters that are assigned since IDNA2008 (Unicode 5.1+. But libidn is IDNA2003 only (Unicode 3.2).

$ idn ᬩᬮᬶ.id
idn: idna_to_ascii_4z: String preparation failed

The work-around is tedious and needs to be amended for every new domain that contains characters not assigned in IDNA2003.

The sane(?) options to go forward are

• drop support for libidn entirely
• call idna_to_ascii_8z() with flag IDNA_ALLOW_UNASSIGNED

The first option is maintainer-friendly, but user's may want to have the libidn option for whatever reason.
The second option may have side-effects, unpredictable for me atm.

[eli-schwartz]

The first option is maintainer-friendly, but user's may want to have the libidn option for whatever reason.
The second option may have side-effects, unpredictable for me atm.

If the "users may want to have the libidn option for whatever reason" ends up being deciding factor, then at least it probably makes sense to remove libidn from being an option for "auto". Users who are positive they want libidn with its "potential side effects the project hasn't looked into" would have to explicitly opt into it (and probably the configure would emit a warning as well).

Personally I wonder who needs libidn when libidn2 exists. In Gentoo packaging for example, we only allow libidn2 (preferred) or libicu.

[bagder]

I vote drop it.

[rockdaboot]

then at least it probably makes sense to remove libidn from being an option for "auto"

Yes, we should start with this.

Personally I wonder who needs libidn when libidn2 exists

Iirc, there are the stringprep functions (RFC 3454, stringprep.h) that no longer exist in libidn2. IDNA2008 favors PRECIS (RFC 8264) which hasn't been implemented. Not sure if anyone ever asked for PRECIS.

On my Debian testing system, the packages that depend on libidn do not depend on libpsl.

[rockdaboot]

I vote drop it.

Debian's curl packaging seems still to depend on libidn-dev, so it looks like curl itself offers to build with libidn. I just tells me, curl builds could end up with pulling in libidn directly and libidn2 indirectly (via libpsl). Interested in your thoughts here.

[bagder]

curl does not use any IDN features in libpsl - it converts IDN itself (using libidn2 on most systems).

curl itself has not used libidn for many years