java.util.ImmutableCollections$ListN can contain null values

[mjrosenb]

Describe the bug
In older java versions, List::of was the only way to create a java.util.ImmutableCollections$ListN object, but now Stream::toList is also capable of creating them, and it does not have the limitation of not containing nulls

To Reproduce
There are two slightly different issues. It should be possible to fix both at the same time.
In this case, immutableWithNulls only has null values in it, so kryo successfully serializes it, but then can not deserialize it because it ends up calling List.of(null, null)

  public void testImmutableDeserialization() throws Exception {
    final Kryo kryo = new Kryo();
    kryo.setRegistrationRequired(false);
    final Output output = new Output(64);
    final List<Integer> immutableWithNulls = Stream.of(null,1, null).filter(x -> x == null || x > 10).toList();

    System.out.println(immutableWithNulls.getClass());
    kryo.writeClassAndObject(output, immutableWithNulls);
    try(Input input = new Input(output.getBuffer())) {
      final Object copy = kryo.readClassAndObject(input);
      assert immutableWithNulls.equals(copy);
    }

In this case, immutableWithNulls has both null and non-null values, so kryo catches that there are null values in a supposedly nonnull container during serialization

  }
  @Test
  public void testImmutableSerialization() throws Exception {
    final Kryo kryo = new Kryo();
    kryo.setRegistrationRequired(false);
    final Output output = new Output(64);
    final List<Integer> immutableWithNulls = Stream.of(null,1, null).toList();
    kryo.writeClassAndObject(output, immutableWithNulls);
    try(Input input = new Input(output.getBuffer())) {
      final Object copy = kryo.readClassAndObject(input);
      assert immutableWithNulls.equals(copy);
    }

Environment:

• OS: Ubuntu running in WSL
• JDK 21 (but Stream::toList was added in 16. I have no clue if that is when it started returning a ListN)
• Kryo 5.5

Additional context
Add any other context about the problem here.

[theigl]

This will be a bit difficult to fix. We either have to write an additional flag if the list contains null values, which adds some overhead and is not backwards compatible, or always use Stream.toList() for reading and copying such collections.

[konlue]

@theigl I was looking at this with the intent of submitting a PR, and I think one of the two options you outlined hits a concrete blocker:

pom.xml pins <javac.target>1.8</javac.target> (project is 5.6.3-SNAPSHOT), but Stream.toList() is JDK 16+. So "always use Stream.toList() for reading and copying such collections" cannot be referenced from Kryo 5.x source without either a reflective call or a baseline bump — neither of which I assume is in scope for 5.6.x. Is that your reading too?

If so, only the flag-based option remains, with the two costs you named (per-write overhead, plus a wire-format change that breaks cross-version reads). A third variant might avoid both:

• keep the element stream byte-identical, so data written by older Kryo still reads;
• let the read side decide: after the element array is built, scan it once (we already have the array in hand) and reconstruct with List.of(...) when there is no null, or Collections.unmodifiableList(new ArrayList<>(Arrays.asList(array))) when there is one. Java 8 compatible, and the extra pass is over an array we just allocated.

The cost is that a null-containing instance no longer comes back as java.util.ImmutableCollections$ListN. Given List.of cannot hold nulls, I don't think class fidelity is reachable there at all — but that is exactly the property worth your call.

One thing I want to confirm before writing code: the second reproducer in the description fails during serialization, not deserialization, so the write path may have its own null rejection that a read-only change would not cover. I'll pin down where each of the two cases actually throws and report back.

So the question I need answered is: do you want to protect exact class identity (accept a wire-format flag), or wire compatibility (accept a different concrete class when nulls are present)? I'll write it either way with tests in ImmutableCollectionsSerializersTest.