Raise ValidationError from the config test validators (#1456)

CI started failing on `v1.x.x` yesterday with three unrelated-looking
config test failures. They come from the `frequenz-quantities` 1.0.2
release, which raised its `marshmallow` floor to 4
(frequenz-floss/frequenz-quantities-python#107) and thereby dragged
`marshmallow` 4 into our environment for the first time. Marshmallow 4
dropped support for validators that return `False` instead of raising,
which is what the config test dataclasses were doing, so the validation
silently stopped happening.

The fix is test-only, but there is a release notes entry because the
same silent acceptance hits anyone who wrote boolean validators for
their own config dataclasses.

Heads-up for whoever reviews: the docs job on this PR will stay red, for
a second and unrelated problem from the same `frequenz-quantities` 1.0.2
release. That one is a docstring in `frequenz-quantities` using a
relative mkdocstrings cross-reference, which misresolves whenever it is
rendered under a path other than the one it was written in, as happens
here with `BaseConfigSchema` inheriting `QuantitySchema.TYPE_MAPPING`.
It cannot be fixed on this side, only worked around by dropping
documentation, and frequenz-floss/frequenz-quantities-python#170 already
fixes it upstream. #1455 is blocked on the same thing.

Author: llucax

RELEASE_NOTES.md

@@ -6,6 +6,8 @@
 
 ## Upgrading
 
+- Custom validators passed through a configuration dataclass field's `metadata` (as used by `load_config()` and `ConfigManager.new_receiver()`) must now raise a `marshmallow.ValidationError` to reject a value. `marshmallow` 4 removed support for validators that signal failure by returning `False`, so such validators are now ignored and the invalid value is accepted silently. This affects you as soon as `marshmallow` 4 is installed, which `frequenz-quantities` 1.0.2 and later require.
+
 <!-- Here goes notes on how to upgrade from previous versions, including deprecations and what they should be replaced with -->
 
 ## New Features

tests/config/test_manager.py

@@ -20,11 +20,24 @@
 from frequenz.sdk.config._manager import _get_key
 
 
+def _validate_name(value: str) -> None:
+    """Validate that a name starts with `test`.
+
+    Args:
+        value: The name to validate.
+
+    Raises:
+        marshmallow.ValidationError: If the name doesn't start with `test`.
+    """
+    if not value.startswith("test"):
+        raise marshmallow.ValidationError("Invalid value.")
+
+
 @dataclass
 class SimpleConfig:
     """A simple configuration class for testing."""
 
-    name: str = dataclasses.field(metadata={"validate": lambda s: s.startswith("test")})
+    name: str = dataclasses.field(metadata={"validate": _validate_name})
     value: int
 
 

tests/config/test_util.py

@@ -14,19 +14,32 @@
 from frequenz.sdk.config._util import load_config
 
 
+def _validate_name(value: str) -> None:
+    """Validate that a name starts with `test`.
+
+    Args:
+        value: The name to validate.
+
+    Raises:
+        marshmallow.ValidationError: If the name doesn't start with `test`.
+    """
+    if not value.startswith("test"):
+        raise marshmallow.ValidationError("Invalid value.")
+
+
 @dataclasses.dataclass
 class SimpleConfig:
     """A simple configuration class for testing."""
 
-    name: str = dataclasses.field(metadata={"validate": lambda s: s.startswith("test")})
+    name: str = dataclasses.field(metadata={"validate": _validate_name})
     value: int
 
 
 @marshmallow_dataclass.dataclass
 class MmSimpleConfig:
     """A simple marshmallow_dataclass configuration class for testing."""
 
-    name: str = dataclasses.field(metadata={"validate": lambda s: s.startswith("test")})
+    name: str = dataclasses.field(metadata={"validate": _validate_name})
     value: int