Score floors per encoder family, not one number for all CLIPs

docs/user-guide/encoders.md

@@ -10,7 +10,7 @@ Three encoders are bundled with PhotoMapAI. They have different strengths, and t
 
 A larger, better-trained successor to the original CLIP. Apple released the `DFN-2B` weights in 2023, retrained on a heavily filtered 2-billion image-caption dataset. It scores meaningfully higher than legacy CLIP on virtually every published benchmark and behaves robustly across messy real-world photo collections.
 
-- **Strengths.** Best general-purpose match for typical photo libraries — family snapshots, vacations, mixed media. Cosine similarities are in a familiar range (matching pairs around 0.20–0.35), so a threshold of 0.2 catches most legitimate hits without flooding the results with false positives. Threshold semantics are predictable and don't need per-album tuning.
+- **Strengths.** Best general-purpose match for typical photo libraries — family snapshots, vacations, mixed media. Its similarity scale sits about 0.1 below legacy CLIP's — matching pairs land around 0.15–0.26 against a median near zero — so the default threshold is 0.1. Measured on a 38,000-image library, that returns the whole match band for ordinary queries while a 0.2 threshold returned nothing at all for most of them.
 - **Weaknesses.** Larger model than legacy CLIP — ~600 MB to download and slower per image to index. First-time indexing of a big album takes noticeably longer than the legacy CLIP option.
 - **Pick this if** you don't have a strong reason to pick something else.
 
@@ -26,7 +26,7 @@ Google's 2024–25 update to SigLIP, trained with a sigmoid loss instead of CLIP
 
 The original 2021 CLIP — the model PhotoMapAI used for everything before the encoder layer was added. It's the smallest of the three (~150 MB, fastest to index) and uses a familiar contrastive cosine similarity.
 
-- **Strengths.** Fastest indexing, smallest disk footprint, well-understood threshold behavior.
+- **Strengths.** Fastest indexing, smallest disk footprint, well-understood threshold behavior. Its scores run higher than OpenCLIP's — matching pairs around 0.24–0.35, and a median near 0.17 even for an unrelated query — so its default threshold is the traditional 0.2.
 - **Weaknesses.** Substantially weaker than the newer alternatives across published benchmarks (e.g. ~63% ImageNet zero-shot vs. 82% for OpenCLIP-DFN). Recall on photo-style content is noticeably worse, and false positives are more frequent.
 - **Pick this if** you're constrained on disk or time, or you want to keep a legacy album working without re-indexing it. Albums created before the encoder layer existed default to this, and re-indexing isn't required just because a newer option is available.
 

docs/user-guide/search.md

@@ -33,7 +33,7 @@ To search PhotoMapAI by image similarity:
 
 The screenshot below shows the results of an image search on a photo of a generic mountain found in Google Images. The match score, a value ranging from 0.0 (no match) to 1.0 (perfect match), appears at the top left. The seek slider at the top lets you select images with particular score ranges. A bit counterintuitively, images with the strongest matches (highest scores) appear earlier to the left, and those with the weakest scores appear later.
 
-The exact distribution of scores depends on which [encoder](encoders.md) the album was indexed with. Classic CLIP-style encoders (OpenAI CLIP, OpenCLIP) put strong matches around 0.20–0.40; SigLIP produces calibrated probabilities that put strong matches near 0.5+ but compresses everything else toward zero, so its useful threshold is much lower. The defaults are set per-encoder, but you can override them per album in the search dialog — see [Tuning Search Per-Album](#tuning-search-per-album) below.
+The exact distribution of scores depends on which [encoder](encoders.md) the album was indexed with, and the three do not share a scale. OpenAI CLIP puts strong matches around 0.24–0.35 against a median near 0.17; OpenCLIP (the recommended default) runs about 0.1 lower on both counts, with strong matches around 0.15–0.26 against a *negative* median; SigLIP produces calibrated probabilities that compress everything but the strongest matches toward zero. The defaults are set per-encoder to match, but you can override them per album in the search dialog — see [Tuning Search Per-Album](#tuning-search-per-album) below.
 
 <img src="../../img/photomap_search_2.png" width="480" alt="Image Search Result" class="img-hover-zoom">
 
@@ -53,7 +53,7 @@ The negative weight is subtracted from the combined score, so it acts as a penal
 
 The search dialog has three tuning controls below the prompt area, all stored per-album in the album's configuration. Their values are loaded automatically when you switch albums and persist back to the album's config when you change them.
 
-- **Min. score.** Results below this similarity score are filtered out. Defaults are encoder-aware: 0.2 for CLIP and OpenCLIP albums, 0.005 for SigLIP albums. SigLIP's calibrated probability distribution is much more compressed than CLIP cosines, so its sensible threshold is roughly 40× lower. If you're seeing zero hits where you expect matches, try lowering the threshold first; if you're seeing too many weak matches, raise it.
+- **Min. score.** Results below this similarity score are filtered out. Defaults are encoder-aware: 0.1 for OpenCLIP albums, 0.2 for OpenAI CLIP albums, 0.005 for SigLIP albums. Those are not arbitrary — each sits just under the band that encoder puts real matches in, and an OpenCLIP album judged at OpenAI CLIP's 0.2 answers most searches with nothing at all. If you're seeing zero hits where you expect matches, try lowering the threshold first; if you're seeing too many weak matches, raise it.
 - **Max. results.** Caps how many top-scoring results are returned. Default is 100. Increase this if you want to see the long tail of borderline matches; decrease it if you only want to see the strongest hits.
 - **Query optimization (SigLIP only).** When enabled, SigLIP wraps each text query in five modality-spanning templates (`"a photo of …"`, `"a drawing of …"`, `"an illustration of …"`, `"a painting of …"`, and the bare query), encodes them all, and averages the resulting embeddings. The intent is to make bare-noun queries (like `"woman"`) match more strongly and to avoid systematically penalizing non-photo content (drawings, illustrations) in mixed-media albums. Effects vary by album — for some it improves recall on short queries; for others it lowers per-image cosines just enough to push everything below the SigLIP calibration cliff. Try both settings on your library; the toggle is greyed out for non-SigLIP albums where it has no effect.
 

photomap/backend/config.py

@@ -15,7 +15,12 @@
 from platformdirs import user_config_dir, user_data_dir
 from pydantic import BaseModel, Field, field_validator, model_validator
 
-from .encoders import LEGACY_ENCODER_SPEC, default_encoder_spec
+from .encoders import (
+    LEGACY_CLIP_MIN_SEARCH_SCORE,
+    LEGACY_ENCODER_SPEC,
+    default_encoder_spec,
+    default_min_search_score,
+)
 from .util import atomic_write_text
 
 logger = logging.getLogger(__name__)
@@ -99,8 +104,8 @@ class Album(BaseModel):
         ),
     )
     # Per-album search controls. min_search_score defaults to None so a
-    # validator can resolve it from the encoder (0.005 for SigLIP's compressed
-    # cosine band, 0.2 for CLIP-style backends) at construction time.
+    # validator can resolve it from the encoder at construction time — the
+    # backends do not share a score scale, see ``default_min_search_score``.
     min_search_score: float | None = Field(
         default=None,
         description="Minimum similarity score below which results are filtered out.",
@@ -162,9 +167,7 @@ def _derive_board_album_fields(cls, data: Any) -> Any:
     @model_validator(mode="after")
     def _resolve_min_search_score(self) -> "Album":
         if self.min_search_score is None:
-            self.min_search_score = (
-                0.005 if self.encoder_spec.startswith("siglip:") else 0.2
-            )
+            self.min_search_score = default_min_search_score(self.encoder_spec)
         return self
 
     @model_validator(mode="after")
@@ -256,10 +259,64 @@ def from_dict(cls, key: str, data: dict[str, Any]) -> "Album":
         )
 
 
+# Bumped when a load-time migration is added below. 1.1.0 re-resolves the
+# score floor of albums that never chose one — see ``_migrate_score_floors``.
+CONFIG_VERSION = "1.1.0"
+
+
+def _version_tuple(version: str) -> tuple[int, ...]:
+    """``"1.10.2"`` -> ``(1, 10, 2)``, for ordering config versions.
+
+    A version this build cannot parse sorts as newer than anything it knows,
+    so an unreadable stamp is never mistaken for an un-migrated old file.
+    """
+    try:
+        return tuple(int(part) for part in version.split("."))
+    except ValueError:
+        return (999,)
+
+
+def _migrate_score_floors(albums: dict[str, "Album"]) -> bool:
+    """Re-resolve score floors left at the old blanket CLIP default.
+
+    Before :func:`default_min_search_score` learned that OpenCLIP scores a
+    tenth of a point below OpenAI CLIP, every non-SigLIP album was given
+    ``0.2`` at creation — a floor above the entire match band of the encoder
+    PhotoMapAI recommends, so those albums answer most searches with nothing
+    at all. The value is stored per album, so fixing the default alone would
+    only help albums created after the upgrade.
+
+    Only the exact machine-chosen 0.2 is touched, and only where the album's
+    encoder now resolves to something else; a floor the user typed stays put.
+    Runs only for a config written before this build (see ``CONFIG_VERSION``),
+    so 0.2 remains a value the user can choose afterwards.
+
+    Returns whether anything changed, for the caller's logging.
+    """
+    changed = False
+    for key, album in albums.items():
+        resolved = default_min_search_score(album.encoder_spec)
+        if album.min_search_score == LEGACY_CLIP_MIN_SEARCH_SCORE != resolved:
+            album.min_search_score = resolved
+            changed = True
+            logger.info(
+                "Album %r: search score floor %.3f -> %.3f, the measured "
+                "default for %s (searches at the old floor returned almost "
+                "nothing). Adjust it in the search dialog if you preferred it.",
+                key,
+                LEGACY_CLIP_MIN_SEARCH_SCORE,
+                resolved,
+                album.encoder_spec,
+            )
+    return changed
+
+
 class Config(BaseModel):
     """Main configuration model."""
 
-    config_version: str = Field("1.0.0", description="Configuration format version")
+    config_version: str = Field(
+        CONFIG_VERSION, description="Configuration format version"
+    )
     albums: dict[str, Album] = Field(
         default_factory=dict, description="Album configurations"
     )
@@ -437,7 +494,7 @@ def load_config(self) -> Config:
             if self._config is None:
                 if not self.config_path.exists():
                     self._config = Config(
-                        config_version="1.0.0",
+                        config_version=CONFIG_VERSION,
                         albums={},
                         locationiq_api_key=None,
                     )
@@ -456,8 +513,22 @@ def load_config(self) -> Config:
                             extra["encoder_idle_timeout_seconds"] = config_data[
                                 "encoder_idle_timeout_seconds"
                             ]
+                        # Migrations run once, against a config older than
+                        # this build. Re-running them on every load would make
+                        # the migrated-away value permanently unsettable — a
+                        # user typing 0.2 back into an OpenCLIP album would
+                        # watch the next read undo it.
+                        stored_version = config_data.get("config_version", "1.0.0")
+                        if _version_tuple(stored_version) < _version_tuple(
+                            CONFIG_VERSION
+                        ):
+                            _migrate_score_floors(albums)
+                            stored_version = CONFIG_VERSION
+                        # A config stamped *newer* than this build keeps its
+                        # stamp: rewriting it as 1.1.0 would tell the build
+                        # that wrote it that its own migrations had run.
                         self._config = Config(
-                            config_version=config_data.get("config_version", "1.0.0"),
+                            config_version=stored_version,
                             albums=albums,
                             locationiq_api_key=config_data.get("locationiq_api_key"),
                             invokeai_url=config_data.get("invokeai_url"),
@@ -466,6 +537,13 @@ def load_config(self) -> Config:
                             invokeai_board_id=config_data.get("invokeai_board_id"),
                             **extra,
                         )
+                        # Deliberately not saved here. Loading a config is a
+                        # read, and a rewrite drops comments and any key this
+                        # build does not know — an unprompted one, on every
+                        # user's first start after upgrading, is not a trade
+                        # worth making. The new stamp rides along with the
+                        # next save the user actually causes, and until then
+                        # the migration is simply re-applied in memory.
 
                     except Exception as e:
                         raise RuntimeError(

photomap/backend/embeddings.py

@@ -39,6 +39,7 @@
     ImageTextEncoder,
     build_encoder,
     capture_download_progress,
+    default_min_search_score,
     get_cached_encoder,
 )
 from .media_types import IMAGE_EXTENSIONS, INDEXABLE_EXTENSIONS, is_video
@@ -1864,7 +1865,7 @@ def search_images_by_text_and_image(
         positive_weight: float = 0.5,
         negative_weight: float = 0.5,
         top_k: int = 5,
-        minimum_score: float = 0.2,
+        minimum_score: float | None = None,
         use_query_optimization: bool | None = None,
     ) -> tuple[list[int], list[float]]:
         """
@@ -1878,14 +1879,20 @@ def search_images_by_text_and_image(
             positive_weight (float): Weight for positive text embedding.
             negative_weight (float): Weight for negative text embedding (should be positive; will be subtracted).
             top_k (int): Number of top results.
-            minimum_score (float): Minimum similarity score.
+            minimum_score (float or None): Minimum similarity score. None
+                resolves the encoder's own default floor — the three backends
+                do not share a score scale, so there is no one number that
+                filters honestly for all of them.
             use_query_optimization (bool or None): Per-album SigLIP toggle. When
                 set, controls prompt-template ensembling for SigLIP encoders.
                 Ignored by other backends. ``None`` keeps the encoder's current
                 setting (the module-level default, typically).
         Returns:
             tuple: (indexes, similarities)
         """
+        if minimum_score is None:
+            minimum_score = default_min_search_score(self.encoder_spec)
+
         data = self.open_cached_embeddings(self.embeddings_path)
         embeddings = data["embeddings"]
         filenames = data["filenames"]

photomap/backend/encoders.py

@@ -36,6 +36,54 @@
 # misfire).
 DEFAULT_ENCODER_SPEC = "open-clip:ViT-L-14/dfn2b_s39b"
 
+# The score floor a search applies when the album has not set one. It is
+# per-encoder because the three backends do not share a scale — the numbers
+# below were measured on the same 400 images with the same eight queries, and
+# again on a 38,000-image photo library:
+#
+#   encoder                        mean top-1   median   strong matches
+#   openai-clip:ViT-B/32              0.267      0.17     0.24 - 0.30
+#   open-clip:ViT-L-14/dfn2b_s39b     0.151     -0.07     0.15 - 0.26
+#   siglip (calibrated probability)   0.015      0.00     compressed to ~0
+#
+# OpenCLIP's band sits roughly 0.1 below OpenAI CLIP's, which is why the old
+# blanket 0.2 for everything CLIP-shaped returned *nothing* on the encoder
+# PhotoMapAI recommends by default: on the 38k library, 0.2 gave zero results
+# for five of eight ordinary queries and fewer than five for two more. 0.1
+# clears the noise floor (that library's median is -0.07) while keeping the
+# whole match band, and costs at most ~8% of the album on a very broad query.
+#
+# The same 0.1 would be far too low for OpenAI CLIP, whose *median* image
+# scores 0.17 against an arbitrary query — it would return the entire album,
+# ranked. Hence a table rather than one number.
+LEGACY_CLIP_MIN_SEARCH_SCORE = 0.2
+
+_MIN_SEARCH_SCORE_BY_BACKEND: dict[str, float] = {
+    "siglip": 0.005,
+    "openai-clip": 0.2,
+    "open-clip": 0.1,
+}
+
+# Anything unrecognized is scored like OpenCLIP: an unknown backend is far
+# likelier to be another modern CLIP variant (they cluster near this band)
+# than a copy of OpenAI's original, and erring low shows weak matches rather
+# than hiding real ones.
+_DEFAULT_MIN_SEARCH_SCORE = 0.1
+
+
+def default_min_search_score(encoder_spec: str) -> float:
+    """The score floor to apply when an album has not chosen one.
+
+    Callers that need to know whether a *change* of encoder warrants
+    re-resolving a stored floor should compare this across the two specs
+    rather than comparing the specs themselves: swapping one OpenCLIP model
+    for another shares a scale and must not discard a hand-tuned value.
+    """
+    backend = encoder_spec.split(":", 1)[0]
+    return _MIN_SEARCH_SCORE_BY_BACKEND.get(backend, _DEFAULT_MIN_SEARCH_SCORE)
+
+
+
 # Encoder assumed when a legacy ``.npz`` cache or pre-swap-layer YAML album
 # omits the ``model_id`` / ``encoder_spec`` field. Before the encoder swap
 # layer existed, legacy CLIP was the only option, so any cache that predates
@@ -133,8 +181,10 @@ def calibrate_similarity(self, cosines: np.ndarray) -> np.ndarray:
 
         Default implementation is the identity, which is appropriate for CLIP-style
         contrastive encoders whose cosine scores are already in a usable range.
-        SigLIP overrides this to apply the learned sigmoid calibration so a single
-        threshold (e.g. 0.2) produces sane recall across encoder choices.
+        SigLIP overrides this to apply the learned sigmoid calibration, which
+        pulls its scores onto a bounded scale — not onto CLIP's. No calibration
+        makes one threshold serve every encoder, which is why the default floor
+        is a per-backend table (:func:`default_min_search_score`).
         """
         return cosines
 
@@ -400,7 +450,10 @@ def calibrate_similarity(self, cosines: np.ndarray) -> np.ndarray:
         calibration, a CLIP-tuned threshold like 0.2 filters out almost every
         true match. The model's ``logit_scale`` and ``logit_bias`` recover
         per-pair match probabilities via ``sigmoid(cos * exp(scale) + bias)``,
-        which restores comparable threshold semantics across encoders.
+        which makes the score readable as a probability. It does *not* make
+        the threshold comparable to CLIP's: calibrated SigLIP probabilities
+        are compressed toward zero, hence its own entry (0.005) in
+        :func:`default_min_search_score`.
         """
         if self._logit_scale is None:
             return cosines