Measurement: what a directory-level vector loses on folders that mix images and text

[ali-basheer]

I ran a pre-registered study of what one vector per directory can and cannot do for retrieval, and part of it bears on L0/L1 directory abstracts.

Under four embedding models, on folders that mix images and texts, the mean of the files' embeddings loses the minority modality. A few vectors per file type recover most of it, and at three per type every folder of the 2,597-folder confirmation set fits in 8 KiB at a cost of at most 0.012 recall@5 (Sections 4 and 12). I also had a language model write a folder abstract from captions of the folder's images and the opening text of its documents, then embedded it. Added to the representatives it gained a little on whole-folder text queries. Alone it did no better than the pooled vector at finding a minority image from its description (Section 11).

The work is a preprint and has not been peer reviewed. Limits are in Section 16; for example, only Zenodo and GitHub folders were measured, and no person wrote queries.

Preprint: https://doi.org/10.5281/zenodo.23178122
Code and data: https://github.com/ali-basheer/dirvec-study