2026 年 33 巻 3 号 p. 1484-1509
Multi-vector retrieval models, such as ColBERT, provide strong retrieval effectiveness by representing each document with a sequence of contextualized token embeddings. This representation is expressive, but it also introduces a substantial index footprint because one vector must traditionally be retained for each document token. We introduce Token Pooling, a simple index-time compression method that clusters token embeddings within each document and replaces them with their mean representation. The method is training-free, requires no model or query-time modification, and is evaluated here on textual and visual late-interaction models. Across standard retrieval benchmarks, we show that hierarchical Token Pooling can reduce ColBERT-style index sizes by approximately 45–50% while largely preserving nDCG@10, and that it remains compatible with aggressive 2-bit PLAID quantization. We further showcase the method’s potential as a general compression interface for late-interaction retrieval by extending the same operation to visual document retrieval, where document pages are represented as sets of patch-level embeddings, and demonstrate similarly strong index size reductions with minimal degradation. Overall, Token Pooling is a competitive, model-agnostic method that can compete with learned compression methods in the settings evaluated, while serving as a useful deployment primitive for both textual and multimodal multi-vector retrieval.
