Nomic AI

Nomic, well known for their embedding models, released a multimodal embedding model based on Qwen2.5-VL. Alongside the usual, single-vector model, they also release a ColBERT-style, multi-vector model, boosting the performance even further. Their blog post reveals some of the tricks used to improve the performance during training.

A truly open-source embedding model, including the data and code to train the model. It also is a MoE, something which is rare for an embedding model. Given that embedding models are an important part of modern information retrieval systems, it makes sense to use a MoE architecture to increase the speed and throughput of the model.