everyone keeps talking about which model we should use or how many dimensions a vector has. it feels like the entire industry is stuck on a loop of discussing embedding generation as if that is the only thing that matters for performance. but if u want to avoid a latency nightmare in ur RAG pipeline, u have to focus on vector database indexing. you can have the most perfect embeddings in the world and still have an engine that feels
painfully slow once you hit proudction scale. indexing is the real secret to making search or recommendations feel instant.
embeddings are just the ingredients; indexing is how you cook them . i was reading about this recently and it clicked that we are all way too focused on the input side of the equation. if ur index setup is inefficient, the quality of your vectors wont save your user experience. has anyone else noticed their retrieval speeds tanking despite using high-quality models?
https://dzone.com/articles/vector-database-indexing-explained