In a RAG system, every document chunk is represented as an embedding: a vector that places semantically similar texts close to each other.
These vectors are usually stored as float32, which means 4 bytes per dimension.
For retrieval to be fast, those vectors usually need to be available in memory.
Reading them from disk for every query would make similarity search much slower, so the practical constraint is often RAM, not just storage on disk.
That detail matters at scale. The storage cost is roughly:
number of embeddings x embedding dimensions x bytes per value
For one million embeddings with 1536 dimensions stored in float32, that is around 6.1 GB before metadata, vector index overhead, replicas, or serving infrastructure.
If the collection grows, or if several indexes need to stay online, embedding storage quickly becomes a constraint.
We studied how far this storage can be reduced before retrieval quality starts to degrade too much.
Two compression levers
There are two direct ways to make the vectors smaller.
The first one is quantization: keep the same number of dimensions, but store each value with fewer bits.
For example, float16 and bfloat16 data types reduce storage by 2x with an almost imperceptible quality
loss, but float8 variants are more interesting: they reduce storage by 4x and stay very close to the float32 baseline quality.
Integer-based quantization, like int8, also reduces storage by 4x, but performs worse in our experiments and
needs a calibration dataset plus an extra calibration step before converting the embeddings, whereas float8
can be cast from float32, which makes it simpler to use.
Finally, binary quantization is the aggressive option. It gives a 32x reduction, but the quality drop can be much larger.
Retrieval quality under different quantization formats.
The second lever is dimensionality reduction: store fewer dimensions while trying to preserve as much useful semantic information as possible.
For this, we tested PCA, Kernel PCA, UMAP, autoencoders, and random projections, and PCA emerged as the best option because
it preserves retrieval quality well under compression and is easy to train and apply to embeddings.
Combining quantization and dimensionality reduction
The strongest results come from combining both levers.
float32 baseline
Compressed
embeddings x dimensions x bytes per valueSmaller
PCA alone can reduce storage, but for comparable compression ratios it usually hurts more than a robust low-precision format such as float8.
The useful pattern is to first use a format that keeps the signal stable, and then reduce dimensions moderately.
Three important observations:
Larger or higher-dimensional embedding models tend to degrade more gracefully under compression.
float8 combined with PCA defines a strong part of the storage-quality frontier between 4x and 32x compression.
A stronger model after compression can outperform a smaller model at a comparable storage size.
↑ Relative performance vs float32 (%)
binaryint8float8 e5m2float8 e4m3bfloat16float16
The relevant trade-off is not one technique in isolation, but the storage-quality frontier.
Reading the trade-off
The score in the chart comes from the MTEB Retrieval benchmark. We use nDCG@10, a metric that rewards
returning relevant documents near the top of the result list. A score of 1 would be a perfect ranking;
here it is useful as a way to compare configurations under the same evaluation setup.
Each point is one configuration: embedding model, numeric format, and PCA level. The x-axis is storage,
and the y-axis is retrieval quality. The lines show the Pareto frontier for each model.
For example, bge-large-en-v1.5 with float8 and 25% PCA uses about 388 MB to store 125k embeddings and
reaches a score around 0.598. bge-small-en-v1.5 with float8 and 90% PCA uses about 524 MB and reaches
around 0.589. In that region, the larger model is both smaller and better after compression.
The practical decision process is straightforward: decide the memory budget, look at the configurations that fit under it,
and choose the one with the best retrieval score.
For moderate compression, float8 is a strong default. For stricter memory budgets, float8 plus PCA is an interesting option.
The broader lesson is that model choice and compression strategy should be selected together: when storage is limited, a
larger model with the right compression can be a better point on the frontier than a smaller model with less compression.
