XNews

Databricks Benchmarks Lakebase Search on 1M–4M Passages, Reports 11–23 ms p50 Latency

Databricks Benchmarks Lakebase Search on 1M–4M Passages, Reports 11–23 ms p50 Latency

Databricks benchmarked Lakebase Search on 1M to 4M passages to assess vector search performance as the index grows. Cache size relative to the vector index mattered more than row count; when the cache roughly matched the index, 95–98% of page reads avoided storage and warm vector search p50 latency stayed between 11 and 23 ms. Databricks stated that sizing the cache alongside the vector index helps maintain low-latency search as datasets grow.

Event version 1.

Key insights

Cache size outweighed row count

Cache size relative to the vector index mattered more than row count in the Lakebase Search benchmark. The benchmark covered datasets of 1M to 4M passages.

Most page reads avoided storage

When the cache roughly matched the index, 95–98% of page reads avoided storage. Warm vector search p50 latency stayed between 11 and 23 ms.

Cache sizing guidance for growing datasets

Databricks stated that sizing the cache alongside the vector index helps maintain low-latency search as datasets grow.

Source posts