Update indexing and quantization docs by jzheng106 · Pull Request #344 · lancedb/docs

jzheng106 · 2026-08-26T18:31:13Z

Substantial changes to indexing documentation, as part of Jason's internship project. Reorganized and rewrote "overview", "vector index", and "quantization" tabs in https://docs.lancedb.com/indexing . Added descriptions of algorithms and improved API references, usage examples, and overall readability/flow. Msg me on slack with questions

wkalt · 2026-08-26T21:58:11Z

-| `SQ` (Scalar Quantization) | Use when you need faster indexing or when vector dimensions have consistent value ranges. | Quantizes each dimension independently. Simpler than PQ but typically provides less compression. |
 | `RQ` (RabitQ Quantization) | Use when you need maximum compression or have specific per-dimension requirements. | Per-dimension quantization using a RabitQ codebook. Provides fine-grained control over compression per dimension. For `IVF_RQ`, vector dimensions must be divisible by `8`. |
+| `SQ` (Scalar Quantization) | Use when you need faster indexing or when vector dimensions have consistent value ranges. | Quantizes each dimension independently. Simpler than PQ but typically provides less compression. |
 | `None/Flat` | Use for binary vectors (with `hamming` distance) or when you need maximum recall and have sufficient storage. | No quantization—stores raw vectors. Provides the highest accuracy but requires more storage and memory. |


I'm not sure if changing this was in scope, but I find these descriptions pretty weak. They would be better if they gave some kind of comparison between each other. For example if we are saying these are all useful because they fall at different points on the compression/speed/accuracy plane, we should describe how they actually relate. Why would I choose PQ over RQ, or SQ over PQ, is not really answered by this table.

Another way to do this is something like,

If you need X [petabyte scale low-latency search on static dataset], choose Y because x, y, z
If you need A, choose B because a, b, c

etc. I have no idea what the real values to plug in here are though.

"I think these descriptions are pretty weak" I agree. Unfortunately, until recently many technical descriptions were written similar to this, and I didn't really know how to approach them (I also didn't have a clearly defined scope for this documentation task) so I just largely took my best guess. I agree that there could be further improvements. Maybe I'll make a note of this and add it to a future commit?

I think the Use Case column makes a stab at this doesn't it? I don't know if it reflects reality though. I read it as...

PQ is the baseline.
RQ achieves better compression than PQ with minimal loss to accuracy.
SQ sacrifices compression but can speed up index builds
Flat discards compression entirely in favor of accuracy

I think the use cases don't do it for me because they don't compare all the options on the same plane. How do I compare (fast indexing, consistent value ranges) to (max compression, particular dimension requirements). Assuming I am reading the right column.

wkalt · 2026-08-26T21:58:48Z

@@ -0,0 +1,172 @@
+9f362963 docs/user-guides/indexing/index.mdx (erik-wang-lancedb 2025-11-30 21:12:29 -0800   1) ---


is this included intentionally?

no. oops. you're right to delete the whole file

westonpace

Thanks for working on this. I think this PR improves things but we can still improve further, especially for a naive user.

We've also removed a lot of information. I think that can be good to focus the guide. I'll try and do a second pass soon to see if we removed anything that might have been important.

westonpace · 2026-08-28T13:00:36Z

-| `SQ` (Scalar Quantization) | Use when you need faster indexing or when vector dimensions have consistent value ranges. | Quantizes each dimension independently. Simpler than PQ but typically provides less compression. |
 | `RQ` (RabitQ Quantization) | Use when you need maximum compression or have specific per-dimension requirements. | Per-dimension quantization using a RabitQ codebook. Provides fine-grained control over compression per dimension. For `IVF_RQ`, vector dimensions must be divisible by `8`. |
+| `SQ` (Scalar Quantization) | Use when you need faster indexing or when vector dimensions have consistent value ranges. | Quantizes each dimension independently. Simpler than PQ but typically provides less compression. |
 | `None/Flat` | Use for binary vectors (with `hamming` distance) or when you need maximum recall and have sufficient storage. | No quantization—stores raw vectors. Provides the highest accuracy but requires more storage and memory. |


I think the Use Case column makes a stab at this doesn't it? I don't know if it reflects reality though. I read it as...

PQ is the baseline.
RQ achieves better compression than PQ with minimal loss to accuracy.
SQ sacrifices compression but can speed up index builds
Flat discards compression entirely in favor of accuracy

westonpace · 2026-08-28T13:45:21Z

+**Using `bypass_vector_index`:**

-Compare ANN results against a flat-scan ground truth to compute recall@k. This is the standard way to pick `nprobes` for your workload.
+Use `bypass_vector_index` to compute an exact **kNN** result. Note that exact queries may be prohibitively slow on production scales.


Do we discuss recall@k anywhere anymore?

I think "how do I know if my vector index is properly configured" or "how do I know how well my vector index is performing" is a common question we should be answering on this page. I'm not sure the old page answered it well either but I think we should strive to include that somewhere (and calculating recall@k is part of the answer)

westonpace · 2026-08-28T13:48:08Z

-</Warning>
+| Parameter | Description |
+| :--- | :--- |
+| `m` | The number of neighbors to select for each vector in the HNSW graph. |


Do we have any guidance on this parameter? What is the default?

Co-authored-by: Weston Pace <weston.pace@gmail.com>

jzheng106 added 30 commits August 3, 2026 11:54

small grammatical, typo, and flow fixes

d650374

light changes

220338a

more small changes

119b857

half done with ivf implementation

bc4d53a

quantization light pass

a7b1460

vector index light pass

cd7899c

whatever

86d2e2d

test

65dc392

reorganized stuff

9a57135

stuff

70c9c4c

d

178949a

lots of changes to vector-index.mdx, finished ivfpq section

c19f0c6

tear down hnsw

c437371

tear down hnsw again

55328b9

gonna rewrite hnsw from scratch

e0157bb

more changes

1510267

vector-index.mdx introduction part

3bd1b53

tear down pq

0156af6

rewrite pq part 1

75fead3

aa

2e67d43

a

fb4a339

almost finished pq

f0adbee

finished pq quantization :)

b0fdfc0

fixing examples sections

d3f7d66

high level organization

de1b3d3

examples and stuff

8a12b55

a

366659a

finished rabitq

7ea0e9e

finished sq quantization

492c6f5

a

29b2c86

jzheng106 requested review from BubbleCal, Xuanwo, westonpace and wjones127 August 26, 2026 18:31

wkalt reviewed Aug 26, 2026

View reviewed changes

westonpace requested changes Aug 28, 2026

View reviewed changes

Apply batched suggestions from code review

582f63c

Co-authored-by: Weston Pace <weston.pace@gmail.com>

mintlify Bot deployed to staging - docs August 28, 2026 17:21 View deployment

jzheng106 and others added 2 commits August 28, 2026 14:40

Apply suggestion from @westonpace

f1264c9

Co-authored-by: Weston Pace <weston.pace@gmail.com>

Apply suggestion from @westonpace

24082e5

Co-authored-by: Weston Pace <weston.pace@gmail.com>

mintlify Bot deployed to staging - docs August 28, 2026 21:41 View deployment

mintlify Bot deployed to staging - docs August 28, 2026 21:42 View deployment

Apply suggestion from @westonpace

2487559

Co-authored-by: Weston Pace <weston.pace@gmail.com>

mintlify Bot deployed to staging - docs August 28, 2026 21:43 View deployment

jzheng106 and others added 2 commits August 28, 2026 14:43

Apply suggestion from @westonpace

4de1f8d

Co-authored-by: Weston Pace <weston.pace@gmail.com>

Apply suggestion from @westonpace

acb019f

Co-authored-by: Weston Pace <weston.pace@gmail.com>

mintlify Bot deployed to staging - docs August 28, 2026 21:44 View deployment

mintlify Bot deployed to staging - docs August 28, 2026 21:45 View deployment

jzheng106 added 2 commits August 31, 2026 11:41

a

7c5e609

a

40070c6

mintlify Bot deployed to staging - docs August 31, 2026 19:03 View deployment

jzheng106 added 3 commits August 31, 2026 12:15

a

c11d2d5

a

1d1ee81

a

273c87f

mintlify Bot deployed to staging - docs August 31, 2026 20:36 View deployment

hnsw stuff

92b43ba

mintlify Bot deployed to staging - docs August 31, 2026 21:54 View deployment

jzheng106 requested review from westonpace and wkalt August 31, 2026 21:56

		@@ -0,0 +1,172 @@
		9f362963 docs/user-guides/indexing/index.mdx (erik-wang-lancedb 2025-11-30 21:12:29 -0800 1) ---

Uh oh!

Conversation

jzheng106 commented Aug 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

westonpace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants