Repository navigation
Native: return rerank_oversample and fused_pq from VectorIndex.get_metadata when the engine has them - #89
Conversation
…dex metadata when the engine has them The native vector-index metadata op listed a fixed set of fields, so VectorIndex.get_metadata() never carried rerank_oversample and fused_pq on an image linked against ArcadeDB 26.11.1 (ArcadeData/arcadedb#10545), although the JVM wrapper returns them. The glue reads them from the index's own definition JSON (LSMVectorIndex.toJSON writes every knob the linked engine's metadata has), so it still compiles and links against 26.10.2, where the keys stay out, which the Python side already expects (_OPTIONAL_METADATA_KEYS). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
|
Parity of the native wheel built from this branch against engine 26.11.1-SNAPSHOT (upstream main 9b7cd9a5eb), full suite, wheel installed, one process per test file:
Details: |
|
Parity on engine 26.11.1-SNAPSHOT (laptop run): 1449 passed, 1 failed, 124 NA-JVM, 0 errors, 0 NOTYET, 0 timeouts (the 26.10.2 reference is 1441 / 0). The one failure, |
What
VectorIndex.get_metadata()on the native build now returnsrerank_oversampleandfused_pqwhen the engine the image links has them (ArcadeDB 26.11.1, ArcadeData/arcadedb#10545). The JVM wrapper already returns them. This is backlog item N-05.How
The native
metadataop inNativeExtras.javalists a fixed set of fields. It now also readsrerankOversampleandfusedPQfromLSMVectorIndex.toJSON(), which writes every knob the linked engine'sLSMVectorIndexMetadatahas. The glue names no field that 26.10.2 lacks, so it compiles and links against both engines (checked:mvn compilewith-Darcadedb.version=26.10.2and with26.11.1-SNAPSHOTbuilt from upstream main 9b7cd9a5eb, both clean). On 26.10.2 the two keys stay out, which_OPTIONAL_METADATA_KEYSinschema_objects.pyalready expects. No Python change is needed.encoding,neighborOverflowFactorandalphaDiversityRelaxationare not added: the JVMget_metadata()does not return them either (the tests that read them go through the Java metadata object and stay NOTYET on native).Why now
tests/test_vector_product_quantization.pyprobes the engine (has_pq_rerank) and assertsmeta["rerank_oversample"]andmeta["fused_pq"]once the engine acceptsrerankOversample. On a 26.11.1 image without this change those assertions fail with a KeyError. On 26.10.2 CI nothing changes.Test plan
mvn compileof the glue against 26.10.2 and against 26.11.1-SNAPSHOT (no errors, no warnings)scripts/check_no_embedded_name.py --root .🤖 Generated with Claude Code