Repository navigation
Sync ako/mxcli: incremental source-catalog rebuilds (re-describe only what changed) - #1679
Merged
Merged
Conversation
…index (#1790) Once a project had a source index, every rebuild described every document again - including the implicit rebuilds `search` and `show references` start after each save (38 s on a 4,753-unit app). A source build now records its basis (source_units: every unit's content hash + container; source_elements: which document came from which unit) and the next build carries a row over when - its unit is unchanged (an entity's unit is its domain model), - no document its refs name changed (a page prints its data-source flow's return entity), - nothing else changed (a folder, module, settings or Java action change describes everything), and - the same mxcli build wrote it (fingerprint: executable path/size/mtime). FORCE describes everything. UnitContentHashes: v2 from the Unit table, v1 from the blob (v1 hashes were left stale before #1159). FactoryManagement, one attribute added: plain refresh 11.2 s vs 38.2 s, 3,525 of 3,718 rows reused, byte-identical to `force`. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82ttaGzay5CwiC2V7cf
…source # Conflicts: # CHANGELOG.md
fix(catalog): re-describe only what changed when rebuilding a source index (#1790)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Syncs
ako/mxcli:maininto upstream: 1 commit from fork PR ako#1802.Refs #1081 and #1329.
Incremental source-catalog rebuilds
Problem: once a project had a source index (
refresh catalog full source), every later rebuild described every document again. That included the implicit rebuilds thatsearchandshow referencesstart after every save. On a 4,753-unit app that took 38 s each time.This follows from #1081's never-narrow guard: the cheapest rebuild that can still be saved is the most expensive one.
Fix: a source build now records what it described from:
source_units: every unit's content hash and container;source_elements: which document came from which unit;The next build carries a document's row over only when all of these hold:
FORCEstill describes everything. Anything the plan can't vouch for is described again, so reuse can make a build faster but can't change its result.Other details:
Reader.UnitContentHashesreads v2 hashes from the Unit table, and computes v1 hashes from the blob, because older v1 writes left them stale.Evidence
TestSourceCatalog_IncrementalBuildEqualsFullBuildcompares the incremental build with a forced full build, row for row, after each of these:alter entity … add attribute, a plainrefresh catalogtakes 11.2 s instead of 38.2 s, reusing 3,525 of 3,718 rows. The result is byte-identical to a forced full rebuild.docs-site/src/tools/refresh-catalog.mdexplains the incremental rebuild and whatFORCEdoes