Repository navigation
feat(updater): closed funds skip fetch and guard, failed new funds keep a catalog-only row, test groups of at most five - #85
Merged
Conversation
…ep a catalog-only row, test groups of at most five - Q1: CLOSED_FUNDS (empty, no confirmed closed WisdomTree fund) with the shared closed-fund helpers; the run queue drops closed funds before any request or MAX_FETCHES budget, index rows are marked closed on every run - Q2: a guard rejection was already failed; a new fund whose first refresh failed (guard rejection or any error) now keeps a catalog-only row instead of no row, a new fund no source answered fails instead of publishing an empty meta.json, and a catalog-only row is never passed on as published previous state - Q12: describe groups split to at most five tests (55 tests, 388 expect calls before and after) Refs open-items etfs Q1, Q2, Q12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
CLOSED_FUNDS: ClosedFund[] = []afterHOLDINGS_GUARD_RELAX(empty, no confirmed closed WisdomTree fund).runUpdatewraps the eligible list withwithoutClosedFunds(...)before the cursor, the queue and the MAX_FETCHES budget, so a closed fund is never fetched, guarded or reported. The final rows go throughmarkClosedFunds(funds, CLOSED_FUNDS)right beforewriteIndex, which runs on every run (also when nothing was updated). Dropped catalog funds are never pruned here (the index is the published rows plus this run's results), and aTICKERS=<closed>run is accepted because published rows join the universe even when the live catalog dropped themfailed(thrown fromprocessFund, counted infailures): no changemeta.json) whose first refresh failed, for any reason including a guard rejection, wrote NO index row. It now getscatalogOnlyRow(fund)(catalog TER and inception,dataFile: null, holdings 0, full metrics key set with nulls plusreturnsBasis)meta.jsonand reported updated. It now fails withno source answeredand keeps the catalog-only rowdataFile: null) was passed toprocessFundas the published previous state: itsnavValue: nullmade a later run retain a null quote instead of Yahoo's price. It is now ignored as previous statesplit-test-groups.ts(controls 8 -> 4+4, parsing 12 -> 4+4+4, metrics 7 -> 4+3, pipeline 21 -> 5+4+4+4+4, network 7 -> 4+3), no shared let/var reportedTests
a closed fund is never fetched or reported failed: ...: BBB inCLOSED_FUNDS, guard-failing holdings and the live catalog no longer listing it: no BBB request, exit code not 1, fund files equal the first run, index row equal plusstatus/closedDate/closedSource, 3 rows, a rerun touches nothing;closedFundthrows on a malformed entry. Fails withoutwithoutClosedFundsand withoutmarkClosedFunds(checked by reverting each)a new fund whose first refresh fails (guard rejection, no source answered) ...: catalog-only rows for both, no fund folders, zero diff on a rerun, and the catalog-only row is not read back (DDD gets Yahoo's 140 close). Fails without each of the three Q2 changes (checked by reverting each)Checks run
bun test: 53 tests / 365 expect calls on main -> 55 / 388 (same 55 / 388 before and after the split), exit 0gates.sh WisdomTree: install,bun test,bun run build-github-pages, test with the workflow env, bothbun build,git diff --check, hub checks, exec bit, tc 64 base errors -> 64, none added:WisdomTree GATES OKTICKERS=USFR REQUEST_SLEEP=0 bun scripts/update-data.ts: rc 0, 1 unchanged, 0 failedclosed-run.sh WisdomTree USFR:[ closed ] USFR, 0 funds processed, 0 failed, index.json diff only the closed marks plusgeneratedAt; data restored, no data committedRefs open-items etfs Q1, Q2, Q12