Skip to content

feat(updater): closed funds skip fetch and guard, failed new funds keep a catalog-only row, test groups of at most five - #85

Merged
daggerok merged 1 commit into
mainfrom
feat/closed-funds-guard-policy
Oct 26, 2026
Merged

daggerok merged 1 commit into
mainfrom
feat/closed-funds-guard-policy

Conversation

@daggerok

Copy link
Copy Markdown
Owner

What changed

  • Q1 closed funds: CLOSED_FUNDS: ClosedFund[] = [] after HOLDINGS_GUARD_RELAX (empty, no confirmed closed WisdomTree fund). runUpdate wraps the eligible list with withoutClosedFunds(...) before the cursor, the queue and the MAX_FETCHES budget, so a closed fund is never fetched, guarded or reported. The final rows go through markClosedFunds(funds, CLOSED_FUNDS) right before writeIndex, which runs on every run (also when nothing was updated). Dropped catalog funds are never pruned here (the index is the published rows plus this run's results), and a TICKERS=<closed> run is accepted because published rows join the universe even when the live catalog dropped them
  • Q2 one reporting policy
    • a guard rejection was already reported failed (thrown from processFund, counted in failures): no change
    • a new fund (no meta.json) whose first refresh failed, for any reason including a guard rejection, wrote NO index row. It now gets catalogOnlyRow(fund) (catalog TER and inception, dataFile: null, holdings 0, full metrics key set with nulls plus returnsBasis)
    • a new fund that no source answered was published as an empty meta.json and reported updated. It now fails with no source answered and keeps the catalog-only row
    • a catalog-only row (dataFile: null) was passed to processFund as the published previous state: its navValue: null made a later run retain a null quote instead of Yahoo's price. It is now ignored as previous state
  • Q12: groups split to at most five tests with split-test-groups.ts (controls 8 -> 4+4, parsing 12 -> 4+4+4, metrics 7 -> 4+3, pipeline 21 -> 5+4+4+4+4, network 7 -> 4+3), no shared let/var reported

Tests

  • a closed fund is never fetched or reported failed: ...: BBB in CLOSED_FUNDS, guard-failing holdings and the live catalog no longer listing it: no BBB request, exit code not 1, fund files equal the first run, index row equal plus status/closedDate/closedSource, 3 rows, a rerun touches nothing; closedFund throws on a malformed entry. Fails without withoutClosedFunds and without markClosedFunds (checked by reverting each)
  • a new fund whose first refresh fails (guard rejection, no source answered) ...: catalog-only rows for both, no fund folders, zero diff on a rerun, and the catalog-only row is not read back (DDD gets Yahoo's 140 close). Fails without each of the three Q2 changes (checked by reverting each)

Checks run

  • bun test: 53 tests / 365 expect calls on main -> 55 / 388 (same 55 / 388 before and after the split), exit 0
  • gates.sh WisdomTree: install, bun test, bun run build-github-pages, test with the workflow env, both bun build, git diff --check, hub checks, exec bit, tc 64 base errors -> 64, none added: WisdomTree GATES OK
  • real run TICKERS=USFR REQUEST_SLEEP=0 bun scripts/update-data.ts: rc 0, 1 unchanged, 0 failed
  • closed-run.sh WisdomTree USFR: [ closed ] USFR, 0 funds processed, 0 failed, index.json diff only the closed marks plus generatedAt; data restored, no data committed

Refs open-items etfs Q1, Q2, Q12

…ep a catalog-only row, test groups of at most five

- Q1: CLOSED_FUNDS (empty, no confirmed closed WisdomTree fund) with the shared closed-fund helpers; the run queue drops closed funds before any request or MAX_FETCHES budget, index rows are marked closed on every run
- Q2: a guard rejection was already failed; a new fund whose first refresh failed (guard rejection or any error) now keeps a catalog-only row instead of no row, a new fund no source answered fails instead of publishing an empty meta.json, and a catalog-only row is never passed on as published previous state
- Q12: describe groups split to at most five tests (55 tests, 388 expect calls before and after)

Refs open-items etfs Q1, Q2, Q12
@daggerok
daggerok merged commit 8a95a2a into main Oct 26, 2026
1 check passed
@daggerok
daggerok deleted the feat/closed-funds-guard-policy branch October 26, 2026 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant