Skip to content

feat: add Kinyarwanda TN/ITN with raw-field tagging - #394

Merged
pengzhendong merged 2 commits into
masterfrom
feat/kinyarwanda-normalization
Oct 31, 2026
Merged

pengzhendong merged 2 commits into
masterfrom
feat/kinyarwanda-normalization

Conversation

@pengzhendong

Copy link
Copy Markdown
Member

Summary

  • Add native Kinyarwanda (rw) TN and a new ITN, completing the direction missing from NeMo. Raw-field taggers keep source text; verbalizers perform the conversion. ITN does not import TN language graphs.
  • Register both directions in the CLI, packaging, cache-manifest tests and README.
  • Apply the shared punctuation joiner. Preserve fixture-required trailing spaces on whole-hour readings, but avoid duplicated joining spaces mid-sentence.

Coverage and limitations

  • Original TN rules: cardinal, time and the transliteration whitelist, with word/punctuation fallback.
  • New ITN is checked against reversed TN fixtures and hand-written raw-field/mapping tests. Five reverse cases remain excluded with explicit reasons: unrecoverable comma formatting, two non-inverted transliterations and two outer-whitespace variants.
  • Ambiguous numeric readings retain n-best alternatives; only acronym whitelist entries that do not conflict with ordinary words are inverted.

Test plan

  • All 100 original TN fixture lines retained verbatim
  • Language and registration regressions pass with NeMo imports blocked
  • Full project suite on the merged master baseline (19707 passed, NeMo imports blocked)
  • flake8 tn itn; git diff --check only flags fixture/TSV trailing spaces, which are significant and excluded by pre-commit

@pengzhendong pengzhendong changed the title feat: add Kinyarwanda ITN and register bidirectional pipelines feat: add Kinyarwanda TN/ITN with raw-field tagging Oct 31, 2026
@pengzhendong
pengzhendong merged commit 0c93950 into master Oct 31, 2026
1 check passed
@pengzhendong
pengzhendong deleted the feat/kinyarwanda-normalization branch October 31, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant