Skip to content

feat: add Hungarian TN/ITN with raw-field tagging - #399

Merged
pengzhendong merged 6 commits into
masterfrom
feat/hungarian-normalization
Oct 31, 2026
Merged

pengzhendong merged 6 commits into
masterfrom
feat/hungarian-normalization

Conversation

@pengzhendong

Copy link
Copy Markdown
Member

Summary

  • Add native Hungarian (hu) TN and a new ITN, completing the direction missing from NeMo. Raw-field taggers keep source text; verbalizers perform the conversion. ITN does not import TN language graphs.
  • Register both directions in the CLI, packaging, cache-manifest tests and README.
  • Apply the shared punctuation joiner. Hyphenated ranges keep the source spacing (100-200 → száz-kétszáz, 100 - 200 → száz - kétszáz).

Coverage and limitations

  • All 205 original TN fixture lines (12 files) are retained verbatim.
  • New ITN is checked against 178 reversed TN cases; the ITN output must be one of the written candidates. 12 are excluded with explicit reasons: each fixture lists only a written style that conflicts with other fixtures (dot vs space grouping, grouped ordinals, milliárd vs mrd, attached units, numeric vs named months) or a telephone layout that cannot be recovered from speech.
  • Hand-written raw-field, semantic and n-best/mapping tests cover the new direction.

Test plan

  • Original, reversed, raw-field and mapping tests
  • CLI and cache registration regressions with NeMo imports blocked
  • Full project suite on the merged master baseline (20104 passed, NeMo imports blocked)
  • flake8 tn itn and git diff --check

@pengzhendong
pengzhendong merged commit 8982d0b into master Oct 31, 2026
1 check passed
@pengzhendong
pengzhendong deleted the feat/hungarian-normalization branch October 31, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant