Skip to content

feat: add Italian TN/ITN with raw-field tagging - #397

Merged
pengzhendong merged 6 commits into
masterfrom
feat/italian-normalization
Oct 31, 2026
Merged

pengzhendong merged 6 commits into
masterfrom
feat/italian-normalization

Conversation

@pengzhendong

Copy link
Copy Markdown
Member

Summary

  • Add native Italian (it) TN and a new ITN, completing the direction missing from NeMo. Raw-field taggers keep source text; verbalizers perform the conversion. ITN does not import TN language graphs.
  • Register both directions in the CLI, packaging, cache-manifest tests and README.
  • Apply the shared punctuation joiner. Electronic ITN stops at the domain: a chiocciola b punto com e poi → [email protected] e poi, while URL paths are still accepted.

Coverage and limitations

  • All 52 original TN fixture lines (7 files) are retained verbatim.
  • New ITN is checked against 55 reversed TN groups; the ITN output must be one of the written candidates. Five are excluded with explicit reasons: d for days is not written back, currency symbols are written after a space, two-digit minor units (4,20 £), and eur vs €.
  • Hand-written raw-field, boundary and n-best/mapping tests cover the new direction.

Test plan

  • Original, reversed, raw-field and mapping tests
  • CLI and cache registration regressions with NeMo imports blocked
  • Full project suite on the merged master baseline (19586 passed, NeMo imports blocked)
  • flake8 tn itn; git diff --check only flags the yen TSV row's two empty minor-unit columns, excluded by pre-commit

@pengzhendong
pengzhendong merged commit 0b465dd into master Oct 31, 2026
1 check passed
@pengzhendong
pengzhendong deleted the feat/italian-normalization branch October 31, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant