Skip to content

feat: add Spanish-English code-mixed ITN with raw-field tagging - #400

Merged
pengzhendong merged 6 commits into
masterfrom
feat/spanish-english-normalization
Oct 31, 2026
Merged

pengzhendong merged 6 commits into
masterfrom
feat/spanish-english-normalization

Conversation

@pengzhendong

Copy link
Copy Markdown
Member

Summary

  • Add native Spanish–English code-mixed ITN (es_en), reusing the existing standalone Spanish and English ITN rules. No TN pipeline is introduced.
  • Keep source text in raw fields. Local rules add code-mixed time, whitelist and prefixed forms, plus English Indian numbering and measure units, without modifying the standalone grammars.
  • Register the ITN-only CLI entry (with --cased, default false), package data, configuration-aware cache tests and README. TN rejects es_en.

Test plan

  • All 976 original fixture lines (22 files) retained verbatim: 749 default and 227 cased cases
  • Hand-written code-mixed, raw-field, escaping, whitespace and n-best/mapping cases
  • CLI and cache registration regressions with NeMo imports blocked
  • Full project suite on the merged master baseline (21943 passed, NeMo imports blocked)
  • flake8 tn itn and git diff --check

@pengzhendong
pengzhendong merged commit c3b4d88 into master Oct 31, 2026
1 check passed
@pengzhendong
pengzhendong deleted the feat/spanish-english-normalization branch October 31, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant