Skip to content

Northern Sámi TN - #480

Open
jimregan wants to merge 38 commits into
NVIDIA:mainfrom
jimregan:northern-sami-tn3
Open

Northern Sámi TN#480
jimregan wants to merge 38 commits into
NVIDIA:mainfrom
jimregan:northern-sami-tn3

Conversation

@jimregan

@jimregan jimregan commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Before your PR is "Ready for review"

Pre checks:

  • Have you signed your commits? Use git commit -s to sign.
  • Do all unittests finish successfully before sending PR?
    1. pytest or (if your machine does not have GPU) pytest --cpu from the root folder (given you marked your test cases accordingly @pytest.mark.run_only_on('CPU')).
    2. Sparrowhawk tests bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...
  • If you are adding a new feature: Have you added test cases for both pytest and Sparrowhawk here.
  • Have you added __init__.py for every folder and subfolder, including data folder which has .TSV files?
  • Have you followed codeQL results and removed unused variables and imports (report is at the bottom of the PR in github review box) ?
  • Have you added the correct license header Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. to all newly added Python files?
  • If you copied nemo_text_processing/text_normalization/en/graph_utils.py your header's second line should be Copyright 2015 and onwards Google, Inc.. See an example here.
  • Remove import guards (try import: ... except: ...) if not already done.
  • If you added a new language or a new feature please update the NeMo documentation (lives in different repo).
  • Have you added your language support to tools/text_processing_deployment/pynini_export.py.

PR Type:

  • New Feature
  • Bugfix
  • Documentation
  • Test

If you haven't finished some of the above items you can still open "Draft" PR.

jimregan and others added 3 commits August 21, 2026 16:26
@jimregan
recreate commits from NVIDIA#11 (Northern Sámi TN)
b6f4d04 
Signed-off-by: Jim O'Regan <[email protected]>
@kathiasi @jimregan
Northern Sami symbol names
3e61af8 
Signed-off-by: Katri Hiovain-Asikainen <[email protected]>
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
Add Northern Sami (se) text normalization
03eb0c0 
Cardinal, ordinal, decimal, fraction, money, measure, date, time,
telephone, electronic, and whitelist coverage for Northern Sami,
including Sparrowhawk grammar export and test cases.

Signed-off-by: Jim O'Regan <[email protected]>
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions Bot added the Stale label Sep 5, 2026
@jimregan
remove Swedishness
c3706f4 
Signed-off-by: Jim O'Regan <[email protected]>
@github-actions github-actions Bot removed the Stale label Sep 8, 2026
@tbartley94 tbartley94 self-assigned this Sep 11, 2026
@tbartley94

tbartley94 commented Sep 11, 2026

Copy link
Copy Markdown
Member

@jimregan Question: how familiar are you with the documented language? (We try and be careful with endangered ones.) Are you a first language speaker, L2 speaker, or simply knowledgeable enough for documentation purposes?

@jimregan

jimregan commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Knowledgeable enough for documentation purposes.

FWIW, this is work I'm doing as part of Språkbanken Tal, the speech component of the Swedish national research infrastructure for language; North Sámi is an official minority language of Sweden so it's in our interest to get this right.

@tbartley94

tbartley94 commented Sep 14, 2026

Copy link
Copy Markdown
Member

Ohhhhh, you're with Sprakbanken? Okay that makes things way easier on our side since you have institutional backing.

Would you be interested in starting an email thread then? If Sprakbanken is interested in this type of work we may be able to set up a more official channel to assist y'all.

@jimregan

jimregan commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Sure: [email protected]

tbartley94
tbartley94 requested changes Sep 25, 2026
View reviewed changes

@tbartley94 tbartley94 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial technical comments, due to size will have to do a second pass after requested changes.

Comment thread nemo_text_processing/text_normalization/se/data/dates/__init__.py Resolved
Comment thread nemo_text_processing/text_normalization/se/data/electronic/__init__.py Resolved
Comment thread nemo_text_processing/text_normalization/se/data/inflection/__init__.py Resolved
Comment thread nemo_text_processing/text_normalization/se/data/inflection/cardinal_alternatives.tsv Resolved
Comment thread nemo_text_processing/text_normalization/se/data/measure/__init__.py Resolved
Comment thread nemo_text_processing/text_normalization/se/taggers/date.py Resolved
Comment thread nemo_text_processing/text_normalization/se/taggers/date.py Resolved
Comment thread nemo_text_processing/text_normalization/se/taggers/decimal.py Resolved
Comment thread nemo_text_processing/text_normalization/se/taggers/ordinal.py Resolved
Comment thread nemo_text_processing/text_normalization/se/utils.py Resolved
@github-actions

github-actions Bot commented Oct 10, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions Bot added the Stale label Oct 10, 2026
jimregan and others added 11 commits October 13, 2026 15:16
@jimregan
replace single-entry file with string_map
04a61f2 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
update year
e879780 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
change arg name
c95682a 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
move asserts outside loop
e2a77f0 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
make acceptors
2bddcaa 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
remove f-strings, use .copy()
16b02dc 
Signed-off-by: Jim O'Regan <[email protected]>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
29e9e4f 
for more information, see https://pre-commit.ci
@jimregan
NEMO_SPACE (+ delete space, explicit transduction of b. -> beaivvis
b1a96b3 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
Merge branch 'northern-sami-tn3' of github.com:jimregan/NeMo-text-pro…
fa38155 
…cessing into northern-sami-tn3
@jimregan
insert_space
ce12a5d 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
move to tsv
0dd5451 
Signed-off-by: Jim O'Regan <[email protected]>
@github-actions github-actions Bot removed the Stale label Oct 14, 2026
@jimregan
jimregan requested a review from tbartley94 October 14, 2026 15:28
tbartley94
tbartley94 requested changes Oct 15, 2026
View reviewed changes

@tbartley94 tbartley94 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting there, copyrights need to be cleared up and some code changes needed for simplification.

Comment thread nemo_text_processing/text_normalization/se/data/money/currency_major.tsv Resolved
Comment thread nemo_text_processing/text_normalization/se/data/money/currency_minor_gen.tsv Resolved
Comment thread nemo_text_processing/text_normalization/se/data/numbers/digit_loc_sg.tsv Resolved
Comment thread Jenkinsfile Resolved
Comment thread tools/text_processing_deployment/pynini_export.py Resolved
Comment thread nemo_text_processing/text_normalization/se/verbalizers/date.py Resolved
Comment thread nemo_text_processing/text_normalization/se/verbalizers/decimals.py Resolved
Comment thread nemo_text_processing/text_normalization/se/verbalizers/measure.py Resolved
Comment thread nemo_text_processing/text_normalization/se/verbalizers/ordinal.py Resolved
Comment thread nemo_text_processing/text_normalization/se/verbalizers/time.py Resolved
jimregan added 11 commits October 16, 2026 18:25
@jimregan
update year
bd6ab3b 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
remove from Jenkinsfile
7794689 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
sync with current main
bd1a95b 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
nothing in these files that I could have had copyright in
7956a18 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
nothing in these files that I could have had copyright in
afca0a2 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
but this one was me
89fbbe3 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
optimize
0486ec1 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
switch from tsv to string_file, change the way the dict is created
06a03b8 
Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
delete duplicate files; illative and locative are same as genitive in…
d4e6a94 
… this context

Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
Ok. These are *attribution* lines, not copyright notices, so let's no…
e79b3a0 
…t pretend otherwise

Signed-off-by: Jim O'Regan <[email protected]>
@jimregan
add a beta warning
1615dbf 
Signed-off-by: Jim O'Regan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tbartley94 tbartley94 tbartley94 requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@tbartley94 tbartley94

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jimregan @tbartley94 @kathiasi