fix: Yamato jobs not testing built player by EmandM · Pull Request #4194 · Unity-Technologies/com.unity.netcode.gameobjects

EmandM · 2026-10-15T21:27:27Z

Purpose of this PR

I noticed in #4188 that the CMB tests have become very flaky. This was blocking CoreCLR changes from landing as fast as they should be able to.

In investigating this, I discovered that the desktop and console standalone tests weren't running the tests on the pre-built players, they were accidentally falling back to using the editor instead. This meant we weren't running our tests on a number of platforms.

This PR:

Updates the Standalone jobs to correctly use the pre-built players to run our tests
Excludes tests (with JIRA tickets) that fail on Playstation platforms
Removes the repetition of the NetworkVariableTests. They already have multiple different values, and many of these tests are duplicated elsewhere in our test framework now. The [Repeat] attribute was causing the tests to be flakey
Fixes NetcodeIntegrationTest to ensure the test scene is always set back to the active scene, and that the test framework will wait until all the scenes loaded by the test are unloaded before finishing teardown.
Updates ObjectNameIdentifier to use the DA compatible OnOwnershipChanged (NetworkObjectSynchronizationTests(DAHost).NetworkObjectDeserializationFailure was failing because of it)
Couple of other small fixes.

Jira ticket

UUM-155844

Changelog

Fixed: various small test framework fixes
API was removed and what they should use instead.

Documentation

No documentation changes or additions were necessary.

Testing & QA (How your changes can be verified during release Playtest)

Functional Testing

Manual testing :

Manual testing done

Automated tests:

Covered by existing automated tests
Covered by new automated tests

Does the change require QA team to:

Review automated tests?
Execute manual tests?
Provide feedback about the PR?

If any boxes above are checked the QA team will be automatically added as a PR reviewer.

Up-port

n/a

Backports

Backported in #4196

u-pr · 2026-10-16T17:21:47Z

⚠️ Some settings were ignored in '.pr_agent.toml (repository root)' repo settings

Check the configuration's TOML syntax, permitted settings, and size limit. See the error below.

Error message:
Ignored prohibited settings: github_app.handle_pr_actions. Permitted settings were applied.

Configuration content:

# This is the configuration for the u-pr bot
# https://github.cds.internal.unity3d.com/unity/u-pr
# For configuration of this file:
# https://developer.portal.internal.unity.com/catalog/default/component/u-pr/docs/configuration/automatic_runs/

[github_app]
handle_pr_actions = ['opened', 'ready_for_review']  # PR events that auto-run pr_commands
pr_commands       = ["/harness-review"]             # what runs on those events
reviewer_commands = ["/harness-review"]             # what runs when @u-pr is added as reviewer

michalChrobot · 2026-10-16T17:33:29Z

    # run_cmb_service.sh builds and starts a release version of the full CMB service (along with the limited echo server)
    - ./Tools/CI/service.cmb/run_cmb_service.sh -e $ECHO_SERVER_PORT -s $CMB_SERVICE_PORT -l artifacts

-    - unity-downloader-cli --fast --wait -u {{ editor }} -c Editor {% if backend == "il2cpp" %} -c il2cpp {% endif %}{% if backend == "coreclr" %}{% unless coreclr_runtime_editors contains editor %} -c {{ platform.coreclr_component }} {% endunless %}{% endif %} --arch {{ platform.architecture }}


don't we need to download the editor?

Or is it because in order to execute tests on a build we don't need to download the editor

Nope! Because we've already built the players with the editor in the separate build step, and apparently UnityTestRunner is pre-installed the bokken images we use.

The job runs faster (no need to grab unity) and really relieves the memory pressure on the machine (no unity taking up resources). And! If we ever have this problem again, without unity there UTR can't silently fall-back to using the editor to run the tests

u-pr

💡 Harness Review

The change makes standalone jobs explicitly execute built players and updates shared test helpers, but several ownership and cleanup safeguards remain incomplete.

_{Reviewed commit 5fe06fc}

_{🤖 Helpful? 👍/👎}

michalChrobot · 2026-10-16T17:36:58Z

    model: {{ platform.model }} # This is set only in platforms where we want non-default model to use (more information in project.metafile)
 {% endif %}
  commands:
-    # Installing editor. We still need the editor to run tests on standalone build and for that the Editor is required


Is this comment false?

Yeah, I think I thought that UTR came from the editor, but apparently it's already on the image.

NoelStephensUnity

Very nice!
Good catches!

codecov-github-com · 2026-10-16T19:23:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

@@              Coverage Diff               @@
##           develop-3.x.x    #4194   +/-   ##
==============================================
  Coverage          76.11%   76.11%           
==============================================
  Files                159      159           
  Lines              27333    27333           
==============================================
  Hits               20805    20805           
  Misses              6528     6528

Flag	Coverage Δ
NGOv2_project_testproject_ubuntu	`75.16% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
com.unity.netcode.gameobjects	`76.11% <ø> (ø)`

ℹ️ Need help interpreting these results?

* fix: Yamato jobs not testing built player (Backport of: #4194) * fix: Yamato jobs not testing built player * Next round of fixes * Set addressables to build on players * dotnet-fix * Fix console tests * Try again to remove graphics from desktop players * Fix DA NetworkObjectDeserializationFailure test * Remove the temporarily added N4E settings * Fix the test failures * Remove fix that was more a symptom than a cause (keep it simple) * Match the deregistration to the registration logic * Don't finish cleaning up if any scene is still loading * Fix the U-PR config * Fix IL2CPP player tests * Remove CoreCLR stuff * feat: Improve error message on connection failure [2.X] (#4208) feat: Improve error message on connection failure * Disabled automatic release preparation Disabled recurring trigger as in our current situation it would be more messy to handle automated release step. I will leave it manual for now * chore: Ensure the rerun strategy is baked into the built players (#4218) * fix: generate canonical bools in NetworkVariable serialization tests (#4198) * test: generate canonical bools in NetworkVariable serialization tests RandGenBytes<bool> filled each bool with a random byte, so most values were neither 0 nor 1. On mac arm64 IL2CPP the List<bool> and NativeHashMap<K, bool> cases then failed with both sides printing identical all-True contents. RandGenBool returns only false or true. * test: use RandGenBool for the NativeArray<bool> case * test: make changed2 always differ from original2 in collection generators With RandGenBool an overwrite keeps the existing value half the time, so when no elements were added or removed, changed2 could equal original2 and the "do not match" assertions failed. Each overwrite now redraws until it differs from the original element. --------- Co-authored-by: Emma <[email protected]> (cherry picked from commit b5c186c) --------- Co-authored-by: Noel Stephens <[email protected]> Co-authored-by: Simon Lemay <[email protected]> Co-authored-by: Michał Chrobot <[email protected]>

EmandM added 10 commits October 15, 2026 17:26

fix: Yamato jobs not testing built player

a779769

Next round of fixes

240eba5

Set addressables to build on players

98846f0

dotnet-fix

701debd

Fix console tests

0007b82

Try again to remove graphics from desktop players

2ef41bd

Fix DA NetworkObjectDeserializationFailure test

0996492

Remove the temporarily added N4E settings

196e4ec

Fix the test failures

a7ebbea

Merge branch 'develop-3.x.x' into fix/yamato-jobs

5fe06fc

EmandM marked this pull request as ready for review October 16, 2026 17:21

EmandM requested review from a team and NoelStephensUnity as code owners October 16, 2026 17:21

Remove fix that was more a symptom than a cause (keep it simple)

60935fc

michalChrobot reviewed Oct 16, 2026

View reviewed changes

u-pr Bot reviewed Oct 16, 2026

View reviewed changes

michalChrobot reviewed Oct 16, 2026

View reviewed changes

NoelStephensUnity approved these changes Oct 16, 2026

View reviewed changes

EmandM added 3 commits October 16, 2026 14:11

Match the deregistration to the registration logic

0fa8a08

Don't finish cleaning up if any scene is still loading

f47f747

Fix the U-PR config

241459e

Fix IL2CPP player tests

6639ab2

NoelStephensUnity merged commit e85a580 into develop-3.x.x Oct 17, 2026
26 checks passed

NoelStephensUnity deleted the fix/yamato-jobs branch October 17, 2026 03:18

EmandM mentioned this pull request Oct 19, 2026

fix: Yamato jobs not testing built player (Backport of: #4194) #4196

Merged

1 of 6 tasks

NoelStephensUnity approved these changes Oct 19, 2026 •

edited

Loading

View reviewed changes

NoelStephensUnity mentioned this pull request Oct 19, 2026

fix: generate canonical bools in NetworkVariable serialization tests #4198

Merged

1 task done

EmandM mentioned this pull request Oct 26, 2026

chore: Ensure the rerun strategy is baked into the built players #4218

Merged

6 tasks

Navigation Menu

Repository navigation

fix: Yamato jobs not testing built player - #4194

fix: Yamato jobs not testing built player#4194
NoelStephensUnity merged 15 commits into
develop-3.x.xfrom
fix/yamato-jobs

EmandM commented Oct 15, 2026 •

edited

Loading

Uh oh!

u-pr Bot commented Oct 16, 2026

Uh oh!

michalChrobot Oct 16, 2026

Uh oh!

michalChrobot Oct 16, 2026

Uh oh!

EmandM Oct 16, 2026

Uh oh!

u-pr Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michalChrobot Oct 16, 2026

Uh oh!

EmandM Oct 16, 2026

Uh oh!

NoelStephensUnity left a comment

Uh oh!

codecov-github-com Bot commented Oct 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

EmandM commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of this PR

Jira ticket

Changelog

Documentation

Testing & QA (How your changes can be verified during release Playtest)

Functional Testing

Up-port

Backports

Uh oh!

u-pr Bot commented Oct 16, 2026

Uh oh!

michalChrobot Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

michalChrobot Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

EmandM Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michalChrobot Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

EmandM Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

NoelStephensUnity left a comment

Choose a reason for hiding this comment

Uh oh!

codecov-github-com Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

EmandM commented Oct 15, 2026 •

edited

Loading

codecov-github-com Bot commented Oct 16, 2026 •

edited

Loading