Skip to content

fix: filter redundant OCI whiteouts before rootfs copy - #3751

Draft
mattteufel-e2b wants to merge 1 commit into
e2b-dev:mainfrom
mattteufel-e2b:fix/oci-whiteout-directories
Draft

mattteufel-e2b wants to merge 1 commit into
e2b-dev:mainfrom
mattteufel-e2b:fix/oci-whiteout-directories

Conversation

@mattteufel-e2b

@mattteufel-e2b mattteufel-e2b commented Oct 26, 2026 •

Copy link
Copy Markdown

OCI layers can introduce a directory containing whiteouts for paths absent from every lower layer. OverlayFS can list those names even though lookup hides them, so rootfs rsync exits 24 and aborts the template build. This patch marks the immediate parents of character-device 0:0 whiteouts with an empty trusted.overlay.origin after extraction, making OverlayFS filter them during directory enumeration.

The helper scans only temporary extracted layers used as read-only lower directories, does not follow symlinks, preserves existing origin/opaque metadata, and propagates xattr errors. Rsync errors remain fatal. It adds one directory walk per layer; the full-image component reproduction measured 0.86 seconds for the walk/mark pass.

Validation:

  • The pinned public astanin__python-tabulate-76 image fails Cloud import twice, including a serial retry. Cloud logs expose only a generic post-extraction error; they do not reveal rsync stderr.
  • A separate Linux replay of all ten verified image layers, using the same archive library/extraction semantics and rsync command, reproduces 32 vanished entries and exit 24. Applying the candidate metadata makes rsync exit 0. Before/after source inventories and before/after copied-destination inventories have matching hashes. This replay uses a replicated helper, not a deployed template builder.
  • The compiled production OCI package passes all 12 tests, including new extraction/overlay/copy regressions, lower-file preservation, opaque directories, same/higher-layer recreation, links, existing xattrs, no-follow behavior, read-only failure, and strict rsync failure propagation. Linux cross-compilation, vet and repository-pinned golangci-lint 2.15.0 pass.
  • Independent source review found no open correctness findings.

The full-image replay uses Linux 6.1.158 with explicitly opted-in legacy mounts. CI separately passes all 12 OCI tests on Linux 6.17 for AMD64 (race detector enabled) and ARM64, using the production mount path without fallback. AMD64 job · ARM64 job. No production mount code was changed. This remains a draft for infra review; no hosted backend was deployed or rebuilt with this patch.

Reproduction inputs, logs and exact scope · Full-image evidence · Kernel mechanism

@cla-bot cla-bot Bot added the cla-signed label Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant