Skip to content

feat(orchestrator): checkpoint change-footprint metric (skip/fs-only opportunity) - #3753

Open
AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:feat/checkpoint-change-footprint-metric
Open

AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:feat/checkpoint-change-footprint-metric

Conversation

@AdaAibaby

Copy link
Copy Markdown
Contributor

What

Add orchestrator.sandbox.checkpoint.change_footprint — a per-pause counter sliced by file_type (memfile|rootfs) and clean (dirty bytes <= one block), emitted from recordSnapshotDiff off data already computed on every pause. No behavior change.

Why

Agent sandboxes checkpoint constantly, and many turns change little. Whether to skip an unchanged checkpoint (or downgrade it to filesystem-only) is only worth building if the no-op rate is actually high — and today there is no metric for it. snapshot.diff.{bytes,ratio} exist but carry the full dirty size; nothing says "this pause was a no-op," and nothing is sliced so a dashboard can read the skip opportunity directly.

This counter is the measurement before the mechanism:

  • per-dimension clean rate = the filesystem-only-downgrade / skip opportunity, measured on real traffic;
  • both dimensions clean on one pause = the full-skip rate (join by sandbox/trace in the dashboard).

It needs no new machinery, no in-guest agent, no kernel features — just exposes the authoritative dirty-page count the orchestrator already derives every pause.

Context

A prior attempt to add skip-unchanged checkpoints (#2586, closed as "complex, benefits uncertain") bet on an in-guest eBPF inspector to manufacture a change signal. But the orchestrator already computes the exact dirty set per pause — the missing piece was never the signal, it was a way to quantify the opportunity so the feature can be decided on data rather than a guess. This PR is exactly that quantification, and nothing more.

Change

  • meters.go: new counter name + description + unit.
  • snapshot_metrics.go: register the counter; emit Add(1, {file_type, clean, team}) in recordSnapshotDiff; extract snapshotDiffClean(dirtyBytes) (threshold = one 4 KiB block) as a tested helper.
  • unit test for the helper.

Caveat (documented separately)

On the destroy-path pause the memfile diff is parented on the original template, not the previous snapshot, so its "clean" reflects the resume working set, not since-last-snapshot change. The rootfs dimension is a true diff and is unaffected. Filed separately as a baseline-semantics clarification so the metric is not misread for memory.

…opportunity)

Add orchestrator.sandbox.checkpoint.change_footprint, a per-pause counter
sliced by file_type (memfile|rootfs) and clean (dirty bytes <= one block),
emitted from recordSnapshotDiff off data already computed every pause.

The per-dimension clean rate is the filesystem-only-downgrade / skip
opportunity measured on real traffic; both dimensions clean on one pause is
the full-skip rate (join by sandbox/trace in the dashboard). This is the
measurement a skip-unchanged feature needs BEFORE adding any mechanism:
it answers how often a checkpoint is a no-op, from existing signals, with
no new machinery, no in-guest agent, and no behavior change.
@AdaAibaby

Copy link
Copy Markdown
Contributor Author

Related issues surfaced while building this metric:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants