Skip to content

perf(cli): stream eligible MP4 export audio - #3318

Open
vedanth-jadhav wants to merge 1 commit into
CapSoftware:mainfrom
vedanth-jadhav:perf/cli-stream-mp4-audio
Open

vedanth-jadhav wants to merge 1 commit into
CapSoftware:mainfrom
vedanth-jadhav:perf/cli-stream-mp4-audio

Conversation

@vedanth-jadhav

@vedanth-jadhav vedanth-jadhav commented Oct 26, 2026 •

Copy link
Copy Markdown

CLI Studio MP4 exports eagerly decode entire recording tracks before rendering, including when only a short clip is selected. Both cap export and cap upload --export currently bypass the bounded streaming preparation already used by desktop MP4 exports.

Prepare by container: MP4 uses build_for_mp4/export_prepared; GIF/MOV retain their existing preparation. A private enum keeps each prepared base paired with its settings and lets both CLI callers share progress/completion handling. There is no new cache, public API, dependency or audio algorithm. Instant recording handling stays before this path.

Measurements

Native export-path benchmark, not complete CLI process timing. One binary compares the original build()+export() calls with the proposed build_for_mp4()+export_prepared() calls. Timers include preparation, rendering, audio processing/full-source validation, encoding and successful completion; exclude process startup and fixture creation.

Workload Eager total, s Streaming total, s Eager peak RSS, MiB Streaming peak RSS, MiB
4s clip / 1h AAC microphone 2.467 1.991 1530.3 217.0
4s clip / 1h independent mic + system AAC 2.706 2.214 2853.9 223.7
4s clip / 10s source control 0.723 0.695 213.2 209.7
4s clip / 1h AAC with Studio Sound 2.648 2.180 1539.1 233.4
120s clip / 1h AAC control 18.691 18.320 1541.6 226.8

Long-source short clips use 85–92% less process RSS and finish in 18–19% less time (Studio Sound: 17.7%). The microphone case removes about 0.48s overall and first rendered-frame progress changes from 1.900s to 0.156s; two sources: 2.217s to 0.162s. First progress is an exporter callback, not window presentation. Full-source validation still runs; this does not make total time independent of source length.

Five fresh processes per mode/workload, alternating order, 50 exports total, warm filesystem/shader caches, no overlapping compilation. All ten outputs within every workload have identical decoded RGBA and float32-audio SHA-256 hashes. One-hour single-source total ranges: eager2.428–2.579s, streaming1.923–2.061s; two-source2.626–2.812s versus2.177–2.325s. The long-control ranges overlap (18.440–19.349s versus18.082–18.518s); no significant long-export throughput improvement is claimed. Short-source first progress is 4ms slower (145→149ms). Earlier exploratory long controls were mixed; the final comparison uses matched runtime settings and repeated controls.

Linux x86_64, AMD EPYC9V74, two-CPU quota/affinity0,1; Rust1.88, FFmpeg7.1.5, Mesa25.0.7 Lavapipe/two threads. Root shipping release is opt-level=s/thinLTO/oneCGU. Harness uses two Tokio workers with16MiB worker stacks. Synthetic sources: one-hour320×180/1fps H.264 and48kHz stereo AAC, output640×360/15fps/Web; two-source inputs are independently encoded440Hz/660Hz tones (96/128kbps). No camera/cursor drawing, captions or imported music. Whole-process RSS includes renderer/native allocations; these are not total desktop-app memory or physical-GPU/macOS measurements. Baseline586522aa12aae61b0008104adb8b26f6000e7cf9.

Correctness and behavior

  • Existing MP4 eligibility, transactional output, full-source validation, internal cancellation/drop cleanup and encoder retry remain in the exporter. Existing destinations and unsupported timelines retain eager fallback. The local false user-cancellation flag matches the CLI's current lack of an export signal handler; it does not disable internal pipeline cancellation.
  • Native regression tests verify actual streaming selection, successful export, existing-destination fallback without mutation, and GIF/MOV preparation. Another damages an AAC packet beyond the selected one-second video and requires an audio decoder error with no final output.
  • Intentional diagnostic difference: corrupt unused audio may now fail after progress begins, with Exporter error rather than Exporter build error. The built CLI was exercised on valid and damaged-tail projects: exit0/final Completed for success; exit1/final JSON Error, no Completed and no final MP4 for corruption. Corrupt tails are still rejected.
  • Existing initial/progress/completion/error JSON handling, swallowed callback stdout failures and zero-rendered-frame guard are retained. Both explicit export and upload's default exporter share the same format-specific preparation.

Validation: cargo fmt --all, cargo check -j2 -p cap --locked,19 ordinary CLI export tests, both new native tests explicitly run, strict CLI all-target Clippy and strict benchmark-example Clippy with --no-deps -- -D warnings, actual CLI build/smokes and git diff --check pass. Native tests are ignored by default because they need FFmpeg/ffprobe and a working wgpu adapter, matching the exporter integration-test convention. Scoped PipeWire/X11 prerequisites were installed locally; no lint suppression or unrelated source fix is included.

Reproduce

Use a disposable Studio .cap project with a four-second unit-speed timeline cut from an hour-long AAC recording, no imported music, and unused output paths. The included example prints complete duration, preparation, first progress, streaming eligibility and Linux high-water RSS. It compares both engine paths on the same revision:

cargo build -p cap-export --release --locked --example export-path-comparison
project=/path/to/disposable-studio.cap
for i in 1 2 3 4 5; do
  modes="eager streaming"
  if [ $((i % 2)) -eq 0 ]; then modes="streaming eager"; fi
  for mode in $modes; do
    taskset -c 0,1 target/release/examples/export-path-comparison "$mode" "$project" "/tmp/cap-cli-$i-$mode.mp4"
  done
done
cargo test -p cap --bin cap --locked export::tests
cargo test -p cap --bin cap --locked export::tests::studio_mp4 -- --ignored --test-threads=1

Use the repository native FFmpeg setup and a valid Vulkan ICD/XDG runtime on headless Linux; set CAP_RENDER_FORCE_SOFTWARE_ADAPTER=1 LP_NUM_THREADS=2 for the software-adapter case. Delete only the disposable prior outputs before repeating. Change source duration/track count, Studio Sound and timeline length for the table's controls. Raw samples, fixtures, saved binary, hashes and driver scripts are retained outside the production diff.

Current main and open PRs rechecked: #3133 is MOV encoding; #2777 normalization and #2308 imported-media features are distinct. No overlapping CLI streaming adoption found. Only Linux/software rendering is validated here.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

Both CLI export callers now use build_for_mp4 and export_prepared for MP4. GIF and MOV keep their previous preparation.

  • Eligible Studio MP4 exports stream audio while rendering.

Review checks were limited to source and existing tests; no runtime checks were run.

Acknowledgment: vedanth-jadhav explicitly identifies later failure reporting for corrupt unused audio as intentional. Such failures may follow progress and use Exporter error instead of Exporter build error.

Reviews (1) · Last reviewed commit: "perf(cli): stream eligible MP4 export au..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant