Repository navigation
feat(gpui): audio scrub while dragging the playhead - #2845
YoavLevavi wants to merge 3 commits into
Conversation
With playback stopped, each ruler move plays a 90ms slice of the mix under the playhead (recording audio plus music) on a small output stream of its own. EditorInstance::audio_snippet renders the slice with the regular AudioRenderer. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
@YoavLevavi is attempting to deploy a commit to the Cap Software Inc Team on Vercel. A member of the Team first needs to authorize it. |
|
Thanks @YoavLevavi, audio scrubbing is something people ask for, and rendering the slice with the normal |
- AudioOutput gains play_snippet: slices go to the shared output thread, which uses ensure_stream (rebuilt on device change or stream failure, every sample format, the usual idle-pause policy), converts them to the device's rate, channels and sample type there, and hands them to the stream callback. Played-out buffers go back to the output thread, so the callback neither allocates nor frees. Playback always takes precedence. A failed open drops slices for 2s instead of retrying per slice. - EditorInstance::scrub_audio renders and plays on its own AudioOutput, so the stream lives and dies with the editor; the process-wide player is gone. - The timeline toolbar has a speaker toggle (saved as editor_preferences.audioScrub, on by default) to turn scrub audio off. - Merges origin/main (keeps its scrub_to_time repaint path). Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Thanks @richiemcilroy, reworked in 78cb060 (also merged main):
@greptileai review |
| if let Some((spent, _)) = snippet.replace((next, 0)) { | ||
| let _ = spent_tx.send(spent); |
There was a problem hiding this comment.
Each slice lasts 90 ms, but a waiting slice can replace it after roughly 60 ms. The callback discards the old slice before its fade-out, jumping from a full-volume sample to the new slice's faded-in start. This can click during a drag. Fade or crossfade at replacement instead of relying only on the buffer's ending fade.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/editor/src/audio_output.rs
Line: 1588-1589
Comment:
**Replaced slices can click**
Each slice lasts 90 ms, but a waiting slice can replace it after roughly 60 ms. The callback discards the old slice before its fade-out, jumping from a full-volume sample to the new slice's faded-in start. This can click during a drag. Fade or crossfade at replacement instead of relying only on the buffer's ending fade.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| let (snippet_tx, snippet_rx) = std_mpsc::channel::<Vec<T>>(); | ||
| let (spent_tx, spent_rx) = std_mpsc::channel::<Vec<T>>(); |
There was a problem hiding this comment.
Audio callback allocates memory
Returning buffers through std_mpsc::channel does not keep memory work out of the audio callback. This unbounded queue can allocate blocks during spent_tx.send, and receiving from snippet_rx can free queue blocks. Memory contention can then delay audio delivery. Use a preallocated queue or buffer exchange in both directions.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/editor/src/audio_output.rs
Line: 1578-1579
Comment:
**Audio callback allocates memory**
Returning buffers through `std_mpsc::channel` does not keep memory work out of the audio callback. This unbounded queue can allocate blocks during `spent_tx.send`, and receiving from `snippet_rx` can free queue blocks. Memory contention can then delay audio delivery. Use a preallocated queue or buffer exchange in both directions.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| let mut renderer = crate::AudioRenderer::new(segments).with_music(music); | ||
| renderer.set_playhead(time.max(0.0), &project); | ||
| let frames = | ||
| (secs.clamp(0.01, 0.5) * f64::from(crate::AudioRenderer::SAMPLE_RATE)) as usize; | ||
| let Some((_, mut samples)) = renderer.render_frame_raw(frames, &project) else { |
There was a problem hiding this comment.
audio_snippet calls render_frame_raw directly inside the Tokio task. For clips with changed speed, that call builds and runs audio filters synchronously. The fresh renderer repeats this setup for every slice, which can hold a Tokio worker and delay other editor tasks during a drag. Move setup and rendering into spawn_blocking or a dedicated worker, as regular playback does.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/editor/src/editor_instance.rs
Line: 881-885
Comment:
**Scrubbing holds Tokio workers**
`audio_snippet` calls `render_frame_raw` directly inside the Tokio task. For clips with changed speed, that call builds and runs audio filters synchronously. The fresh renderer repeats this setup for every slice, which can hold a Tokio worker and delay other editor tasks during a drag. Move setup and rendering into `spawn_blocking` or a dedicated worker, as regular playback does.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| section.insert("isolation".into(), serde_json::to_value(isolation).unwrap()); | ||
| set_store_value(AUDIO_ENHANCEMENT, Value::Object(section)) | ||
| } | ||
| /// Editor preferences that belong to the user rather than a project. |
There was a problem hiding this comment.
The new comment above EDITOR_PREFERENCES only restates the constant's purpose. AGENTS.md requires comments to explain non-obvious context and explicitly bans narration of the code. Remove this comment and the similar narration above audio_scrub_enabled and SCRUB_SLICE_SECS; the pattern also appears elsewhere in the additions. This repository requirement must be satisfied before merging.
Context Used: AGENTS.md (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/src/store.rs
Line: 1099
Comment:
**Comments repeat the code**
The new comment above `EDITOR_PREFERENCES` only restates the constant's purpose. `AGENTS.md` requires comments to explain non-obvious context and explicitly bans narration of the code. Remove this comment and the similar narration above `audio_scrub_enabled` and `SCRUB_SLICE_SECS`; the pattern also appears elsewhere in the additions. This repository requirement must be satisfied before merging.
**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…nc workers Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Both fixed in a637871:
@greptileai review |
Audio scrubbing: with playback stopped, dragging on the ruler plays a short slice of the mix under the playhead, like Premiere and CapCut.
EditorInstance::audio_snippet(time, secs)(cap-editor) renders the slice with the regularAudioRenderer(recording audio plus music tracks) and fades its edges so back-to-back slices don't click.scrub_audio.rsplays each slice on a small cpal output stream owned by its own thread, so the playback pipeline is untouched. A new slice replaces what is left of the previous one, and the stream pauses after 2s idle.Testing:
cargo test --bin cap-gpui,cargo check -p cap-editor.🤖 Generated with Claude Code
The callback-memory and Tokio-worker findings are fixed, but the existing comment-policy requirement still needs to be satisfied before merging.
Findings
Fix with agent prompt
Summary
Adds audio previews while dragging the ruler with playback stopped.
Reviews (2) · Last reviewed commit: "Hand scrub slices through preallocated r..." · Reviewed by Greptile