feat: keyframes for text and image overlays (engine + GPUI) by YoavLevavi · Pull Request #2846 · CapSoftware/Cap

YoavLevavi · 2026-10-21T15:48:07Z

Keyframes for text and image overlays: animate position, size and opacity over a segment.

Engine (shared crates)

OverlayKeyframes { position, size, opacity } on TextSegment and ImageSegment (#[serde(default)], so existing projects load unchanged). Times are seconds from the segment start; values ease between keys and hold outside them.
prepare_texts and the image layer sample them per frame. Segment helpers: opacity_at, set_opacity_at, add_keyframe_at, plus upsert and remove on OverlayKeyframes.

GPUI editor

The text and image panels get a Keyframes section: add a key at the playhead, jump to a key, remove one, or clear all.
Once a property is keyed, dragging the overlay on the canvas or moving the opacity slider edits (or adds) the key under the playhead. Text keys position and opacity; its box follows the font size.
The canvas handles and snap targets follow the animated position. Keys show as small marks on the timeline boxes.

Testing: cargo test -p cap-project --lib (sampling, editing, segment helpers), cargo test -p cap-rendering --lib text image, cargo test --bin cap-gpui.

🤖 Generated with Claude Code

Confidence Score: 3/5

The PR still needs fixes because deleting clips and changing clip speed leave overlay keys at the wrong times.

Findings

Deleted clips leave keys behind ▶
Faster clips cut animations short ▶
Centering ignores animated positions ▶
Moving changes the size animation ▶
Comments repeat the code ▶

Fix with agent prompt

### Issue 1
apps/desktop-gpui/src/editor_edits.rs:683-685
Deleting a clip does not retime keys on text or images that cross the removed interval. Both still use `ripple_delete_track`, which changes only their bounds. For an overlay spanning 10–20 seconds, deleting 12–14 seconds shortens it to 10–18, but a key originally at 18 stays there instead of moving to 16. Add key-aware ripple deletion for both overlay types.

### Issue 2
crates/project/src/configuration.rs:1439-1440
`set_clip_segment_timescale` retimes text and image bounds but leaves their new keys unchanged. Speeding a 10-second clip to 2× shortens a matching overlay to 5 seconds, but its final opacity key remains at relative time 10. The overlay disappears before reaching that value. Map each overlay key through the same change in time, as the mask path already does.

### Issue 3
crates/project/src/configuration.rs:undefined-1306
Once position keys exist, `sample` ignores changes to the base `center`. The existing “Center on canvas” button still writes only `segment.center`, so it stops moving an image after the user adds a keyframe. Switching keyed text to `TextLayout::Fullscreen` has the same problem. Route these position changes through the keys, or explicitly reset the position animation.

### Issue 4
apps/desktop-gpui/src/editor_canvas.rs:1269-1272
Moving or nudging an image inserts a size key even when its size did not change. With size keys at 0 and 10 seconds, moving the image at 5 seconds inserts a third size key. Because each span eases separately, this changes the size animation before and after the move. Write a size key only when the gesture changes size.

### Issue 5
apps/desktop-gpui/src/editor_panels.rs:5687-5689
This new comment narrates how the controls work. `AGENTS.md` explicitly bans comments that merely describe the code and allows them only for non-obvious context. Remove this description. The same pattern appears above `KeyframeTarget` and `KEYFRAME_MARK`, and elsewhere in the added comments. This repository requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds position, size, and opacity keys for text and image overlays, with controls in the GPUI editor.

Text and image overlays animate between saved keyframes.
The overlay panels let editors manage keyframes at the playhead.
Canvas edits and timeline marks follow overlay animation.

_{Reviews (2) · Last reviewed commit: "Scan keyframes without sorting, mark onl..." · Reviewed by Greptile}

Text and image segments gain optional keyframes for position, size and opacity (text: position and opacity), in seconds from the segment start, eased between keys. The renderer samples them per frame. In the GPUI editor the text and image panels get a Keyframes section: add a key at the playhead, jump to or remove keys. Once a property is keyed, dragging on the canvas or moving the opacity slider edits the key under the playhead. Keys show as marks on the timeline boxes. Co-Authored-By: Claude Opus 5.5 <[email protected]>

YoavLevavi · 2026-10-21T15:48:22Z

@greptileai

Co-Authored-By: Claude Opus 5.5 <[email protected]>

vercel · 2026-10-21T16:11:22Z

@YoavLevavi is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

richiemcilroy · 2026-10-21T17:46:01Z

Thanks @YoavLevavi, this is a substantial feature, and the engine side is clean: OverlayKeyframes::sample is used the same way in prepare_texts, the image layer and the canvas, and segments without keys render exactly as before. A few things need fixing. 1) Keys are seconds from the segment start, but TextSegment and ImageSegment still use the plain impl_track_segment! (apps/desktop-gpui/src/editor_edits.rs:590 and :592). So a split clones the keys into the tail, which replays the animation from its own start, and dragging the start handle shifts every key in output time. Could you override trim_start_to and split_tail for both types to shift the keys by the change in start, the way #2829 does for mask keyframes, and add split/trim tests? 2) The canvas still checks the base segment.opacity: render_image_box filter at apps/desktop-gpui/src/editor_canvas.rs:1427, snap targets at :607, and the selection check at :1043. Once opacity is keyed the base value is frozen, so an image keyed 0 -> 1 from a base of 0 renders but can never be selected or dragged on the canvas. Please use segment.opacity_at(time) > 0. there. 3) Please add skip_serializing_if = "OverlayKeyframes::is_empty" to both keyframes fields (crates/project/src/configuration.rs:1422 and :1933), so existing projects save unchanged. The image serde test then will not need the empty object. Optional: the chips use ◆, which no other GPUI UI does. An svg icon would avoid font fallback differences on Windows and Linux.

- Text and image segments get their own TrackSegmentOps whose trim_start_to shifts the keys by the change in start (as masks do), so a start trim or the tail half of a split keeps the animation where it was in output time. Split and trim tests for both types. - The canvas selection, snap targets and the image box filter use opacity_at(time), so an image keyed up from a base opacity of 0 can be selected and dragged. - keyframes is skipped when empty, so existing projects save unchanged (attribute order keeps the generated TypeScript field optional); bindings regenerated and the image serde test is back to its original shape. - The keyframe chips and button use an svg diamond and x icon instead of text glyphs. Merges origin/main. Co-Authored-By: Claude Opus 5.5 <[email protected]>

YoavLevavi · 2026-10-25T15:49:52Z

Thanks @richiemcilroy, all three are in 3782cd2 (also merged main), plus the optional one:

TextSegment and ImageSegment now have their own TrackSegmentOps, whose trim_start_to shifts the keys by the change in start (as masks do). A start trim, or the tail of a split, keeps the animation in place in output time. There are split and trim tests for both types, and they also check that the end handle leaves keys alone.
The canvas snap targets, selection check and image box filter use opacity_at(time), so an image keyed up from a base opacity of 0 can be selected and dragged.
keyframes is skipped when empty, so existing projects save unchanged, and the image serde test is back to its original shape. One detail: specta reads skip_serializing_if after default as "required", so the attribute is written skip_serializing_if = ..., default to keep the generated field keyframes?:. Bindings are regenerated and tsc is clean.
The chips and the Add button use an svg diamond (and the svg x) instead of text glyphs.

@greptileai review

greptile-apps · 2026-10-25T15:56:58Z

+impl_keyed_overlay_segment!(TextSegment);
 impl_track_segment!(cap_project::StyleSegment, lane: track);
-impl_track_segment!(cap_project::ImageSegment, lane: track);
+impl_keyed_overlay_segment!(cap_project::ImageSegment);


Deleted clips leave keys behind

Deleting a clip does not retime keys on text or images that cross the removed interval. Both still use ripple_delete_track, which changes only their bounds. For an overlay spanning 10–20 seconds, deleting 12–14 seconds shortens it to 10–18, but a key originally at 18 stays there instead of moving to 16. Add key-aware ripple deletion for both overlay types.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/editor_edits.rs Line: 683-685 Comment: **Deleted clips leave keys behind** Deleting a clip does not retime keys on text or images that cross the removed interval. Both still use `ripple_delete_track`, which changes only their bounds. For an overlay spanning 10–20 seconds, deleting 12–14 seconds shortens it to 10–18, but a key originally at 18 stays there instead of moving to 16. Add key-aware ripple deletion for both overlay types. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+    #[serde(skip_serializing_if = "OverlayKeyframes::is_empty", default)]
+    pub keyframes: OverlayKeyframes,


Faster clips cut animations short

set_clip_segment_timescale retimes text and image bounds but leaves their new keys unchanged. Speeding a 10-second clip to 2× shortens a matching overlay to 5 seconds, but its final opacity key remains at relative time 10. The overlay disappears before reaching that value. Map each overlay key through the same change in time, as the mask path already does.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/project/src/configuration.rs Line: 1438-1439 Comment: **Faster clips cut animations short** `set_clip_segment_timescale` retimes text and image bounds but leaves their new keys unchanged. Speeding a 10-second clip to 2× shortens a matching overlay to 5 seconds, but its final opacity key remains at relative time 10. The overlay disappears before reaching that value. Map each overlay key through the same change in time, as the mask path already does. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+        relative_time: f64,
+    ) -> (XY<f64>, XY<f64>, f64) {
+        (
+            sample_vector_keys(&self.position, relative_time).unwrap_or(center),


Centering ignores animated positions

Once position keys exist, sample ignores changes to the base center. The existing “Center on canvas” button still writes only segment.center, so it stops moving an image after the user adds a keyframe. Switching keyed text to TextLayout::Fullscreen has the same problem. Route these position changes through the keys, or explicitly reset the position animation.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/project/src/configuration.rs Line: 1307 Comment: **Centering ignores animated positions** Once position keys exist, `sample` ignores changes to the base `center`. The existing “Center on canvas” button still writes only `segment.center`, so it stops moving an image after the user adds a keyframe. Switching keyed text to `TextLayout::Fullscreen` has the same problem. Route these position changes through the keys, or explicitly reset the position animation. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+                if segment.keyframes.size.is_empty() {
+                    segment.size = size;
+                } else {
+                    segment.keyframes.set_size(at, size);


Moving changes the size animation

Moving or nudging an image inserts a size key even when its size did not change. With size keys at 0 and 10 seconds, moving the image at 5 seconds inserts a third size key. Because each span eases separately, this changes the size animation before and after the move. Write a size key only when the gesture changes size.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/editor_canvas.rs Line: 1260-1263 Comment: **Moving changes the size animation** Moving or nudging an image inserts a size key even when its size did not change. With size keys at 0 and 10 seconds, moving the image at 5 seconds inserts a third size key. Because each span eases separately, this changes the size animation before and after the move. Write a size key only when the gesture changes size. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+                if segment.keyframes.position.is_empty() {
+                    segment.center = center;
+                } else {
+                    segment.keyframes.set_position(at, center);


New keys lack timeline marks

Keys added by dragging or nudging do not appear on the timeline. write_overlay_rect calls project_changed_live, which does not rebuild the stored TimelineModel; ending the drag only saves. The new marks read key times copied into that model, so they remain stale until another edit rebuilds it. Refresh those times when the canvas edit finishes.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/editor_canvas.rs Line: 1258 Comment: **New keys lack timeline marks** Keys added by dragging or nudging do not appear on the timeline. `write_overlay_rect` calls `project_changed_live`, which does not rebuild the stored `TimelineModel`; ending the drag only saves. The new marks read key times copied into that model, so they remain stale until another edit rebuilds it. Refresh those times when the canvas edit finishes. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+    let mut sorted: Vec<&T> = keys.iter().filter(|k| key_time(k).is_finite()).collect();
+    sorted.sort_by(|a, b| key_time(a).total_cmp(&key_time(b)));


Every frame sorts keys again

keyframe_span allocates and sorts a new list every time a property is sampled. A full sample repeats this for position, size and opacity on every frame, and opacity_at repeats all three just to read opacity. This adds avoidable work as overlays and keys grow. Find the surrounding keys without allocating, or keep a sorted list ready for sampling.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/project/src/configuration.rs Line: 1315-1316 Comment: **Every frame sorts keys again** `keyframe_span` allocates and sorts a new list every time a property is sampled. A full `sample` repeats this for position, size and opacity on every frame, and `opacity_at` repeats all three just to read opacity. This adds avoidable work as overlays and keys grow. Find the surrounding keys without allocating, or keep a sorted list ready for sampling. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-10-25T15:56:59Z

+        for time in keyframes.iter() {
+            let x = ((time / duration).clamp(0., 1.) as f32) * width;


Hidden keys appear at edges

Trimming and splitting deliberately retain keys outside the segment to preserve the animation. This clamp draws negative times as start marks and later times as end marks, showing keys where none exist. Skip marks outside the visible segment while keeping those keys for sampling.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/editor_timeline.rs Line: 2808-2809 Comment: **Hidden keys appear at edges** Trimming and splitting deliberately retain keys outside the segment to preserve the animation. This clamp draws negative times as start marks and later times as end marks, showing keys where none exist. Skip marks outside the visible segment while keeping those keys for sampling. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

greptile-apps · 2026-10-25T15:56:59Z

+    /// The Keyframes section of the text and image panels: add a key at the
+    /// playhead, then any move, resize or opacity change at another time adds
+    /// or edits a key there. Each key is a chip: click to jump, × to remove.


Comments repeat the code

This new comment narrates how the controls work. AGENTS.md explicitly bans comments that merely describe the code and allows them only for non-obvious context. Remove this description. The same pattern appears above KeyframeTarget and KEYFRAME_MARK, and elsewhere in the added comments. This repository requirement must be satisfied before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/editor_panels.rs Line: 5687-5689 Comment: **Comments repeat the code** This new comment narrates how the controls work. `AGENTS.md` explicitly bans comments that merely describe the code and allows them only for non-obvious context. Remove this description. The same pattern appears above `KeyframeTarget` and `KEYFRAME_MARK`, and elsewhere in the added comments. This repository requirement must be satisfied before merging. **Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md)) --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…fresh marks after canvas edits Co-Authored-By: Claude Opus 5.5 <[email protected]>

YoavLevavi · 2026-10-28T05:38:25Z

All four fixed in a06f4de:

Sorting. keyframe_span scans the keys with min_by/max_by instead of collecting and sorting a copy, with ties resolved the same way the stable sort did. The cap-project tests pass unchanged.
Stale marks. A canvas drag over a text or image overlay rebuilds the timeline model when it ends, and so does each keyboard nudge, so new keys get their marks right away.
Marks at the edges. Keys outside 0..=duration aren't drawn; they're still kept for sampling.
Comments. Removed the narrating ones on the Keyframes section, KeyframeTarget, KEYFRAME_MARK and the OverlayKeyframes/segment helpers.

@greptileai review

Include empty overlay keyframes in the image serde-defaults test

09ebd74

Co-Authored-By: Claude Opus 5.5 <[email protected]>

greptile-apps Bot reviewed Oct 25, 2026

View reviewed changes

Scan keyframes without sorting, mark only keys inside the segment, re…

a06f4de

…fresh marks after canvas edits Co-Authored-By: Claude Opus 5.5 <[email protected]>

		#[serde(skip_serializing_if = "OverlayKeyframes::is_empty", default)]
		pub keyframes: OverlayKeyframes,

		let mut sorted: Vec<&T> = keys.iter().filter(\|k\| key_time(k).is_finite()).collect();
		sorted.sort_by(\|a, b\| key_time(a).total_cmp(&key_time(b)));

		for time in keyframes.iter() {
		let x = ((time / duration).clamp(0., 1.) as f32) * width;

Conversation

YoavLevavi commented Oct 21, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 3/5

Findings

Summary

Uh oh!

YoavLevavi commented Oct 21, 2026

Uh oh!

vercel Bot commented Oct 21, 2026

Uh oh!

richiemcilroy commented Oct 21, 2026

Uh oh!

YoavLevavi commented Oct 25, 2026

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

YoavLevavi commented Oct 28, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

YoavLevavi commented Oct 21, 2026 •

edited by greptile-apps Bot

Loading