Skip to content

perf: faster Instant processing and early playback for everyone - #3530

Merged
richiemcilroy merged 8 commits into
mainfrom
perf/instant-processing-speed
Oct 29, 2026
Merged

richiemcilroy merged 8 commits into
mainfrom
perf/instant-processing-speed

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Makes an Instant recording reach its processed, verified MP4 sooner after Stop, and lets the owner's stopped tab start playing straight away for everyone.

flowchart LR
  subgraph before[media server, before]
    direction LR
    A1[download] --> B1[remux] --> C1["packet proof<br/>(SHA-256 per packet)"] --> D1[chunk decode] --> E1[PUT] --> F1["GET whole object<br/>+ SHA-256"] --> G1[upload previews]
  end
  subgraph after[media server, after]
    direction LR
    A2[download] --> B2[remux] --> C2["packet proof (murmur3)<br/>‖ chunk decode<br/>‖ SHA-256 + MD5 in own process"] --> E2["PUT ‖ previews"] --> F2["ETag == local MD5?<br/>yes: done · no: GET + SHA-256"]
  end
Loading

Media server (/video/mux-segments)

  • Upload proof without the read-back. hashRecordingUpload (apps/media-server/src/lib/recording-verification.ts) computes the SHA-256 plus the ETag S3 returns for the same bytes: "md5" for a single PUT, "md5(part md5s)-N" for multipart at the upload's own part size (apps/media-server/src/lib/upload-digest.ts). When the stored object's ETag (read with If-Match, size checked) equals it, the whole-object GET + hash is skipped. Any other ETag (SSE-KMS, Drive, other providers) still re-reads every byte exactly as before.
  • Hashing off the event loop. The output hash now runs in its own bun process (same pattern as recording-audio-timing.ts), so it no longer takes turns with the decoder output the verification parses on the main thread.
  • Packet proof uses murmur3 for video. readTrack (apps/media-server/src/lib/recording-packet-proof.ts) compared source and output packets by FFmpeg's plain-C SHA-256, the slowest read in a long recording's proof. Both sides are hashed by the same ffprobe and only compared with each other, so video packets now use murmur3 (128-bit, ~4x faster). Audio keeps SHA-256 because its tail is checked against a hash computed outside ffprobe.
  • Proof and decode run together. verifyRemuxedRecording starts the fragment decode at the same time as the packet proof. The decode only reads the proof for its final count and timing checks; a failed proof still wins (a mismatch is thrown, the decoders are stopped), and a proof error still falls back to the full source inspection.
  • Fragment decode starts during the remux. The chunk decode reads only the downloaded fragments, so it starts while the remux writes the output (apps/media-server/src/routes/video.ts). The packet proof and the audio decode still wait for the finished output, a failed remux still fails the job as a remux error and stops the decoders before cleanup, and the decoder stall timer ignores the wait for the output. The output probe runs during verification.
  • Previews upload beside the recording. Thumbnail and preview GIF go to this attempt's own candidate keys, which nothing reads before the recording is published, so they upload while the recording uploads and is checked. The job still completes only after both.

Fixture runs on a g6.4xlarge (same type and env as the media workers), main and this branch alternating, medians of 3. Totals are the media server's own time from job start to the completion webhook.

Recordings prepared with the desktop's 30 s preverification:

Fixture main total PR total main verify PR verify main upload + check PR upload + check
60 s real recording 2.20 s 1.44 s 1.25 s 0.81 s 0.52 s 0.26 s
5 min 720p 2.35 s 1.63 s 1.18 s 0.66 s 0.51 s 0.37 s
13 min 720p 3.87 s 2.80 s 1.96 s 1.31 s 0.73 s 0.26 s
28 min 720p 6.96 s 5.19 s 3.40 s 2.66 s 1.31 s 0.31 s

Recordings with nothing preverified (clients that do not prepare, Drive and custom buckets):

Fixture main total PR total main verify PR verify
60 s real recording 2.23 s 1.49 s 1.28 s 0.83 s
5 min 720p 4.11 s 2.93 s 2.75 s 1.97 s
13 min 720p 8.48 s 6.45 s 6.37 s 4.74 s

Web

  • One listing per commit step (apps/web/lib/desktop-recording-source.ts). Plan, enumerate and copy share one bucket listing; nothing in those phases writes a copy before the copy phase does, and the only other writer (segment preparation) at worst leaves a copy unlisted and it is copied again. Verification still lists afresh, since it is what proves the originals did not change. Four listing rounds per commit become two.
  • Preverified run listing overlaps the inventory read in dispatchDesktopRecordingJob (apps/web/workflows/finalize-desktop-recording.ts).
  • Early playback for everyone (apps/web/lib/early-instant-playback.ts). The stopped owner tab plays the uploaded prefix while the tail uploads instead of waiting for the whole upload. CAP_EARLY_INSTANT_PLAYBACK=disabled turns it off; any other value (including the current owner list) now means on.

Source commit against real S3 from us-east-1, main and this branch alternating, 16 rounds each:

Recording main p50 PR p50
1 min 444 ms 356 ms
5 min 441 ms 378 ms

Rollout

  • Web and media server are independent: an old media server ignores nothing new, and a new one only changes its own work.
  • After deploy, CAP_EARLY_INSTANT_PLAYBACK can be removed from the production env.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable new issues were found.

Summary

This PR overlaps recording checks and uploads to reduce the wait after Stop.

  • Instant recording checks and uploads run alongside each other.
  • Matching upload tags let the server skip reading the whole MP4 back.
  • Desktop recording commits reuse bucket listings across their phases.
  • Stopped Instant recordings start playback before the upload finishes.

No actionable new issues were found. The merge bases differ, so the review covered the full PR diff. No previous review threads were supplied, and no runtime tests were run.

Reviews (3) · Last reviewed commit: "Merge branch 'main' into perf/instant-pr..." · Reviewed by Greptile

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Oct 29, 2026 11:08am UTC

@vercel
vercel Bot temporarily deployed to Preview – cap-status October 29, 2026 02:38 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status October 29, 2026 11:08 Inactive
@richiemcilroy
richiemcilroy marked this pull request as ready for review October 29, 2026 11:15
@richiemcilroy
richiemcilroy merged commit f65cb86 into main Oct 29, 2026
47 checks passed

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — 937d8963 Deployed Oct 29, 2026 by vercel[bot]
Preview – cap-status — 937d8963 Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant