SetReplay: Correction-First Workout Intelligence

Corrections are a first-class control.

Exercise, set, and rep estimates can be wrong, so the interface is designed around fast correction before, during, and after a workout. User-entered targets remain distinct from what the tracker observes, and the final timeline stays editable instead of presenting an AI result as unquestionable truth.

• Live controls can correct the current exercise, set, and rep count.
• Rest, weight, RPE, and the final exercise timeline remain editable.
• Workout templates and consent-based screenshot or text import reduce repeated setup.

Immediate estimates, deeper review later.

Apple Vision extracts body-pose joints on device. Exercise-specific joint-angle and excursion rules propose live rep-completion events, with side-aware handling and manual corrections. These inspectable rules run locally; a separate queued Gemini video pass proposes post-workout exercises, sets, and reps with more context.

Queue the expensive work.

The native client and Cloudflare backend separate recording from analysis. Workout media is stored in R2, D1 holds durable state, and a Cloudflare Queue runs video analysis with retry and generation fences so stale or duplicate completion attempts cannot silently replace newer work.

Fix the details that block a workout.

Tester feedback led to fixes for weight entry and per-side targets in exercises such as Bulgarian split squats. Upload improvements persist confirmed multipart receipts across relaunch and retry pending work when the app returns to the foreground, while retaining the original recording until server finalization.

Video intelligence starts with permission.

Before the first upload, the app asks permission to store workout video with Cloudflare and share it with Gemini for analysis. Revocation stops future uploads and new analysis. Optional Apple Health matching remains on the iPhone, and the camera flow does not record audio.

Measure events, not just a final count.

The supporting evaluation harness compares video-model outputs and replays pose traces against event-level labels. The working criteria include completion precision and recall, false completions, missed reps, set boundaries, and visibility conditions; no accuracy claim is published before representative evaluation supports it.