Skip to main content

Label & Review

Two pages, one continuous flow: configure a labeling run, then refine the output. SAM 3 produces the boxes, the human refines.

Label page​

Route: /label/:videoId or /label/collection/:projectId — Source: ui/src/pages/LabelPage.tsx

Configure and start a SAM 3 auto-labeling job. This is where text prompts become bounding boxes.

Label page

Workflow​

  1. Pick a video (or a whole collection — the route variant /label/collection/:projectId runs against every video in a dataset).
  2. Enter prompts, one per line. Each prompt becomes a class. Use natural phrases — SAM 3.1 is grounded by free text, not a closed vocabulary.
  3. Adjust threshold and resolution. 0.5 is a reasonable default; lower it for tiny objects, raise it to suppress false positives.
  4. Preview — runs SAM 3 on a handful of frames and returns base64 JPEGs so you can sanity-check the prompt before committing the run.
  5. Start labeling. The job is queued to the labeler worker and progress streams over WebSocket. Switch to Review as soon as the first frames complete.

Visual prompts​

Toggle to exemplar mode and draw a box around an example object on the first frame. SAM 3 finds visually similar objects across the video. Useful when text is ambiguous (e.g. "the kind of vehicle in this image, but not all vehicles").

Click-to-segment​

The fine-grained segmentation tool calls POST /api/v1/label/segment-points on every click. Useful for polishing individual annotations once review is underway.

Review page​

Route: /review/:jobId — Source: ui/src/pages/ReviewPage.tsx

The review canvas shows each frame in the labeling job with overlaid boxes. Accept, reject, edit, or redraw — every action PATCHes back to the API and updates the dataset that downstream training will consume.

Inspect the original video​

Filter saved findings by source clip and class, then select Inspect on a finding. The inspector keeps its saved frame and review controls beside the original video. Use the video controls to watch the surrounding event. This opens stored evidence; it does not run the model again.

Source timestamps seek to the recorded time. FPS-derived timestamps are labeled approximate. Unknown or invalid times do not trigger a seek. A saved frame mask stays on the frame; it is not extended across moving video. Sampled results do not establish full-video coverage or an object's geographic position.

Some containers expose a different browser timeline. The player checks that it can map the saved time into that timeline and verifies the seek. If timing is inconsistent or unavailable, it explains the limit and leaves playback manual.

If a playback link expires, the player makes one automatic renewal attempt. If playback still fails, use its retry control. The saved frame and annotation editor remain available. Image-only sources have no video. Browser codec support still applies, including for HEVC recordings.

Keyboard shortcuts​

KeyAction
J / KPrevious / next frame
SpaceToggle play
DDelete the highlighted box
Shift+dragDraw a new box
RReject the entire frame (excluded from export)
Cmd/Ctrl+ZUndo last edit
1–9Quick-assign class by index

Bulk operations​

  • Merge classes — POST /api/v1/annotations/merge-classes collapses two class names into one across the whole job (e.g. truck + lorry → truck).
  • Delete a class — drops every annotation of a class from the job.
  • Add a class — re-runs SAM 3 just for the new class without touching existing labels.
  • Duplicate a job — clone with the same prompts as a starting point for a new run.

Stats panel​

The right-hand drawer shows per-class counts, confidence histograms, and frames-per-class so you can spot imbalance early. Toggle the AI Insights tab to ask the agent for a written summary of the current job (see Agent).