Skip to main content

Architecture Overview

Waldo's API and workers share a Postgres database, Redis broker, and MinIO object store. Linux workers run in containers; make up on macOS runs labeler and trainer workers natively for MLX/MPS access.

┌─────────────────┐
│ Browser │
└────────┬────────┘
│ HTTPS
┌────────▼────────┐
│ FastAPI app │ ── REST + WebSocket
│ (uvicorn :8000)│
└─┬──────┬──────┬─┘
writes │ │ │ enqueues
┌─────▼──┐ │ │
│Postgres│ │ ┌──▼────┐
└────────┘ │ │ Redis │
│ └──┬────┘
reads │ │ Celery tasks
┌────────────▼┐ ┌──▼────────────┐
│ MinIO │◄─┤ labeler / │
│ (S3 store) │ │ trainer │
└─────────────┘ │ workers │
└───────────────┘

Services​

ServiceImagePurpose
apppython:3.11-slim + uvFastAPI HTTP/WebSocket API. Stateless.
labelerCUDA/CPU container on Linux; native process on macOSCelery worker running SAM 3 / SAM 3.1 inference.
trainernvidia/cuda (GPU) or localCelery worker running YOLO26 training.
postgrespostgres:16-alpinePrimary store for users, projects, jobs, annotations, models.
redisredis:7-alpineCelery broker + WebSocket pubsub + ephemeral cache.
miniominio/minioS3-compatible blob storage for videos, frames, and exported datasets.

Request flow: auto-labeling​

  1. Upload — the browser POSTs a video to /api/v1/upload. The app stores it in MinIO and inserts a Video row.
  2. Labeling job — POST /api/v1/label creates a LabelingJob row and enqueues inference.
  3. Processing — the PyTorch path extracts sampled frames with FFmpeg; the native MLX path reads sampled video frames directly. Current task dispatch selects MLX for project jobs without checking platform, so Linux collection labeling needs correction.
  4. Streaming — the labeler publishes detections to a Redis pubsub channel as it goes. The app forwards them over WebSocket so the UI updates live.
  5. Review — the user opens /review/<job>. Annotations are loaded from Postgres, edits PATCH back to the API.
  6. Export — clicking "Export" generates a YOLO-format dataset (images + label txt files) into MinIO. The download endpoint streams it back to the browser.

Tech choices​

ConcernChoiceWhy
APIFastAPIAsync, type-checked, generates OpenAPI for free
ORMSQLAlchemy 2.xMature, async-friendly, alembic migrations
Task queueCelery + RedisBattle-tested for long-running ML jobs
Object storeMinIOS3-compatible, runs anywhere, no vendor lock-in
Detection modelYOLO26 (Ultralytics)Fast, accurate, easy to fine-tune
Segmentation modelSAM 3 / SAM 3.1 MLXPyTorch video sessions or MLX image detection with IoU tracking, depending on path
UIReact + Vite + TailwindFast HMR, modern hooks, zero config
AuthJWT bearer + API keysUser authentication; workspace authorization remains incomplete

See Data Model for the schema and Security for the trust model.