Historic Fashion Archive — Research Assistant & Full-Stack Build
Related-item quality 0.70 → 0.93 nDCG@5, blind-judged" (recommended lead)

drag to turn · scroll to zoom
- Role
- Research Assistant, UVA Historic Fashion Archive (2024–present)
- Stack
- Next.js 16 (App Router), React 19, TypeScript 5, Tailwind CSS v4; Three.js / @react-three/fiber / @react-three/drei v10 for the 3D viewer
- Backend
- CollectiveAccess (Providence), custom profile on the Costume Core metadata standard; cookie + Basic Auth integration, 5-minute in-memory cache with static-JSON fallback
- Scope
- both the backend metadata schema and configuration and the public-facing museum site, end to end
- Recommendations (learned)
- a re-ranker over FashionCLIP image embeddings and extracted attributes replaced the original fixed-weight score (era 30%, type 25%, color 20%, material 15%, decade 10%). nDCG@5 rose from 0.70 to 0.93 (paired 95% CI +0.15 to +0.31) on 50 query garments with 731 blind relevance judgments, under nested cross-validation. Gains were largest for accessories (0.50 to 0.91), where one catch-all "accessory" type had made the old score close to random
- Search (deployed)
- keyword (BM25) and FashionCLIP text-to-image retrieval fused by reciprocal rank fusion, served by a CPU-only FastAPI service on a VPS behind a Cloudflare Tunnel (about 100 ms per query). nDCG@5 0.80 vs 0.68 for keyword search on 40 blind-judged queries; a local-LLM version (Qwen3-8B parsing queries into filters, with grounded, citation-checked answers) reached 0.88 but needs a GPU
- Attributes
- material, color and technique extracted from 1,400+ catalog descriptions (F1 0.95-0.99 against 200 hand-labeled records), and predicted from photos for undescribed garments by a linear probe trained on those weak labels (color F1 0.58 vs 0.39 zero-shot)
- Evaluation practice
- dev/test splits frozen before tuning, blind pooled judging, an intra-rater check (weighted kappa 0.82-0.87), a blind re-judge when one round of grades turned out non-blind, and GitHub Actions regression gates on a committed fixture
- ML stack
- Python, PyTorch, OpenCLIP (Marqo FashionCLIP), scikit-learn, FastAPI, Ollama, MLflow
- 3D viewer
- garments rendered from real photogrammetry scans (see garment-capture-pipeline.md); per-scan front-facing calibration; the home page is a walkable 3D museum with a wing per era (pre-1920 onward), first-person drag/WASD/click-to-walk controls and click-to-inspect garments
- Data pipeline
- filtering, search, analytics tracking, JSON/CSV/PDF export for the collection
- Security work
- HMAC-signed sessions, rate limiting, CSP headers, timing-safe auth checks, hardened from an initial shared-secret header into a real session system
- Live site
- uvahcc.org
The problem
Before this, the Archive's public-facing presence was 2D photography and text metadata — you could see a photo of a garment and read its catalog entry, but you couldn't actually examine it: turn it, compare its silhouette to another piece, or get a sense of how it moved or draped. That's a real limit for a fashion archive, where shape and construction are the point. I built this to close that gap: a 3D viewer built from real photogrammetry scans, a recommendation engine that surfaces related garments by actual visual and material attributes instead of shared tags, and a backend metadata schema to support all of it.
What I'm proudest of
What I'm proudest of is that this wasn't a frontend skin on someone else's backend — I designed the CollectiveAccess metadata schema, built the auth and caching layer, and built the public site on top of it, so I own the system end to end. It's also the first time I've worked on software inside a research context rather than a typical CRUD app, which changed how I thought about the problem: the "users" are curators and researchers who need the data model to reflect real museum cataloging practice (the Costume Core standard), not just whatever fields were convenient to build.
How I knew it worked
The honest answer is that I didn't trust any number until I'd tried to break it. There was no click data to learn from, so I built the ground truth myself: I hand-labeled 200 catalog descriptions, then judged 731 garment pairs blind, with candidates from every system pooled and shuffled so I couldn't tell which one had suggested what. Most of what I learned came from catching my own mistakes. My first text extractor scored almost perfectly, and it took a while to see why: I'd been cleaning the labels with a keyword checker, so the labels had quietly drifted toward exactly what keywords see. I froze a test set, stopped using the checker on it, and the real score turned out lower. When the learned re-ranker beat everything, I realized I'd graded its new picks knowing whose they were, so I re-graded them blind, mixed in with pairs I'd already judged. The result held, and the re-grade doubled as a consistency check on my own judging. The finding I didn't expect was the split: the old weighted score was already good for clothing and nearly random for accessories, because the catalog files fans, shoes and hats under one type. Photos fixed that, and the learned model combined both. The search service now runs the same code the benchmark measured, and a regression check fails the build if quality drops.
What's next
The most concrete next step is feeding the garment-capture pipeline's output back into CollectiveAccess — right now the 3D pipeline and the Archive site are connected by shared demo garments, but not yet by a real data pipeline between them. Beyond that, growing the number of garments in the 3D viewer past the initial pilot set is the other clear milestone.




