OpenID

Small workbench for a visual search PoC as part of the IMT4126 course in Biometrics at NTNU Gjøvik.

Stack

• Next.js App Router for the web application
• TypeScript for the application and API layer
• Python for the model service
• Docker Compose for deployment

Application pages

• / Workbench
• /benchmark Benchmark runner for immediate baseline comparison and result visualization
• /feeds Continuous monitoring page for configured public feeds
• /dataset/images Image management
• /dataset/videos Video management

Search flow

The application uses a service-backed search pipeline:

1. The Next.js API loads the benchmark metadata from disk.
2. The Python model service computes reference embeddings for each identity.
3. A user can search by:
   • public name or subject alias
   • uploaded reference image
   • watched live feed snapshot URLs
   • a weighted mix of both
4. Feedback from the training panel promotes or suppresses identities on subsequent searches.

The current model service uses a lightweight visual embedding pipeline that is compatible with the dataset and container-friendly. It is isolated behind a dedicated service so it can later be replaced by a stronger pretrained model without rewriting the app.

Current integrated retrieval models:

• handcrafted_v1 original histogram-and-edge baseline
• spatial_hybrid_v2 stronger multi-region comparison descriptor
• center_focus_v1 center-cropped comparison descriptor with less background influence
• conservative_hybrid_v3 center-weighted descriptor with stricter open-set behavior for unrelated imagery and the current default in the app
• openclip_vit_b32_v1 pretrained OpenCLIP baseline
• insightface_buffalo_l_v1 pretrained InsightFace face-embedding baseline

OpenCLIP and InsightFace both download model weights on first use. They are subsequently cached.

Identity labels

Only identity 1 retains a verified public name from the original material. The remaining entries use stable aliases such as Subject 014 because the source footage does not provide reliable verified names for every person. This keeps the benchmark usable without fabricating unsupported civil identity claims.

Each identity record includes:

• display_name
• sex
• approx_age
• reference_images
• reference_source
• notes

The dataset contains more than thirty reference files in total.

Run with Docker Compose

The primary development and demo path is Docker Compose:

docker compose up --build

This starts:

• openid on port 3000
• openid_model on the internal compose network at port 8000

Then open:

http://localhost:3000

The dataset directories are mounted into both containers at runtime, so the build does not copy the full media corpus into the Docker image.

The model container now starts fast with only the core Python runtime installed in the image. Heavy pretrained dependencies are bootstrapped into the mounted model_cache volume in the background after startup instead of blocking docker compose build.

Evaluation

The repository now includes a baseline evaluation script:

.venv-model/bin/python scripts/evaluate_benchmark.py

It currently runs a two-protocol benchmark suite and writes reports to:

• reports/evaluation/benchmark_results.json
• reports/evaluation/benchmark_results.md

Protocols:

• Reference Hold-Out leave-one-reference-out retrieval over the indexed gallery
• Target Videos + Distractors one sampled frame per target video plus 50 distractor clips for open-set false-positive pressure

Current measured baselines:

• handcrafted_v1
• spatial_hybrid_v2
• center_focus_v1
• conservative_hybrid_v3

Latest saved results:

• Reference Hold-Out
  • handcrafted_v1: Top-1 0.667, Top-5 0.667, MRR 0.714
  • spatial_hybrid_v2: Top-1 0.667, Top-5 0.667, MRR 0.690
  • conservative_hybrid_v3: Top-1 0.333, Top-5 0.667, MRR 0.462
• Target Videos + Distractors
  • handcrafted_v1: Top-1 0.800, Top-5 0.933, MRR 0.846, negative rejection 0.240
  • spatial_hybrid_v2: Top-1 0.867, Top-5 1.000, MRR 0.925, negative rejection 0.640
  • conservative_hybrid_v3: Top-1 0.733, Top-5 0.900, MRR 0.829, negative rejection 1.000

Important protocol note: only 3 of the 32 reference queries are currently eligible under leave-one-out, because most identities only have one reference image. The benchmark UI and reports surface this explicitly.

The most important current finding is the tradeoff between retrieval and safety:

• spatial_hybrid_v2 is the strongest measured retrieval baseline on target clips
• conservative_hybrid_v3 is the safest measured open-set baseline on distractors

OpenCLIP is also integrated as a stronger pretrained comparison model for local evaluation, but the latest saved report focuses on the three faster baselines above so iteration stays practical.

Run without Docker

To run without Docker:

npm install
npm run dev

In a separate shell, start the model service:

.venv-model/bin/pip install -r model/requirements.txt
.venv-model/bin/uvicorn model.app.main:app --host 0.0.0.0 --port 8000

Then open:

http://localhost:3000