SignLanguageDetector

From the README

Sign Language Detector

End-to-end sign language recognition — collect images, train a classifier, and serve real-time browser inference with model analytics, all from one project.

Features

• Real-time browser inference — webcam → MediaPipe hand landmarks → RandomForest → prediction, no plugins
• Interactive training CLI — guided flow for image collection, dataset creation, training, and reporting
• Model analytics sidebar — F1 scores, top/weak classes, accuracy rendered from JSON reports
• Headless REST API — POST /api/predict accepts file upload or base64 JSON, returns prediction
• Hot model switching — swap .pkl files from the UI dropdown without restarting the server
• One-command setup — uv run app provisions Python, creates venv, installs deps, and starts the server

Quick Start

Requires uv — install once with pip install uv or irm https://astral.sh/uv/install.ps1 | iex on Windows.

git clone https://github.com/Life-Experimentalist/SignLanguageDetector.git
cd SignLanguageDetector
uv run app

Open http://localhost:5000. uv run app auto-syncs the environment on first run — no manual pip install needed.

Commands

All commands run inside the managed uv environment — no activation needed.

Training a Model

Run uv run train and follow the prompts. Each stage is optional — skip any you've already completed.

Or run individual stages directly:

uv run python training/collect_imgs.py
uv run python training/create_dataset.py
uv run python training/train_classifier.py
uv run convert-reports

Inference API

Start the server (uv run app), then call POST /api/predict from any HTTP client.

Multipart file upload:

$form = @{
  image          = Get-Item .\data\0\sample.jpg
  show_landmarks = "false"
  include_visuals = "false"
}
Invoke-RestMethod -Uri "http://127.0.0.1:5000/api/predict" -Method Post -Form $form

Base64 JSON:

$bytes   = [System.IO.File]::ReadAllBytes(".\data\0\sample.jpg")
$payload = @{
  image_base64   = [System.Convert]::ToBase64String($bytes)
  show_landmarks  = $false
  include_visuals = $false
} | ConvertTo-Json

Invoke-RestMethod -Uri "http://127.0.0.1:5000/api/predict" `
  -Method Post -ContentType "application/json" -Body $payload

Response:

{
  "prediction": "A",
  "brightness": 142.5,
  "contrast": 68.3,
  "low_brightness": false,
  "model": "model.pkl"
}

Optional flags: show_landmarks (bool, default false), include_visuals (bool, default false).

Configuration

Copy .env.example to .env and adjust as needed. All variables have sensible defaults.

Project Structure

SignLanguageDetector/
├── app.py                   # Single-client Flask server
├── app_multi_client.py      # Multi-client Flask server (session isolation)
├── interactive_cli.py       # Training pipeline CLI
├── pyproject.toml           # Dependencies + uv run scripts
├── .env.example             # Environment variable reference
├── training/                # collect_imgs, create_dataset, train_classifier, convert_model_reports
├── utils/                   # config, app_utils, utils, scripts
├── models/                  # Trained .pkl files + .txt/.json reports
├── templates/               # Jinja2 HTML templates (layout, index, quiz, quiz_demo, sidebar)
├── static/css/              # model-info.css
└── docs/                    # GitHub Pages landing site

GitHub Pages

The landing page at https://sign.vkrishna04.me is served from the docs/ folder.

• Entry: docs/index.html
• Styles: docs/styles.css
• Stats: docs/landing.js
• SEO: docs/robots.txt, docs/sitemap.xml
• Custom domain: docs/CNAME → sign.vkrishna04.me
• Branding assets (optional): docs/static/branding/ — see docs/BRANDING_PROMPTS.md

Documentation

License

Apache License 2.0. See LICENSE.md.

Anonymous telemetry (a single view-counter ping per page load) is sent via CFlair-Counter. Set DISABLE_ANONYMOUS_TELEMETRY=true in .env to opt out. Training quality matters — better capture conditions produce better predictions. The dataset builder automatically skips frames without detectable hand landmarks and frames flagged as too blurry.