WiFall Watch — Privacy-First Fall Detection via WiFi + LLM Reasoning

WiFall Watch detects elderly falls using WiFi Channel State Information (CSI) — zero cameras, zero wearables — with an LLM reasoning layer on top that decides whether an event actually needs a human response.

Built for the AMD Developer Hackathon: ACT II (lablab.ai), Track 3 (Unicorn Track), by team Ocarina with Downtime.

The idea

WiFi signals bend, reflect and scatter around a human body as it moves. A CNN-LSTM model trained on CSI time series learns to tell a fall apart from lying down, sitting, walking or a normal transition — without a single frame of video or any device worn by the person being monitored.

The differentiator versus academic CSI research: before anything reaches a human, an LLM reasoning layer (Gemma via Fireworks AI) looks at the detected event plus context — time of day, how long the person has been immobile, recent activity history, the elderly person's profile — and decides:

IGNORE — not worth surfacing
OBSERVE — log it, keep watching
ALERT — generate an emergency summary for responders and notify the family caregiver

Safety guardrail (this is WiFall Watch's core thesis): the LLM reasons, but safety never depends on it. A high-confidence fall combined with long immobility always triggers an alert, even if the LLM call fails, times out, or returns garbage.

Architecture

[Public CSI datasets / OpenWrt router sensor]
        │
        ▼
[1. Data Pipeline]        CSI preprocessing: denoising, normalization,
        │                 sliding window, PCA
        ▼
[2. Detection Model]      CNN-LSTM · PyTorch + ROCm · trained on AMD Instinct GPUs
        │                 classes: fall / lying / idle / transition / walking
        ▼
[3. Inference Service]    FastAPI consuming the CSI stream (replay or live)
        │
        ▼
[4. Reasoning Layer]      Gemma via Fireworks AI — event + context in,
        │                 {decision, confidence, justification, summary} JSON out
        ▼
[5. Alert Engine]         Safety guardrail + AI summary + Twilio SMS/call to caregiver
        │
        ▼
[6. Dashboard]            Next.js — live signal chart, activity timeline,
                          elderly status card, LLM decision log, demo scenario buttons

Everything ships as Docker Compose services. Inference runs on CPU for the public demo — only model training needs the GPU.

Tech stack

Layer	Tech
Training	PyTorch + ROCm on AMD Instinct GPUs (AMD Developer Cloud)
Detection model	CNN-LSTM on WiFi CSI time series (UT-HAR dataset, test acc 94%)
Reasoning	Gemma 3 27B via Fireworks AI, behind a swappable abstracted client
Backend	Python 3.12, FastAPI
Notifications	Twilio SMS + voice call to caregiver
Frontend	Next.js 14 / React
Infra	Docker, Docker Compose

Repo structure

/model        CSI preprocessing, CNN-LSTM architecture, training script, exported model
/api          FastAPI backend: inference service, Gemma client, alert engine, Twilio notifier
/dashboard    Next.js monitoring dashboard
/infra        OpenWrt router sensor package (zero-hardware deployment)
/data         Dataset documentation, download script, pre-recorded demo scenarios
/docs         API contract, architecture notes, evaluation charts, brand assets

Quick start — local

Requirements: Docker 24+, Docker Compose v2, Git, Make.

git clone https://github.com/leonardogouvea/ai-hackathon.git wifallwatch
cd wifallwatch

# 1. Configure environment
cp .env.example .env
# Edit .env and set FIREWORKS_API_KEY (required for Gemma reasoning)
# See "Environment variables" section below for all options

# 2. Build images (first time only)
make build

# 3. Start all services with hot-reload
make dev

Dashboard → http://localhost:3000
API docs (Swagger UI) → http://localhost:8000/docs

The first make build takes a few minutes (downloading base images). After that, make dev starts instantly. In dev mode, editing any file under api/app/ reloads the API in under 1 second — no rebuild needed.

Common commands

Command	What it does
`make dev`	Start all services with hot-reload (development)
`make build`	Build Docker images
`make rebuild`	Full rebuild without cache (use after changing `requirements.txt`)
`make down`	Stop and remove containers
`make logs`	Stream API logs
`make logs-all`	Stream logs from all services
`make shell`	Open a bash shell inside the running API container
`make test`	Run pytest
`make lint`	Run ruff linter

Running a demo scenario

Once everything is up, open http://localhost:3000 and click one of the two demo buttons:

Simulate fall — plays a labeled real-fall CSI clip through the full pipeline. The system detects the fall, calls Gemma to reason about it, and fires an ALERT with an emergency summary. If Twilio is configured, the caregiver receives an SMS and a voice call.
Simulate false positive — plays a fast sit-down clip the model correctly classifies as non-fall. Gemma agrees: IGNORE. No alert fires.

Stopping

make down

Environment variables

Copy .env.example to .env and set the values below. Only FIREWORKS_API_KEY is required.

Variable	Required	Default	Description
`FIREWORKS_API_KEY`	Yes	—	Fireworks AI API key for Gemma. Get one at app.fireworks.ai → API Keys
`FIREWORKS_MODEL`	No	`accounts/fireworks/models/gemma-3-27b-it`	Gemma model ID. Change to swap models without touching code
`GUARDRAIL_FALL_CONFIDENCE`	No	`0.85`	Min confidence (0–1) to trigger the safety guardrail
`GUARDRAIL_IMMOBILITY_SECONDS`	No	`30.0`	Min immobility duration (seconds) to trigger the safety guardrail
`TWILIO_ACCOUNT_SID`	No	—	Twilio account SID. Without this, caregiver alerts are logged only
`TWILIO_AUTH_TOKEN`	No	—	Twilio auth token
`TWILIO_FROM_NUMBER`	No	—	Twilio sender number in E.164 format (e.g. `+15005550006`)
`NEXT_PUBLIC_API_URL`	No	`http://localhost:8000`	API base URL used by the dashboard. Set to the public URL for remote deploys

Getting a Fireworks AI key (Gemma)

Go to app.fireworks.ai and create an account
Navigate to API Keys → Create API Key
Copy the key and paste it as FIREWORKS_API_KEY in your .env
The hackathon grants all participants $50 in Fireworks credits automatically

Training on AMD Developer Cloud

The CNN-LSTM model was trained on AMD Instinct GPUs via AMD Developer Cloud (DigitalOcean infrastructure). This section shows how to reproduce the training run.

1. Provision a GPU droplet

Log in to amd.digitalocean.com
Create a new droplet:
- Image: rocm/pytorch (pre-installed ROCm + PyTorch — search in Marketplace)
- Size: GPU droplet with AMD Instinct MI210 or MI250 (1× GPU is sufficient)
- Region: any
SSH into the droplet:

ssh root@<droplet-ip>

2. Sanity-check ROCm + PyTorch

rocm-smi                        # should list the AMD GPU
python3 -c "import torch; print(torch.cuda.is_available())"   # should print True
python3 -c "import torch; print(torch.cuda.get_device_name(0))"

3. Clone the repo and download the dataset

git clone https://github.com/leonardogouvea/ai-hackathon.git wifallwatch
cd wifallwatch

pip install -r model/requirements-train.txt

# Download UT-HAR dataset (~200 MB)
bash data/download_dataset.sh

4. Run training

python model/train.py
# Checkpoints saved to model/exports/ every epoch
# Expected: val_acc > 0.95 by epoch 10, test_acc ~0.94

Training takes ~15 minutes on MI210 for 20 epochs.

Cost discipline: stop/destroy the droplet immediately after training. The model checkpoint is small — copy it back before destroying.

5. Copy the model back to your machine

# From your local machine
scp root@<droplet-ip>:~/wifallwatch/model/exports/cnn_lstm.pt ./model/exports/cnn_lstm.pt
scp root@<droplet-ip>:~/wifallwatch/model/exports/normalizer.npz ./model/exports/normalizer.npz
scp root@<droplet-ip>:~/wifallwatch/model/exports/pca.pkl ./model/exports/pca.pkl

The API Dockerfile copies these files into the container at build time.

Production deployment

To run WiFall Watch on a remote server (VPS, cloud VM, etc.):

1. Set the public API URL

The dashboard is a Next.js app. The API URL is baked in at build time, so you must set NEXT_PUBLIC_API_URL to your server's public address before building the image.

# On your server — .env
FIREWORKS_API_KEY=your_key_here
NEXT_PUBLIC_API_URL=https://api.yourserver.com   # or http://<ip>:8000
TWILIO_ACCOUNT_SID=ACxxx...
TWILIO_AUTH_TOKEN=xxx...
TWILIO_FROM_NUMBER=+15005550006

2. Build and start

git clone https://github.com/leonardogouvea/ai-hackathon.git wifallwatch
cd wifallwatch
cp .env.example .env    # fill in the values above

make build
make prod               # detached, no hot-reload

Services:

API → http://<server>:8000
Dashboard → http://<server>:3000

3. Expose ports (firewall)

Open ports 8000 and 3000 in your server's firewall / security group. For a cleaner setup with HTTPS, put both services behind an nginx reverse proxy.

4. Check health

curl http://<server>:8000/health
# {"status":"ok","version":"0.1.0","timestamp":"..."}

Demo

The demo runs entirely from pre-recorded CSI replay — never live capture — via two buttons on the dashboard:

Simulate fall — a labeled real-fall CSI clip through the full pipeline
Simulate false positive — a fast sit-down the system correctly does not alert on

Links:

Video walkthrough: coming soon
Live public demo: coming soon
Slides (PDF): coming soon

Documentation

/docs/api-contract.md — REST/SSE API + Gemma JSON contract
/data/README.md — dataset choice (UT-HAR), rationale and license
/infra/openwrt/README.md — router-native deployment guide

Roadmap — Zero-Hardware Deployment

The hackathon version can run a pre-recorded replay on any laptop. The product roadmap eliminates the need for extra hardware entirely for most customers:

Customer opens the WiFall Watch app
        │
        ├── "Checking your router..."
        │
        ├── Router IS compatible (OpenWrt + CSI-capable chipset)
        │         └── App installs WiFall Watch directly on the router via SSH
        │             Zero extra hardware purchased — 2-minute setup
        │
        └── Router is NOT compatible
                  └── "We recommend the WiFall Watch Hub ($49)"
                      Pre-configured device, plug-and-play

Why this works: WiFi CSI is captured by the router's own chip — the signal already travels through the room. On routers running OpenWrt (TP-Link, GL.iNet and 1,000+ compatible models), a software package exposes that CSI data to WiFall Watch's inference engine running locally on the router itself. No extra hardware, no cloud dependency, full privacy — the caregiver app just receives the alert.

Dual revenue model:

SaaS subscription for all users (monitoring + caregiver alerts)
Hardware sales for the ~40% of homes with incompatible routers

Router compatibility API, OpenWrt CSI module and auto-install are already implemented in this repo — see /infra/openwrt and POST /router/check.

Honest limitations

We'd rather say this upfront than have it discovered:

Environment sensitivity. CSI models trained in one room degrade in another. Roadmap answer: per-home automatic calibration (out of scope for the hackathon submission).
Not a medical device. This is an assistive alert system. It makes no diagnostic claims and is not a substitute for medical monitoring.

Team

Member	Role
Leo — @leonardogouvea	Backend, ML pipeline, model training, Gemma/Fireworks integration, alert engine
Fernando — @FernandSa	AMD Developer Cloud, Docker, deployment, CI, README
João — @jpsoaresXy	Dashboard (Next.js), visual identity, demo UX

Team Ocarina with Downtime on lablab.ai.

License

MIT — see LICENSE.

Navigation Menu

Repository files navigation

WiFall Watch — Privacy-First Fall Detection via WiFi + LLM Reasoning

The idea

Architecture

Tech stack

Repo structure

Quick start — local

Common commands

Running a demo scenario

Stopping

Environment variables

Getting a Fireworks AI key (Gemma)

Training on AMD Developer Cloud

1. Provision a GPU droplet

2. Sanity-check ROCm + PyTorch

3. Clone the repo and download the dataset

4. Run training

5. Copy the model back to your machine

Production deployment

1. Set the public API URL

2. Build and start

3. Expose ports (firewall)

4. Check health

Demo

Documentation

Roadmap — Zero-Hardware Deployment

Honest limitations

Team

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
api		api
dashboard		dashboard
data		data
docs		docs
infra		infra
model		model
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
docker-compose.override.yml		docker-compose.override.yml
docker-compose.yml		docker-compose.yml
pyproject.toml		pyproject.toml

Latest commit

History

Folders and files

Repository files navigation

WiFall Watch — Privacy-First Fall Detection via WiFi + LLM Reasoning

The idea

Architecture

Tech stack

Repo structure

Quick start — local

Common commands

Running a demo scenario

Stopping

Environment variables

Getting a Fireworks AI key (Gemma)

Training on AMD Developer Cloud

1. Provision a GPU droplet

2. Sanity-check ROCm + PyTorch

3. Clone the repo and download the dataset

4. Run training

5. Copy the model back to your machine

Production deployment

1. Set the public API URL

2. Build and start

3. Expose ports (firewall)

4. Check health

Demo

Documentation

Roadmap — Zero-Hardware Deployment

Honest limitations

Team

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages