Skip to content

architecture.md

Latest commit

 

History

History
107 lines (81 loc) · 5.51 KB

architecture.md

File metadata and controls

107 lines (81 loc) · 5.51 KB
Raw

DIRA System Architecture & Refactor Plan

This document outlines the system architecture of the DIRA (Thermodynamic Economic Resilience Engine), details the design principles of its four main pillars, describes the data access layer, and outlines future refactoring plans.

🏗️ High-Level System Architecture

DIRA is built using a modern, multi-tier decoupled structure. The three main components communicate over secure REST APIs and structured data models.

graph TB
    subgraph Client Layer
        Web[React PWA / Dashboard]
        WA[WhatsApp Chat Client]
    end

    subgraph Interface / Bridge Layer
        Nginx[Nginx Reverse Proxy & SSL]
        Bridge[WhatsApp Bridge Node.js + Baileys]
    end

    subgraph Backend Core
        API[FastAPI Web Framework]
        Scheduler[Hive Mind APScheduler]
        Gemini[Google Gemini API]
    end

    subgraph Storage Layer
        PG[(PostgreSQL Database)]
    end

    %% Routing Flow
    Web -->|HTTPS| Nginx
    WA -->|WebSocket/Chat| Bridge
    Bridge -->|API Actions| Nginx
    Nginx -->|Route /api| API
    API -->|Intent Extraction| Gemini
    API -->|Read/Write| PG
    Scheduler -->|Read/Write| PG
Loading

⚡ Core Architecture: The Four Pillars

DIRA is designed around four pillars that provide telemetry and resilience for informal micro-economies (like campus vendors at Pwani University).

1. Ambient WhatsApp Node (whatsapp_bridge/ & ambient.py)

  • Purpose: Low-friction data entry. Informal vendors do not have time to update complex inventory apps. Instead, they interact via WhatsApp.
  • Workflow:
    1. The WhatsApp Bridge intercepts raw chats in vendor groups or 1:1 sessions.
    2. Chats are sent to the backend /api/ambient/process endpoint.
    3. The backend uses the Google Gemini API (via Pydantic structured output) to extract the user's intent.
    4. Intent is classified into: ORDER, OUT_OF_STOCK, DEBT_PROMISE, CASUAL, or INQUIRY.
    5. Detected items, quantities, and estimated values are saved as ambient_chat_logs.

2. M-Pesa Reconciliation Engine (mpesa.py)

  • Purpose: Financial tracking and automatic debt/order reconciliation.
  • Workflow:
    1. Micro-economy vendors process most transactions via M-Pesa.
    2. The frontend allows vendors to upload their raw M-Pesa statements or SMS streams.
    3. The backend parses transaction data (extracting Receipt No, Sender, Amount, Transaction Type, and Timestamp).
    4. The engine searches for matching pending ambient_chat_logs (e.g. an unpaid order matching the sender and amount) and automatically upgrades their status from PENDING_PAYMENT to RECONCILED.

3. Thermodynamic Kinetic Sovereignty Scorer (vendors.py / dashboard.py)

  • Purpose: Simulating and measuring economic resilience under cash-flow shocks (e.g., student Higher Education Loans Board - HELB delays).
  • Core Metrics:
    • Kinetic Sovereignty (KS): Measures financial velocity and the capacity of the vendor to maintain operational momentum without external credit. A score $\ge 1.0$ is resilient, while a score approaching $0.0$ signifies imminent collapse.
    • Base Entropy: Measures operational friction and overhead costs.
    • Daily Sustenance & Fixed Monthly Costs: Dynamic inputs used to calculate thermodynamic recovery timeframes.

4. Hive Mind Supply-Gap Router (hive_mind.py)

  • Purpose: Autonomous coordination. If one vendor flags an item as OUT_OF_STOCK or if demand signals identify an unmet need, the system creates a SupplyGap.
  • Workflow:
    1. An asynchronous background scheduler runs every 30 minutes.
    2. It aggregates unfulfilled demand patterns by location cluster (e.g., Pwani_Hostels, Main_Gate).
    3. Real-time supply gaps are routed to active, resilient vendors (high KS score) in the same geographic cluster, matching their current inventory profiles.

🗄️ Database & Storage Layer

Postgres Firestore facade

Although the backend utilizes a robust PostgreSQL database, it accesses data using a Firebase Firestore-compatible wrapper (PostgresDB) in app/services/postgres.py.

This facade translates NoSQL collection streams, documents, and filter queries (where(...), limit(...)) into optimized SQL statements:

  • Standard collections map directly to dedicated, indexed relational tables (e.g. vendors, ambient_chat_logs, mpesa_transactions).
  • Dynamic, schema-free fields are stored inside PostgreSQL JSONB columns, leveraging GIN indexes for fast nested key lookups.

🚧 System Refactor Plan

To transition DIRA from research validation to production scale, the following refactoring steps are proposed:

Phase 1: Database Migration Cleanups

  • Goal: Formally deprecate direct SQL table creations from init_db() and completely migrate schema initialization to Alembic revision scripts.
  • Impact: Provides proper database version control and clean rollbacks in production environments.

Phase 2: Decouple Firestore Facade to Native SQLAlchemy ORM

  • Goal: Rewrite app/services/postgres.py to use a native SQLAlchemy session model.
  • Reasoning: While the Firestore facade was useful for rapid prototyping, native ORM mapping provides better type checking, transaction safety, and performance optimizations.

Phase 3: Real-time Pub/Sub WebSockets for Dashboard Updates

  • Goal: Introduce a FastAPI WebSocket router to stream incoming WhatsApp messages and reconciled payments immediately to the React frontend.
  • Current State: React relies on 10-second polling to fetch dashboard updates.