Tau.Acuvim/docs/acuvim-spec-08.md

Phase 8: Console Application Backend

Objective

Build the server-side backend for the console application that manages the ESP32 device fleet. The backend handles device registration, configuration management, telemetry ingestion, OTA firmware distribution, health monitoring, and exposes a REST API for the frontend (Phase 9).

Prerequisites

• Phase 7 complete (device firmware feature-complete)
• Server/VPS for hosting (Linux recommended)
• Domain name (optional, can use IP for development)
• PostgreSQL database
• Mosquitto MQTT broker

Deliverables

1. .NET 10 Web API application
2. PostgreSQL database schema and migrations
3. MQTT client bridge (subscribe to device topics, publish commands)
4. REST API for frontend and external integrations
5. Firmware binary hosting and version management
6. Authentication and authorization
7. Deployment configuration (Docker)

---

8.1 Technology Stack

Component	Technology	Justification
Runtime	.NET 10	Strong typing, high performance, built-in DI
Web Framework	ASP.NET Core Minimal API	Lightweight, fast, less boilerplate
Database	PostgreSQL 16	Robust, JSON support, time-series extensions
ORM	Entity Framework Core 8	Migrations, LINQ queries, strong typing
MQTT Client	MQTTnet	.NET-native, high-performance MQTT v5 client
Auth	ASP.NET Identity + JWT	Built-in user management, token-based API auth
File Storage	Local filesystem + optional S3	Firmware binaries
Caching	In-memory + optional Redis	Device status, frequent lookups
Logging	Serilog	Structured logging
API Docs	Swagger / OpenAPI	Auto-generated API documentation
Containerization	Docker + Docker Compose	Reproducible deployment

8.2 Project Structure

console/
├── src/
│   ├── Tau.Acuvim.Console/              # Main Web API project
│   │   ├── Program.cs
│   │   ├── appsettings.json
│   │   ├── Controllers/                  # API endpoints (if using controllers)
│   │   ├── Endpoints/                    # Minimal API endpoint groups
│   │   │   ├── DeviceEndpoints.cs
│   │   │   ├── TelemetryEndpoints.cs
│   │   │   ├── FirmwareEndpoints.cs
│   │   │   ├── ConfigEndpoints.cs
│   │   │   ├── AlertEndpoints.cs
│   │   │   └── AuthEndpoints.cs
│   │   ├── Services/
│   │   │   ├── MqttBridgeService.cs      # MQTT client (background service)
│   │   │   ├── DeviceService.cs
│   │   │   ├── TelemetryService.cs
│   │   │   ├── FirmwareService.cs
│   │   │   ├── CommandService.cs
│   │   │   ├── HealthMonitorService.cs
│   │   │   └── AlertService.cs
│   │   ├── Models/
│   │   │   ├── Device.cs
│   │   │   ├── TelemetryRecord.cs
│   │   │   ├── FirmwareVersion.cs
│   │   │   ├── DeviceConfig.cs
│   │   │   ├── Alert.cs
│   │   │   ├── Command.cs
│   │   │   └── User.cs
│   │   ├── Data/
│   │   │   ├── AppDbContext.cs
│   │   │   └── Migrations/
│   │   ├── DTOs/                          # Request/response models
│   │   ├── Middleware/
│   │   │   └── ApiKeyMiddleware.cs
│   │   └── Hubs/
│   │       └── DeviceHub.cs              # SignalR for real-time UI updates
│   │
│   └── Tau.Acuvim.Console.Tests/         # Unit and integration tests
│
├── docker-compose.yml
├── Dockerfile
└── README.md

8.3 Database Schema

Entity Relationship

Devices ──< TelemetryRecords
Devices ──< Alerts
Devices ──< Commands
Devices ──< DeviceConfigs (1:1 current, history)
FirmwareVersions (standalone)
Users (standalone)

Tables

devices

CREATE TABLE devices (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    device_id       VARCHAR(32) UNIQUE NOT NULL,    -- e.g., "ACV-AABBCCDDEEFF"
    name            VARCHAR(128),                    -- Friendly name
    description     TEXT,
    mac_address     VARCHAR(17),
    imei            VARCHAR(15),
    iccid           VARCHAR(20),
    hardware        VARCHAR(64),                     -- "TTGO T-Call v1.4"
    firmware_version VARCHAR(16),
    capabilities    JSONB,                           -- ["wifi","gsm","sd","modbus"]
    status          VARCHAR(16) DEFAULT 'offline',   -- online, offline, degraded
    last_heartbeat  TIMESTAMPTZ,
    last_telemetry  TIMESTAMPTZ,
    connection_type VARCHAR(8),                      -- wifi, gsm
    signal_strength INTEGER,
    ip_address      VARCHAR(45),
    boot_count      INTEGER DEFAULT 0,
    group_id        UUID REFERENCES device_groups(id),
    tags            JSONB DEFAULT '[]',
    created_at      TIMESTAMPTZ DEFAULT NOW(),
    updated_at      TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_devices_device_id ON devices(device_id);
CREATE INDEX idx_devices_status ON devices(status);

telemetry_records

CREATE TABLE telemetry_records (
    id          BIGSERIAL PRIMARY KEY,
    device_id   VARCHAR(32) NOT NULL REFERENCES devices(device_id),
    timestamp   TIMESTAMPTZ NOT NULL,
    data        JSONB NOT NULL,                      -- Full telemetry payload
    source      VARCHAR(8) DEFAULT 'live',           -- live, sd (buffered)
    connection  VARCHAR(8),                          -- wifi, gsm
    received_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_telemetry_device_ts ON telemetry_records(device_id, timestamp DESC);
CREATE INDEX idx_telemetry_timestamp ON telemetry_records(timestamp DESC);

-- Partitioning by month (for large-scale deployments)
-- Consider TimescaleDB extension for automatic time-series management

alerts

CREATE TABLE alerts (
    id          BIGSERIAL PRIMARY KEY,
    device_id   VARCHAR(32) NOT NULL REFERENCES devices(device_id),
    alert_type  VARCHAR(32) NOT NULL,                -- overvoltage, modbus_loss, etc.
    severity    VARCHAR(16) NOT NULL,                -- info, warning, critical
    message     TEXT NOT NULL,
    value       DOUBLE PRECISION,
    threshold   DOUBLE PRECISION,
    metadata    JSONB,                               -- Additional context
    acknowledged BOOLEAN DEFAULT FALSE,
    acknowledged_by VARCHAR(128),
    acknowledged_at TIMESTAMPTZ,
    created_at  TIMESTAMPTZ DEFAULT NOW(),
    resolved_at TIMESTAMPTZ
);

CREATE INDEX idx_alerts_device ON alerts(device_id, created_at DESC);
CREATE INDEX idx_alerts_unresolved ON alerts(device_id) WHERE resolved_at IS NULL;

commands

CREATE TABLE commands (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    device_id   VARCHAR(32) NOT NULL REFERENCES devices(device_id),
    request_id  VARCHAR(64) UNIQUE NOT NULL,
    command     VARCHAR(32) NOT NULL,
    params      JSONB,
    status      VARCHAR(16) DEFAULT 'pending',       -- pending, sent, ack, success, error, timeout
    response    JSONB,
    created_by  VARCHAR(128),
    created_at  TIMESTAMPTZ DEFAULT NOW(),
    sent_at     TIMESTAMPTZ,
    completed_at TIMESTAMPTZ
);

CREATE INDEX idx_commands_device ON commands(device_id, created_at DESC);
CREATE INDEX idx_commands_pending ON commands(status) WHERE status IN ('pending', 'sent');

firmware_versions

CREATE TABLE firmware_versions (
    id           UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    version      VARCHAR(16) UNIQUE NOT NULL,         -- "1.2.0"
    filename     VARCHAR(128) NOT NULL,
    file_path    VARCHAR(256) NOT NULL,
    file_size    INTEGER NOT NULL,
    checksum     VARCHAR(128) NOT NULL,               -- SHA256
    release_notes TEXT,
    hardware     VARCHAR(64),                          -- Target hardware (null = all)
    mandatory    BOOLEAN DEFAULT FALSE,
    active       BOOLEAN DEFAULT TRUE,                 -- Available for deployment
    uploaded_by  VARCHAR(128),
    created_at   TIMESTAMPTZ DEFAULT NOW()
);

device_groups

CREATE TABLE device_groups (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name        VARCHAR(64) UNIQUE NOT NULL,
    description TEXT,
    created_at  TIMESTAMPTZ DEFAULT NOW()
);

8.4 MQTT Bridge Service

MqttBridgeService.cs

A background service that maintains an MQTT connection to the broker, subscribes to all device topics, processes incoming messages, and publishes commands.

public class MqttBridgeService : BackgroundService
{
    // Subscribe to:
    //   acuvim/+/telemetry     -> TelemetryService.Ingest()
    //   acuvim/+/heartbeat     -> HealthMonitorService.ProcessHeartbeat()
    //   acuvim/+/resp          -> CommandService.ProcessResponse()
    //   acuvim/+/status        -> DeviceService.UpdateStatus()
    //   acuvim/+/alerts        -> AlertService.ProcessAlert()
    //   devices/register       -> DeviceService.Register()

    // Publish to:
    //   acuvim/{device_id}/cmd -> CommandService.Send()
}

Topic Routing

private async Task HandleMessage(MqttApplicationMessage msg)
{
    var topic = msg.Topic;
    var payload = Encoding.UTF8.GetString(msg.PayloadSegment);

    // Parse topic: acuvim/{device_id}/{type}
    var parts = topic.Split('/');

    if (topic == "devices/register")
    {
        await _deviceService.Register(payload);
        return;
    }

    if (parts.Length >= 3)
    {
        var deviceId = parts[1];
        var type = parts[2];

        switch (type)
        {
            case "telemetry":
                await _telemetryService.Ingest(deviceId, payload);
                break;
            case "heartbeat":
                await _healthMonitor.ProcessHeartbeat(deviceId, payload);
                break;
            case "resp":
                await _commandService.ProcessResponse(deviceId, payload);
                break;
            case "status":
                await _deviceService.UpdateConnectionStatus(deviceId, payload);
                break;
            case "alerts":
                await _alertService.ProcessAlert(deviceId, payload);
                break;
        }
    }
}

8.5 REST API Endpoints

Devices

GET    /api/devices                    List all devices (with filtering, pagination)
GET    /api/devices/{deviceId}         Get device details
PUT    /api/devices/{deviceId}         Update device (name, description, group, tags)
DELETE /api/devices/{deviceId}         Remove device from console

GET    /api/devices/{deviceId}/status  Get current device status
POST   /api/devices/{deviceId}/command Send command to device
GET    /api/devices/{deviceId}/commands Command history

Telemetry

GET    /api/devices/{deviceId}/telemetry        Latest telemetry
GET    /api/devices/{deviceId}/telemetry/history Historical data (with time range, aggregation)
GET    /api/devices/{deviceId}/telemetry/export  Export as CSV

Firmware

GET    /api/firmware                   List all firmware versions
POST   /api/firmware                   Upload new firmware version
GET    /api/firmware/{version}         Get firmware details
DELETE /api/firmware/{version}         Delete firmware version
GET    /api/firmware/download/{version} Download firmware binary (used by devices)
GET    /api/firmware/check             Check for update (used by devices)
POST   /api/firmware/deploy            Deploy firmware to device(s)

Alerts

GET    /api/alerts                     List alerts (with filtering)
GET    /api/devices/{deviceId}/alerts  Device-specific alerts
PUT    /api/alerts/{id}/acknowledge    Acknowledge alert

Configuration

GET    /api/devices/{deviceId}/config           Get device configuration
POST   /api/devices/{deviceId}/config           Push configuration to device
GET    /api/devices/{deviceId}/config/wifi/scan  Trigger WiFi scan on device

Groups

GET    /api/groups                     List device groups
POST   /api/groups                     Create group
PUT    /api/groups/{id}                Update group
DELETE /api/groups/{id}                Delete group
POST   /api/groups/{id}/command        Send command to all devices in group
POST   /api/groups/{id}/deploy         Deploy firmware to group

Auth

POST   /api/auth/login                 Login, get JWT token
POST   /api/auth/register              Register new user (admin only)
POST   /api/auth/refresh               Refresh JWT token
GET    /api/auth/me                    Get current user info

Dashboard

GET    /api/dashboard/summary          Fleet summary (total, online, offline, alerts)
GET    /api/dashboard/stats            Aggregate statistics

8.6 Command Service Flow

Sending a Command

1. Frontend calls POST /api/devices/{deviceId}/command
   Body: { "command": "wifi_scan" }
2. CommandService generates request_id, saves to DB (status: pending)
3. CommandService publishes to MQTT: acuvim/{deviceId}/cmd
4. Device receives command, processes, publishes to acuvim/{deviceId}/resp
5. MqttBridge receives response
6. CommandService matches request_id, updates DB (status: success, response: {...})
7. SignalR notifies frontend of command completion
8. If no response in 60 seconds: mark as timeout

Command Timeout

public class CommandTimeoutService : BackgroundService
{
    // Every 30 seconds, check for commands where:
    //   status == 'sent' AND sent_at < (now - 60 seconds)
    // Mark as 'timeout'
}

8.7 Health Monitor Service

HealthMonitorService.cs

Background service that monitors device health based on heartbeats:

public class HealthMonitorService : BackgroundService
{
    // Every 30 seconds:
    // - Check all devices where last_heartbeat < (now - 3 * heartbeat_interval)
    //   -> Mark as "degraded"
    // - Check all devices where last_heartbeat < (now - 5 * heartbeat_interval)
    //   -> Mark as "offline"
    // - Optionally send notification (email, webhook) for offline devices

    public async Task ProcessHeartbeat(string deviceId, string payload)
    {
        // Parse heartbeat JSON
        // Update device: status = "online", last_heartbeat, firmware_version,
        //                connection_type, signal_strength, ip_address, boot_count
        // Update health metrics
        // Check for anomalies (high error rate, low memory, brownout resets)
        // Notify frontend via SignalR
    }
}

8.8 Firmware Service

Upload Flow

public async Task<FirmwareVersion> Upload(IFormFile file, string version,
    string releaseNotes, bool mandatory)
{
    // 1. Validate file is a valid ESP32 binary (check magic bytes)
    // 2. Calculate SHA256 checksum
    // 3. Save to /firmware/{version}/firmware.bin
    // 4. Create DB record
    // 5. Return firmware version info
}

Deployment Flow

public async Task Deploy(string version, List<string> deviceIds)
{
    var firmware = await GetVersion(version);

    foreach (var deviceId in deviceIds)
    {
        await _commandService.Send(deviceId, "ota_update", new {
            url = $"{_baseUrl}/api/firmware/download/{version}",
            version = firmware.Version,
            checksum = firmware.Checksum
        });
    }
}

Device Check Endpoint

// GET /api/firmware/check?device_id=X&current_version=Y&hardware=Z
public async Task<FirmwareCheckResponse> Check(string deviceId,
    string currentVersion, string hardware)
{
    var latest = await _db.FirmwareVersions
        .Where(f => f.Active)
        .Where(f => f.Hardware == null || f.Hardware == hardware)
        .OrderByDescending(f => f.CreatedAt)
        .FirstOrDefaultAsync();

    if (latest != null && IsNewer(latest.Version, currentVersion))
    {
        return new FirmwareCheckResponse
        {
            UpdateAvailable = true,
            Version = latest.Version,
            Url = $"{_baseUrl}/api/firmware/download/{latest.Version}",
            Size = latest.FileSize,
            Checksum = latest.Checksum,
            ReleaseNotes = latest.ReleaseNotes,
            Mandatory = latest.Mandatory
        };
    }

    return new FirmwareCheckResponse { UpdateAvailable = false };
}

8.9 Real-Time Updates (SignalR)

DeviceHub.cs

SignalR hub for pushing real-time updates to the frontend:

public class DeviceHub : Hub
{
    // Methods the server calls on clients:
    // - DeviceStatusChanged(deviceId, status)
    // - TelemetryReceived(deviceId, data)
    // - HeartbeatReceived(deviceId, data)
    // - AlertCreated(deviceId, alert)
    // - CommandCompleted(deviceId, requestId, response)
    // - DeviceRegistered(device)
}

The frontend subscribes to these events for live dashboard updates without polling.

8.10 Authentication

JWT-Based Auth

• Users register/login to get a JWT token
• Token included in Authorization: Bearer <token> header
• Token expiry: 24 hours
• Refresh token: 7 days
• Roles: Admin, Operator, Viewer

Device API Key

Devices authenticate firmware downloads using their device_id as an API key or via MQTT credentials. The firmware check/download endpoints require either a valid JWT or a valid device_id.

8.11 Docker Deployment

docker-compose.yml

version: '3.8'

services:
  console:
    build: .
    ports:
      - "5000:5000"
    environment:
      - ConnectionStrings__DefaultConnection=Host=db;Database=acuvim;Username=acuvim;Password=secret
      - Mqtt__Broker=mqtt
      - Mqtt__Port=1883
    depends_on:
      - db
      - mqtt
    volumes:
      - firmware-data:/app/firmware

  db:
    image: postgres:16
    environment:
      - POSTGRES_DB=acuvim
      - POSTGRES_USER=acuvim
      - POSTGRES_PASSWORD=secret
    volumes:
      - postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  mqtt:
    image: eclipse-mosquitto:2
    ports:
      - "1883:1883"
      - "9001:9001"
    volumes:
      - mosquitto-data:/mosquitto/data
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf

volumes:
  postgres-data:
  mosquitto-data:
  firmware-data:

8.12 Configuration

appsettings.json

{
  "ConnectionStrings": {
    "DefaultConnection": "Host=localhost;Database=acuvim;Username=acuvim;Password=secret"
  },
  "Mqtt": {
    "Broker": "localhost",
    "Port": 1883,
    "Username": "console",
    "Password": "",
    "TopicPrefix": "acuvim",
    "ClientId": "acuvim-console"
  },
  "Firmware": {
    "StoragePath": "./firmware",
    "BaseUrl": "https://console.example.com"
  },
  "Jwt": {
    "Secret": "your-secret-key-min-32-chars-long-here",
    "Issuer": "Tau.Acuvim.Console",
    "ExpiryHours": 24
  },
  "HealthMonitor": {
    "CheckIntervalSeconds": 30,
    "DegradedThresholdMultiplier": 3,
    "OfflineThresholdMultiplier": 5
  },
  "Telemetry": {
    "RetentionDays": 365,
    "CleanupIntervalHours": 24
  }
}

8.13 Testing & Validation

Test	Method	Pass Criteria
API starts	dotnet run	Swagger UI at /swagger
DB migration	dotnet ef database update	Tables created
MQTT connection	Start broker, run console	MQTT connected, subscribed
Device registration	Publish registration via MQTT	Device appears in DB
Telemetry ingest	Publish telemetry via MQTT	Record saved, queryable via API
Heartbeat processing	Publish heartbeat	Device status updated
Health monitor	Stop device heartbeats	Device marked degraded then offline
Send command	POST /api/devices/{id}/command	Command published to MQTT, response received
Firmware upload	POST /api/firmware with binary	File saved, DB record created
Firmware check	GET /api/firmware/check	Correct update response
Firmware download	GET /api/firmware/download/{v}	Binary downloaded
Deploy firmware	POST /api/firmware/deploy	OTA command sent to devices
Auth login	POST /api/auth/login	JWT token returned
Auth protected	GET /api/devices without token	401 Unauthorized
SignalR	Connect from browser	Real-time events received
Docker	docker-compose up	All services start, communicate
Alert processing	Publish alert via MQTT	Alert saved, queryable

8.14 Phase 8 Completion Criteria

• .NET 10 Web API project builds and runs
• PostgreSQL schema migrated, all tables created
• MQTT bridge subscribes to all device topics
• Device registration works (MQTT -> DB)
• Telemetry ingestion works (MQTT -> DB -> API)
• Heartbeat processing updates device status
• Health monitor marks devices degraded/offline
• Command send/receive flow complete
• Firmware upload, check, and download working
• Firmware deployment (push OTA via MQTT)
• Alert processing and querying
• JWT authentication on all endpoints
• SignalR real-time updates
• Docker Compose deployment working
• API documented via Swagger/OpenAPI

---

Previous Phase: Phase 7 — Heartbeat, Health & Registration Next Phase: Phase 9 — Console Application Frontend