Tau.Acuvim/docs/acuvim-spec-04.md
Renier Forster 84a0668c54 Initial commit: Tau Acuvim IoT monitoring system
Complete IoT monitoring platform for Acuvim II power meters via ESP32.

Firmware (Phases 1-7):
- ESP32-WROVER-B (TTGO T-Call v1.4) with RS485 Modbus RTU
- WiFi STA+AP concurrent mode with GSM/GPRS failover
- Transport abstraction layer with 4 priority modes
- MQTT protocol with 20 commands, LWT, QoS, exponential backoff
- SD card offline buffering with JSONL rotation and non-blocking drain
- OTA firmware updates with dual partition rollback protection
- Watchdog timer, crash loop detection, Acuvim health monitoring
- Captive portal provisioning with AP mode

Console backend (Phase 8):
- .NET 10 minimal API with PostgreSQL + EF Core
- JWT authentication, SignalR real-time updates
- MQTTnet 5.x bridge service with health monitoring
- Device, telemetry, firmware, alert, group management
- Rate limiting, security headers, Swagger/OpenAPI

Frontend (Phase 9):
- React 18 + TypeScript + Vite with Ant Design 5
- ECharts telemetry visualization, TanStack Query
- SignalR live updates, device management UI
- Dashboard, fleet management, firmware deployment

Testing & Production (Phase 10):
- 28 firmware unit tests (Modbus, JSON, config, version)
- 23 xUnit backend tests (device, telemetry, command, alert)
- Docker Compose with nginx, TLS MQTT, PostgreSQL
- Production deployment, commissioning, and troubleshooting docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-16 19:05:32 +02:00

13 KiB

Phase 4: GSM Integration & Transport Failover

Objective

Integrate the SIM800L (and later SIM7600) GSM module using TinyGSM, implement a transport abstraction layer that provides seamless failover between WiFi and GSM, and enable MQTT communication over cellular when WiFi is unavailable.

Prerequisites

  • Phase 3 complete (WiFi, MQTT, Web UI working)
  • Active SIM card with data plan inserted in the TTGO T-Call nano SIM slot
  • APN details from the mobile carrier
  • MQTT broker accessible from the public internet (for GSM connectivity)

Deliverables

  1. GSM modem driver using TinyGSM (SIM800L, forward-compatible with SIM7600)
  2. Transport abstraction layer (WiFi / GSM unified interface)
  3. Automatic failover: WiFi -> GSM with configurable priority
  4. MQTT over GSM using TinyGSM's TCP client
  5. GSM signal quality monitoring
  6. GSM configuration via web UI and MQTT commands

4.1 TinyGSM Integration

Library Addition

Add to platformio.ini:

lib_deps =
    ...existing...
    vshymanskyy/TinyGSM@^0.11.7

build_flags =
    ...existing...
    -DTINY_GSM_MODEM_SIM800        ; Change to TINY_GSM_MODEM_SIM7600 for SIM7600
    -DTINY_GSM_RX_BUFFER=1024

Modem Pin Mapping (from Phase 1)

MODEM_RST      = GPIO 5      # Reset (active LOW)
MODEM_PWRKEY   = GPIO 4      # Power key toggle
MODEM_POWER_ON = GPIO 23     # Power enable (active HIGH)
MODEM_TX       = GPIO 27     # ESP32 TX -> SIM800L RX
MODEM_RX       = GPIO 26     # ESP32 RX <- SIM800L TX

gsm_manager.h / gsm_manager.cpp

class GsmManager {
public:
    bool begin();                           // Power on modem, initialize
    bool connect(const char* apn,           // Connect to GPRS
                 const char* user = "",
                 const char* pass = "");
    void disconnect();                      // Disconnect GPRS
    bool isConnected();                     // GPRS connection status
    void powerOff();                        // Power down modem
    void restart();                         // Reset modem

    int getSignalQuality();                 // CSQ value (0-31, 99=unknown)
    int getSignalPercent();                 // 0-100% mapped from CSQ
    String getOperator();                   // Network operator name
    String getIMEI();                       // Modem IMEI
    String getICCID();                      // SIM card ICCID
    bool isSimInserted();                   // SIM detection

    TinyGsmClient& getClient();             // TCP client for MQTT

private:
    TinyGsm modem;
    TinyGsmClient gsmClient;
    bool modemReady;

    void powerOnModem();
    void powerOffModem();
    bool waitForNetwork(uint32_t timeoutMs = 60000);
};

Modem Power Sequence

The SIM800L on the TTGO T-Call v1.4 requires a specific power-on sequence:

void GsmManager::powerOnModem() {
    // Enable modem power
    pinMode(MODEM_POWER_ON, OUTPUT);
    digitalWrite(MODEM_POWER_ON, HIGH);

    // Pull reset HIGH (not resetting)
    pinMode(MODEM_RST, OUTPUT);
    digitalWrite(MODEM_RST, HIGH);

    // Toggle PWRKEY to start modem
    pinMode(MODEM_PWRKEY, OUTPUT);
    digitalWrite(MODEM_PWRKEY, HIGH);
    delay(100);
    digitalWrite(MODEM_PWRKEY, LOW);
    delay(1000);
    digitalWrite(MODEM_PWRKEY, HIGH);

    // Wait for modem to boot (~3-5 seconds)
    delay(3000);
}

SIM7600 Migration Notes

When migrating from SIM800L to SIM7600:

  1. Change build flag: -DTINY_GSM_MODEM_SIM7600
  2. Update pin mapping (SIM7600 boards may use different GPIOs)
  3. Power sequence differs for SIM7600 (check specific board schematic)
  4. TinyGSM API remains identical — no application code changes needed
  5. SIM7600 supports 4G/LTE vs SIM800L's 2G — faster data, better coverage
  6. Consider: SIM7600 draws more peak current (~2A vs ~1A) — ensure power supply

4.2 Transport Abstraction Layer

transport_manager.h / transport_manager.cpp

Provides a unified interface for network connectivity, abstracting whether WiFi or GSM is the active transport. All higher-level modules (MQTT, HTTP, OTA) use this instead of directly accessing WiFi or GSM.

enum class TransportType {
    NONE,
    WIFI,
    GSM
};

enum class TransportPriority {
    WIFI_PREFERRED,     // Try WiFi first, fall back to GSM
    GSM_PREFERRED,      // Try GSM first, fall back to WiFi
    WIFI_ONLY,          // Only use WiFi
    GSM_ONLY            // Only use GSM
};

class TransportManager {
public:
    void begin(WifiManager& wifi, GsmManager& gsm, ConfigManager& config);
    void loop();                              // Check connections, handle failover

    bool isConnected();                       // Any transport connected
    TransportType getActiveTransport();        // Which transport is in use
    Client& getClient();                      // Returns WiFiClient or TinyGsmClient

    void setPriority(TransportPriority prio);
    TransportPriority getPriority();

    int getSignalStrength();                  // RSSI (WiFi) or CSQ% (GSM)
    String getConnectionInfo();               // Human-readable status

    void onTransportChange(std::function<void(TransportType)> callback);

private:
    WifiManager* wifi;
    GsmManager* gsm;
    ConfigManager* config;
    TransportType activeTransport;
    TransportPriority priority;

    void evaluateTransport();                 // Decision logic
    void switchTransport(TransportType newType);
    unsigned long lastEvaluation;
    static const unsigned long EVAL_INTERVAL = 10000; // 10 seconds
};

Failover Logic

evaluateTransport() runs every 10 seconds:

IF priority == WIFI_ONLY:
    Use WiFi. If WiFi down, transport = NONE.

ELSE IF priority == GSM_ONLY:
    Use GSM. If GSM down, transport = NONE.

ELSE IF priority == WIFI_PREFERRED:
    IF WiFi connected:
        Use WiFi (even if GSM also connected)
    ELSE IF GSM connected:
        Use GSM
    ELSE:
        Try WiFi first (15s timeout)
        If WiFi fails, try GSM
        If both fail, transport = NONE

ELSE IF priority == GSM_PREFERRED:
    IF GSM connected:
        Use GSM
    ELSE IF WiFi connected:
        Use WiFi
    ELSE:
        Try GSM first (30s timeout)
        If GSM fails, try WiFi
        If both fail, transport = NONE

ON transport change:
    - Fire onTransportChange callback
    - MQTT client reconnects using new Client
    - Log transport switch event

Transport Switch and MQTT Reconnect

When transport changes, the MQTT client must be updated:

void TransportManager::switchTransport(TransportType newType) {
    if (newType == activeTransport) return;

    TransportType oldType = activeTransport;
    activeTransport = newType;

    // Notify listeners (MQTT client will reconnect with new Client)
    if (onChangeCallback) {
        onChangeCallback(newType);
    }

    Serial.printf("Transport: %s -> %s\n",
        transportName(oldType), transportName(newType));
}

4.3 MQTT Client Update

The MqttClient class from Phase 2 is updated to accept a Client& from the transport manager instead of a hardcoded WiFiClient:

// Phase 2 (WiFi only):
WiFiClient wifiClient;
mqtt.begin(wifiClient);

// Phase 4 (transport abstracted):
mqtt.begin(transport.getClient());

// On transport change:
transport.onTransportChange([&](TransportType type) {
    mqtt.disconnect();
    mqtt.begin(transport.getClient());
    mqtt.connect();
});

The MQTT client does not need to know whether it's using WiFi or GSM — PubSubClient works with any Client implementation.

4.4 Telemetry Update

Add transport info to telemetry payload:

{
  "ts": 1716000000,
  "dev": "ACV-AABBCCDDEEFF",
  "conn": "gsm",
  "signal": 65,
  "operator": "Vodacom",
  ...acuvim data...
}
  • conn: "wifi" or "gsm"
  • signal: WiFi RSSI (dBm, negative) or GSM signal percent (0-100)
  • operator: GSM network name (only when on GSM)

4.5 GSM Data Efficiency

GSM data costs money. Minimize data usage:

  • Compact JSON keys (already defined in Phase 2)
  • Configurable poll interval for GSM: Option to use a longer interval when on GSM (e.g., 30s vs 5s on WiFi)
  • Batch telemetry: Optionally accumulate N readings and send as array (reduces MQTT overhead)
  • MQTT QoS 0 on GSM: Reduce overhead (QoS 1 on WiFi where bandwidth is free)
  • No retain on GSM: Skip retain flag to reduce broker storage messages

Add to DeviceConfig:

uint16_t gsm_poll_interval_sec;   // Polling interval when on GSM (default: 30)
bool gsm_batch_enabled;            // Batch readings over GSM
uint8_t gsm_batch_size;            // Number of readings per batch (default: 6)

4.6 Web UI Updates

GSM Tab (added to captive portal)

┌──────────────────────────────────────┐
│ GSM / Cellular Configuration         │
│                                      │
│ [✓] GSM Enabled                      │
│                                      │
│ Status: ● Connected (Vodacom)        │
│ Signal: ████████░░ 65%               │
│ IMEI: 123456789012345                │
│ ICCID: 8927xxxxxxxxxxxxxx            │
│                                      │
│ APN:      [internet_________]        │
│ Username: [________________]         │
│ Password: [________________]         │
│                                      │
│ Transport Priority:                  │
│ (●) WiFi preferred (GSM fallback)    │
│ ( ) GSM preferred (WiFi fallback)    │
│ ( ) WiFi only                        │
│ ( ) GSM only                         │
│                                      │
│ GSM Data Saving:                     │
│ Poll interval: [30] seconds          │
│ [✓] Batch readings (6 per batch)     │
│                                      │
│           [Save]                     │
└──────────────────────────────────────┘

API Endpoints (additions)

GET  /api/gsm/status
Response:
{
  "enabled": true,
  "connected": true,
  "signal_quality": 18,
  "signal_percent": 65,
  "operator": "Vodacom",
  "imei": "123456789012345",
  "iccid": "8927xxxxxxxxxxxxxx",
  "sim_inserted": true
}

POST /api/transport/priority
Body:
{
  "priority": "wifi_preferred"
}

4.7 Devices Without GSM

Some devices will not have a GSM module. The firmware must handle this gracefully:

  • On boot, attempt to communicate with the modem via AT commands
  • If no response after 3 attempts: set gsm_available = false
  • Hide GSM-related UI elements when GSM is not available
  • Transport priority automatically becomes WIFI_ONLY
  • All GSM code paths are skipped (no error logs for expected absence)
  • This detection is automatic — no configuration needed
bool GsmManager::begin() {
    // Try AT command 3 times
    for (int i = 0; i < 3; i++) {
        if (modem.testAT(1000)) {
            modemReady = true;
            return true;
        }
    }
    modemReady = false;  // No modem present
    return false;
}

4.8 Testing & Validation

Test Method Pass Criteria
GSM connects Insert SIM, configure APN GPRS connected, has IP
MQTT over GSM Disable WiFi, monitor MQTT Telemetry received via GSM
WiFi->GSM failover Disconnect WiFi router Switches to GSM within 30s
GSM->WiFi fallback Re-enable WiFi Switches back to WiFi
No SIM card Remove SIM Graceful failure, WiFi-only
No GSM module Run on ESP32 without SIM800L Auto-detect, WiFi-only mode
Signal strength Monitor GSM tab CSQ value updates correctly
GSM data saving Monitor payload sizes Compact JSON, correct batch size
Transport priority Change via web UI Correct transport selection
MQTT reconnect on switch Force transport change MQTT reconnects on new transport
Continuous GSM Run on GSM for 24h Stable, no memory leaks
GSM power cycle Power cycle modem Recovers and reconnects

4.9 Phase 4 Completion Criteria

  • SIM800L modem initialized with correct power sequence
  • GPRS connection established with configurable APN
  • MQTT works over GSM (publish telemetry, subscribe to commands)
  • Transport failover: WiFi -> GSM automatic switch
  • Transport fallback: GSM -> WiFi when WiFi becomes available
  • Transport priority configurable (WiFi preferred, GSM preferred, etc.)
  • Signal quality reported in telemetry and web UI
  • Devices without GSM module detected and handled gracefully
  • GSM data-saving options working (longer interval, batching)
  • Web UI updated with GSM configuration tab

Previous Phase: Phase 3 — Captive Portal Web Interface Next Phase: Phase 5 — SD Card Offline Buffering