Tau.Acuvim/docs/acuvim-spec-02.md
Renier Forster 84a0668c54 Initial commit: Tau Acuvim IoT monitoring system
Complete IoT monitoring platform for Acuvim II power meters via ESP32.

Firmware (Phases 1-7):
- ESP32-WROVER-B (TTGO T-Call v1.4) with RS485 Modbus RTU
- WiFi STA+AP concurrent mode with GSM/GPRS failover
- Transport abstraction layer with 4 priority modes
- MQTT protocol with 20 commands, LWT, QoS, exponential backoff
- SD card offline buffering with JSONL rotation and non-blocking drain
- OTA firmware updates with dual partition rollback protection
- Watchdog timer, crash loop detection, Acuvim health monitoring
- Captive portal provisioning with AP mode

Console backend (Phase 8):
- .NET 10 minimal API with PostgreSQL + EF Core
- JWT authentication, SignalR real-time updates
- MQTTnet 5.x bridge service with health monitoring
- Device, telemetry, firmware, alert, group management
- Rate limiting, security headers, Swagger/OpenAPI

Frontend (Phase 9):
- React 18 + TypeScript + Vite with Ant Design 5
- ECharts telemetry visualization, TanStack Query
- SignalR live updates, device management UI
- Dashboard, fleet management, firmware deployment

Testing & Production (Phase 10):
- 28 firmware unit tests (Modbus, JSON, config, version)
- 23 xUnit backend tests (device, telemetry, command, alert)
- Docker Compose with nginx, TLS MQTT, PostgreSQL
- Production deployment, commissioning, and troubleshooting docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-16 19:05:32 +02:00

10 KiB

Phase 2: WiFi, MQTT Client & NVS Configuration

Objective

Add WiFi station connectivity, MQTT client for telemetry streaming, and NVS-based persistent configuration storage. At the end of this phase, the ESP32 will connect to a configured WiFi network, publish Acuvim II data to an MQTT broker, and persist all settings across reboots.

Prerequisites

  • Phase 1 complete (Modbus reads working)
  • WiFi network with known SSID/password
  • MQTT broker running (Mosquitto recommended for development)
  • MQTT broker address, port, credentials

Deliverables

  1. NVS configuration manager (read/write all device settings)
  2. WiFi station connection with auto-reconnect
  3. MQTT client with QoS 1 publishing
  4. Acuvim II telemetry published as structured JSON to MQTT
  5. Connection status monitoring

2.1 NVS Configuration Manager

config_manager.h / config_manager.cpp

Manages all persistent device settings using ESP32 Non-Volatile Storage (Preferences library). This is the single source of truth for device configuration. All other modules read from this.

Stored configuration:

struct DeviceConfig {
    // Device Identity
    char device_id[32];           // Unique device ID (default: ESP32 MAC)

    // WiFi
    char wifi_ssid[64];
    char wifi_password[64];
    bool wifi_enabled;            // default: true

    // MQTT
    char mqtt_broker[128];        // hostname or IP
    uint16_t mqtt_port;           // default: 1883
    char mqtt_username[64];
    char mqtt_password[64];
    char mqtt_topic_prefix[64];   // default: "acuvim"
    bool mqtt_tls;                // default: false

    // GSM (Phase 4)
    char gsm_apn[64];
    char gsm_user[32];
    char gsm_pass[32];
    bool gsm_enabled;             // default: false

    // Console
    char console_url[128];        // Console application URL

    // Acuvim II
    uint8_t modbus_slave_addr;    // default: 1
    uint32_t modbus_baud_rate;    // default: 9600
    uint16_t poll_interval_sec;   // default: 5

    // Sleep
    uint16_t sleep_duration_min;  // deep sleep duration (0 = disabled)
    uint16_t wake_duration_sec;   // active time before sleep

    // Heartbeat
    uint16_t heartbeat_interval_sec; // default: 60

    // Firmware
    char firmware_version[16];    // current firmware version
};

Key methods:

class ConfigManager {
public:
    void begin();                          // Load config from NVS
    void save();                           // Write all config to NVS
    void reset();                          // Factory reset (clear NVS)
    DeviceConfig& get();                   // Get reference to current config
    String toJson();                       // Serialize config to JSON
    bool fromJson(const String& json);     // Deserialize and apply config
    String getDeviceId();                  // MAC-based unique ID
};

NVS namespace: "acuvim_cfg"

Design notes:

  • Config is loaded into RAM at boot and written back on changes — NVS is not read on every access
  • toJson() excludes passwords from output unless a includeSecrets flag is set
  • fromJson() performs validation before applying (e.g., port range, non-empty SSID)
  • getDeviceId() generates from WiFi MAC address if not explicitly set: ACV-AABBCCDDEEFF

2.2 WiFi Station Manager

wifi_manager.h / wifi_manager.cpp

Handles WiFi station (STA) mode connection, auto-reconnect, and event monitoring.

Key methods:

class WifiManager {
public:
    void begin(const char* ssid, const char* password);
    bool isConnected();
    void disconnect();
    int8_t getRSSI();                    // Signal strength for health reporting
    String getIPAddress();
    String getMACAddress();
    void onConnect(std::function<void()> callback);
    void onDisconnect(std::function<void()> callback);

private:
    void handleEvent(WiFiEvent_t event);
    unsigned long lastReconnectAttempt;
    static const unsigned long RECONNECT_INTERVAL = 30000; // 30 seconds
};

Behavior:

  • On boot: attempt WiFi connection if wifi_enabled is true and SSID is configured
  • Connection timeout: 15 seconds per attempt
  • Auto-reconnect: on disconnect, retry every 30 seconds
  • Events: fire callbacks for connect/disconnect so MQTT and transport layers can react
  • If no WiFi configured: skip WiFi entirely (device may be GSM-only)

WiFi events to handle:

  • WIFI_EVENT_STA_CONNECTED — log, trigger MQTT connect
  • WIFI_EVENT_STA_DISCONNECTED — log, mark transport down, start reconnect timer
  • WIFI_EVENT_STA_GOT_IP — log IP, fire onConnect callback

2.3 MQTT Client

mqtt_client.h / mqtt_client.cpp

Wraps PubSubClient for MQTT publish/subscribe with connection management.

Topic structure:

{topic_prefix}/{device_id}/telemetry      # Acuvim II data (publish)
{topic_prefix}/{device_id}/heartbeat      # Health data (publish, Phase 7)
{topic_prefix}/{device_id}/cmd            # Inbound commands (subscribe)
{topic_prefix}/{device_id}/resp           # Command responses (publish)
{topic_prefix}/{device_id}/status         # Online/offline (LWT)
devices/register                           # Device registration (publish, Phase 7)

Last Will and Testament (LWT):

Topic:   {prefix}/{device_id}/status
Payload: {"status":"offline","timestamp":<epoch>}
QoS:     1
Retain:  true

On successful connect, publish:

Topic:   {prefix}/{device_id}/status
Payload: {"status":"online","timestamp":<epoch>,"ip":"...","fw":"..."}
QoS:     1
Retain:  true

Key methods:

class MqttClient {
public:
    void begin(Client& networkClient);    // WiFiClient or TinyGSMClient
    void setConfig(const DeviceConfig& config);
    bool connect();
    void disconnect();
    bool isConnected();
    void loop();                          // Call in main loop for keepalive

    bool publishTelemetry(const AcuvimData& data);
    bool publishHeartbeat(/* Phase 7 */);
    bool publishResponse(const String& requestId, const String& payload);
    bool publish(const char* topic, const char* payload, bool retain = false);

    void onCommand(std::function<void(const String& topic, const String& payload)> callback);

private:
    PubSubClient client;
    void handleCallback(char* topic, byte* payload, unsigned int length);
    bool subscribeToCommands();
    unsigned long lastReconnectAttempt;
};

Reconnect behavior:

  • On disconnect, retry every 10 seconds
  • Use exponential backoff: 10s, 20s, 40s, 60s (max)
  • After MQTT reconnect, re-subscribe to command topic

2.4 Telemetry JSON Format

Each poll cycle publishes to {prefix}/{device_id}/telemetry:

{
  "ts": 1716000000,
  "dev": "ACV-AABBCCDDEEFF",
  "v": {
    "a": 230.1,
    "b": 231.4,
    "c": 229.8,
    "ab": 399.2,
    "bc": 400.1,
    "ca": 398.7
  },
  "i": {
    "a": 15.2,
    "b": 14.8,
    "c": 15.5
  },
  "p": {
    "total": 10.5,
    "a": 3.5,
    "b": 3.4,
    "c": 3.6,
    "reactive": 2.1,
    "apparent": 10.7,
    "pf": 0.98
  },
  "f": 50.01,
  "e": {
    "imp_act": 12345.6,
    "exp_act": 0.0,
    "imp_react": 1234.5,
    "exp_react": 0.0
  },
  "d": {
    "act": 10.2,
    "max_act": 15.0,
    "react": 2.0
  },
  "thd": {
    "va": 2.1,
    "vb": 2.3,
    "vc": 2.0,
    "ia": 5.4,
    "ib": 5.1,
    "ic": 5.6
  },
  "conn": "wifi",
  "rssi": -45
}

Design notes:

  • Short keys to minimize payload size (important for GSM data costs in Phase 4)
  • conn field indicates transport type (wifi, gsm)
  • rssi is WiFi signal or GSM CSQ depending on active transport
  • Timestamp uses UTC epoch seconds from NTP (synced on WiFi connect)

2.5 NTP Time Synchronization

Configure NTP on WiFi connect to ensure accurate timestamps:

configTime(0, 0, "pool.ntp.org", "time.nist.gov");
  • Timezone: UTC (console application handles local time display)
  • If NTP unavailable: use millis() offset from last known time
  • Store last known epoch in NVS to provide approximate time after reboot without NTP

2.6 Updated Main Application

main.cpp (Phase 2)

// Pseudocode
ConfigManager config;
WifiManager wifi;
MqttClient mqtt;
AcuvimReader acuvim;

void setup() {
    Serial.begin(115200);
    config.begin();

    acuvim.begin(config.get().modbus_slave_addr, config.get().modbus_baud_rate);

    if (config.get().wifi_enabled) {
        wifi.begin(config.get().wifi_ssid, config.get().wifi_password);
        wifi.onConnect([]() {
            mqtt.connect();
        });
    }

    WiFiClient wifiClient;
    mqtt.begin(wifiClient);
    mqtt.setConfig(config.get());

    mqtt.onCommand([](const String& topic, const String& payload) {
        handleCommand(payload);  // Phase 7 expands this
    });
}

void loop() {
    mqtt.loop();

    if (millis() - lastPoll >= config.get().poll_interval_sec * 1000) {
        AcuvimData data;
        if (acuvim.readAll(data)) {
            if (mqtt.isConnected()) {
                mqtt.publishTelemetry(data);
            }
        }
        lastPoll = millis();
    }
}

2.7 Library Dependencies Update

Add to platformio.ini:

lib_deps =
    4-20ma/ModbusMaster@^2.0.1
    bblanchon/ArduinoJson@^7.0.0
    knolleary/PubSubClient@^2.8.0

2.8 Testing & Validation

Test Method Pass Criteria
WiFi connects Configure valid SSID/password Gets IP, NTP syncs
WiFi auto-reconnect Disable/re-enable router Reconnects within 30s
WiFi disabled Set wifi_enabled = false No WiFi attempts
MQTT connects Point to Mosquitto Connected, subscribed to cmd topic
MQTT publish Monitor with mosquitto_sub Telemetry JSON received at poll interval
MQTT reconnect Restart Mosquitto Client reconnects, re-subscribes
MQTT LWT Kill ESP32 power Broker publishes offline status
NVS save/load Change config, reboot Settings persist across reboot
NVS factory reset Trigger reset All settings cleared to defaults
JSON format Validate published JSON Valid JSON, all fields present
NTP time Check timestamps Within 1 second of actual time
Memory stability Run 24 hours No heap fragmentation or OOM

2.9 Phase 2 Completion Criteria

  • ConfigManager stores and retrieves all settings from NVS
  • WiFi connects on boot with auto-reconnect
  • MQTT client connects, publishes telemetry, subscribes to commands
  • Telemetry JSON contains all Acuvim II register data
  • LWT correctly reports online/offline status
  • NTP time synchronization working
  • All settings survive reboot
  • Factory reset clears all NVS data

Previous Phase: Phase 1 — Project Setup & Modbus Next Phase: Phase 3 — Captive Portal Web Interface