Tau.Acuvim/docs/acuvim-spec-06.md

# Phase 6: OTA Firmware Updates

## Objective

Implement over-the-air firmware update capability via both WiFi and GSM. The device checks for updates from the console application's firmware server, downloads and validates the binary, and performs a safe update with rollback protection. Supports both push (console-initiated) and pull (device-initiated) update models.

## Prerequisites

- Phase 5 complete (all transport and storage working)
- Console application firmware hosting endpoint (Phase 8, or temporary HTTP server for testing)
- Dual OTA partition table configured

## Deliverables

1. OTA partition table (two app partitions for safe rollback)
2. HTTP-based firmware download over WiFi and GSM
3. Push OTA via MQTT command from console
4. Pull OTA via periodic version check
5. Update validation and automatic rollback on boot failure
6. Firmware version reporting

---

## 6.1 Partition Table

### Custom Partition Table for OTA

Replace the default partition table with an OTA-capable layout. Create `firmware/partitions_ota.csv`:

```csv
# Name,    Type,  SubType,  Offset,    Size,     Flags
nvs,       data,  nvs,      0x9000,    0x5000,
otadata,   data,  ota,      0xe000,    0x2000,
app0,      app,   ota_0,    0x10000,   0x1E0000,
app1,      app,   ota_1,    0x1F0000,  0x1E0000,
littlefs,  data,  spiffs,   0x3D0000,  0x20000,
nvs_keys,  data,  nvs_keys, 0x3F0000,  0x1000,
```

**Layout summary (8MB flash):**

| Partition | Size | Purpose |
|-----------|------|---------|
| nvs | 20KB | Non-volatile storage (config) |
| otadata | 8KB | OTA state tracking |
| app0 | 1,920KB | Application slot 0 |
| app1 | 1,920KB | Application slot 1 |
| littlefs | 128KB | Web UI files |
| nvs_keys | 4KB | NVS encryption keys (optional) |

**Update `platformio.ini`:**

```ini
board_build.partitions = partitions_ota.csv
```

**Note:** Each app partition is ~1.9MB. Monitor firmware binary size — if it approaches this limit, consider optimizing or increasing partition size by reducing LittleFS.

## 6.2 OTA Manager

### `ota_manager.h` / `ota_manager.cpp`

```cpp
enum class OtaStatus {
    IDLE,
    CHECKING,
    DOWNLOADING,
    INSTALLING,
    SUCCESS,
    FAILED,
    ROLLING_BACK
};

struct FirmwareInfo {
    String version;           // Semantic version (e.g., "1.2.0")
    String url;               // Download URL
    uint32_t size;             // Binary size in bytes
    String checksum;           // MD5 or SHA256 hash
    String releaseNotes;       // Optional description
    bool mandatory;            // Force update regardless of version
};

class OtaManager {
public:
    void begin(ConfigManager& config, TransportManager& transport);
    void loop();                                // Periodic version check

    // Pull model (device checks for updates)
    bool checkForUpdate(FirmwareInfo& info);     // Query server for latest version
    bool isUpdateAvailable();

    // Push model (console sends update command)
    bool startUpdate(const String& url, const String& expectedChecksum = "");

    // Status
    OtaStatus getStatus();
    uint8_t getProgress();                       // 0-100%
    String getStatusMessage();
    String getCurrentVersion();

    // Rollback
    bool markValid();                            // Mark current firmware as good
    bool rollback();                             // Revert to previous firmware

    void onProgress(std::function<void(uint8_t percent)> callback);
    void onComplete(std::function<void(bool success, const String& message)> callback);

private:
    bool performUpdate(const String& url);
    bool validateChecksum(const String& expected);
    bool compareVersions(const String& current, const String& available);

    ConfigManager* config;
    TransportManager* transport;
    OtaStatus status;
    uint8_t progress;
    unsigned long lastCheck;
};
```

## 6.3 Firmware Version Scheme

### Semantic Versioning

Format: `MAJOR.MINOR.PATCH` (e.g., `1.2.3`)

- **MAJOR:** Breaking changes (protocol changes, incompatible config format)
- **MINOR:** New features (new register groups, new settings)
- **PATCH:** Bug fixes, optimizations

### Version in Firmware

Define in `version.h`:

```cpp
#define FW_VERSION_MAJOR 1
#define FW_VERSION_MINOR 0
#define FW_VERSION_PATCH 0
#define FW_VERSION "1.0.0"
#define FW_BUILD_DATE __DATE__
#define FW_BUILD_TIME __TIME__
```

### Version Comparison

```cpp
// Returns true if available > current
bool OtaManager::compareVersions(const String& current, const String& available) {
    int curMaj, curMin, curPat;
    int avlMaj, avlMin, avlPat;
    sscanf(current.c_str(), "%d.%d.%d", &curMaj, &curMin, &curPat);
    sscanf(available.c_str(), "%d.%d.%d", &avlMaj, &avlMin, &avlPat);

    if (avlMaj != curMaj) return avlMaj > curMaj;
    if (avlMin != curMin) return avlMin > curMin;
    return avlPat > curPat;
}
```

## 6.4 Pull Model (Device-Initiated Check)

### Version Check Endpoint

The device periodically queries the console application:

```
GET {console_url}/api/firmware/check?device_id={id}&current_version={ver}&hardware={hw}

Response (update available):
{
  "update_available": true,
  "version": "1.2.0",
  "url": "https://console.example.com/api/firmware/download/1.2.0",
  "size": 1234567,
  "checksum": "sha256:abcdef1234567890...",
  "release_notes": "Added THD monitoring improvements",
  "mandatory": false
}

Response (no update):
{
  "update_available": false,
  "current_version": "1.0.0"
}
```

### Check Interval

- Default: every 6 hours
- Configurable via NVS (`ota_check_interval_hours`)
- Also checks immediately on boot (after 60 second delay to allow connections to stabilize)
- Can be triggered manually via MQTT command or web UI

### Auto-Update Policy

Configurable behavior when an update is found:

- **Auto-update:** Download and install immediately (default for patch versions)
- **Notify only:** Report available update to console, wait for push command
- **Manual only:** Never auto-update, require explicit trigger

Add to `DeviceConfig`:

```cpp
uint8_t ota_check_interval_hours;  // default: 6
uint8_t ota_auto_update;           // 0=manual, 1=notify, 2=auto
```

## 6.5 Push Model (Console-Initiated)

### MQTT OTA Command

The console publishes to `{prefix}/{device_id}/cmd`:

```json
{
  "cmd": "ota_update",
  "request_id": "req-12345",
  "url": "https://console.example.com/api/firmware/download/1.2.0",
  "version": "1.2.0",
  "checksum": "sha256:abcdef1234567890...",
  "mandatory": false
}
```

### Device Response Flow

```
1. Receive OTA command
2. Publish ACK: {"request_id":"req-12345","status":"accepted","message":"Starting update..."}
3. Publish progress: {"request_id":"req-12345","status":"downloading","progress":45}
4. Publish progress: {"request_id":"req-12345","status":"installing","progress":100}
5. Reboot
6. After reboot, publish: {"status":"updated","version":"1.2.0"}
   OR if rollback: {"status":"rollback","version":"1.0.0","reason":"Boot validation failed"}
```

## 6.6 Update Process

### Download and Install

```cpp
bool OtaManager::performUpdate(const String& url) {
    status = OtaStatus::DOWNLOADING;

    // Use HTTPClient with either WiFiClient or TinyGSMClient
    HTTPClient http;
    Client& client = transport->getClient();
    http.begin(client, url);

    int httpCode = http.GET();
    if (httpCode != HTTP_CODE_OK) {
        status = OtaStatus::FAILED;
        return false;
    }

    int contentLength = http.getSize();
    if (contentLength <= 0) {
        status = OtaStatus::FAILED;
        return false;
    }

    // Start OTA update
    if (!Update.begin(contentLength)) {
        status = OtaStatus::FAILED;
        return false;
    }

    status = OtaStatus::INSTALLING;

    // Stream firmware to flash
    WiFiClient* stream = http.getStreamPtr();
    uint8_t buf[1024];
    int totalRead = 0;

    while (http.connected() && totalRead < contentLength) {
        int available = stream->available();
        if (available > 0) {
            int read = stream->readBytes(buf, min(available, (int)sizeof(buf)));
            Update.write(buf, read);
            totalRead += read;
            progress = (totalRead * 100) / contentLength;
            if (progressCallback) progressCallback(progress);
        }
        yield();
    }

    if (Update.end(true)) {
        status = OtaStatus::SUCCESS;
        return true;
    } else {
        status = OtaStatus::FAILED;
        return false;
    }
}
```

### GSM Considerations for OTA

- SIM800L is 2G only — download speeds ~10-20 KB/s
- A 1MB firmware takes ~50-100 seconds over GPRS
- SIM7600 (4G) will be much faster (~100+ KB/s)
- Use chunked download with progress reporting
- Implement download resume if connection drops (Range header)
- Timeout: 10 minutes for GSM download, 2 minutes for WiFi

## 6.7 Rollback Protection

### Boot Validation

ESP32's OTA library tracks which partition is "pending verification." On first boot after update:

```cpp
void setup() {
    // Check if this is first boot after OTA
    const esp_partition_t* running = esp_ota_get_running_partition();

    if (esp_ota_check_rollback_is_possible()) {
        // Run self-test
        bool healthy = selfTest();

        if (healthy) {
            esp_ota_mark_app_valid_cancel_rollback();
            Serial.println("OTA update validated");
        } else {
            Serial.println("OTA self-test failed, rolling back");
            esp_ota_mark_app_invalid_rollback_and_reboot();
        }
    }
}
```

### Self-Test Criteria

Before marking the new firmware as valid:

1. Serial port initialized (basic hardware check)
2. NVS configuration loads successfully
3. WiFi or GSM connects within 60 seconds
4. Modbus communication with Acuvim II succeeds (at least one read)
5. MQTT connection succeeds
6. No crash within first 30 seconds

If any critical test fails: rollback to previous firmware automatically.

### Rollback Flow

```
1. New firmware boots
2. Self-test fails (e.g., MQTT won't connect)
3. esp_ota_mark_app_invalid_rollback_and_reboot() called
4. ESP32 reboots into previous firmware
5. Previous firmware publishes rollback notification to MQTT
6. Console marks update as failed for this device
```

## 6.8 Web UI Updates

### Device Tab Addition

```
┌──────────────────────────────────────┐
│ Firmware                             │
│                                      │
│ Current Version: v1.0.0              │
│ Build Date: May 16 2026              │
│ Partition: app0                      │
│                                      │
│ Auto-Update: [Notify Only ▼]         │
│ Check Interval: [6] hours            │
│                                      │
│ [Check for Update]                   │
│                                      │
│ ┌──────────────────────────────────┐ │
│ │ Update Available: v1.2.0        │ │
│ │ Size: 1.2 MB                    │ │
│ │ Notes: Added THD improvements   │ │
│ │                                 │ │
│ │ [████████░░░░░░░░] 45%         │ │
│ │        [Install Update]         │ │
│ └──────────────────────────────────┘ │
│                                      │
│ Manual Upload:                       │
│ [Choose File] firmware.bin           │
│        [Upload & Install]            │
└──────────────────────────────────────┘
```

### Manual Upload Endpoint

For cases where the device has no internet access (local AP mode):

```
POST /api/ota/upload
Content-Type: multipart/form-data
Body: firmware binary file

Response:
{
  "success": true,
  "message": "Firmware uploaded. Installing and rebooting..."
}
```

This allows uploading a `.bin` file directly through the web browser.

### API Endpoints

```
GET /api/ota/status
Response:
{
  "current_version": "1.0.0",
  "build_date": "May 16 2026",
  "partition": "app0",
  "update_available": true,
  "available_version": "1.2.0",
  "auto_update": "notify",
  "check_interval_hours": 6,
  "last_check": 1716000000,
  "ota_status": "idle",
  "ota_progress": 0
}

POST /api/ota/check
Response:
{
  "update_available": true,
  "version": "1.2.0",
  "size": 1234567,
  "release_notes": "Added THD improvements"
}

POST /api/ota/install
Body:
{
  "url": "https://console.example.com/api/firmware/download/1.2.0"
}
Response:
{
  "success": true,
  "message": "Download started..."
}

POST /api/ota/rollback
Response:
{
  "success": true,
  "message": "Rolling back to previous firmware..."
}
```

## 6.9 Security

### Firmware Integrity

- **Checksum validation:** SHA256 hash comparison before applying update
- **Size validation:** Compare Content-Length with expected size
- **HTTPS:** Download firmware over HTTPS when possible (WiFi). GSM may use HTTP if TLS memory is too high on SIM800L.
- **Signed firmware (future):** ESP32 supports secure boot with RSA-3072 signed images. Can be enabled later for production.

### Preventing Bricked Devices

- Dual partition scheme ensures there's always a working firmware to fall back to
- Self-test on boot with automatic rollback
- Manual upload via web UI as last resort (AP mode always works)
- Factory firmware can be flashed via USB as absolute fallback

## 6.10 Testing & Validation

| Test | Method | Pass Criteria |
|------|--------|---------------|
| OTA check (pull) | Set up test HTTP server | Device detects available update |
| OTA download WiFi | Trigger update over WiFi | Firmware downloaded, installed, reboots |
| OTA download GSM | Disable WiFi, trigger OTA | Firmware downloaded over GPRS |
| Push OTA via MQTT | Publish OTA command | Device downloads and installs |
| Progress reporting | Monitor MQTT during update | Progress updates received (0-100%) |
| Checksum validation | Serve firmware with wrong checksum | Update rejected |
| Rollback | Upload firmware that fails self-test | Auto-rollback to previous version |
| Manual upload | Upload .bin via web UI | Firmware installed via browser |
| Version comparison | Check with same version | No update triggered |
| No internet | Check without connectivity | Graceful failure, no crash |
| Large firmware | Upload near-max-size binary | Succeeds within partition limit |
| Interrupted download | Kill network mid-download | Clean failure, no corruption |
| Post-update config | Update firmware | All NVS config preserved |

## 6.11 Phase 6 Completion Criteria

- [ ] Dual OTA partition table configured and working
- [ ] Pull model: device checks console for firmware updates
- [ ] Push model: MQTT command triggers firmware update
- [ ] OTA works over WiFi
- [ ] OTA works over GSM
- [ ] SHA256 checksum validation before applying update
- [ ] Progress reported via MQTT and web UI
- [ ] Self-test on first boot after update
- [ ] Automatic rollback on failed self-test
- [ ] Manual firmware upload via web UI (AP mode)
- [ ] Version reported in heartbeat and status API
- [ ] All NVS configuration preserved across updates
- [ ] OTA settings configurable (auto-update policy, check interval)

---

**Previous Phase:** [Phase 5 — SD Card Offline Buffering](acuvim-spec-05.md)
**Next Phase:** [Phase 7 — Heartbeat, Health & Device Registration](acuvim-spec-07.md)