# Integration Test Scenarios

These scenarios require hardware-in-the-loop testing with physical ESP32 + Acuvim II devices.

## Pre-requisites

- 3+ ESP32 TTGO T-Call v1.4 devices with RS485 wired to Acuvim II
- Console application deployed (Docker or local)
- MQTT broker running (Mosquitto)
- At least one device with SIM card for GSM tests
- At least one device with SD card for buffering tests

---

## Scenario 1: Full Device Lifecycle

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Flash factory firmware to ESP32 | Boot complete, serial shows startup log | |
| 2 | Scan for WiFi networks on phone | AP "Acuvim-XXXXXX" visible | |
| 3 | Connect phone to AP | Captive portal opens automatically | |
| 4 | Configure WiFi + MQTT via web UI | Settings saved confirmation | |
| 5 | Wait for device to connect | Serial shows WiFi connected, MQTT connected | |
| 6 | Check console dashboard | Device appears as "online" | |
| 7 | Wait 30 seconds | Telemetry visible in console device detail | |
| 8 | Wait for heartbeat interval | Heartbeat shows in console | |
| 9 | Push config change (poll interval) | Device applies new config, response received | |
| 10 | Upload firmware to console | Firmware listed in Firmware page | |
| 11 | Deploy firmware to device | Device downloads, installs, reboots | |
| 12 | Wait for device to come back online | Console shows new firmware version | |

## Scenario 2: Transport Failover

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Verify device online via WiFi | Console shows connection type: WiFi | |
| 2 | Power off WiFi router | Device detects WiFi loss within 30s | |
| 3 | Wait for GSM failover | Serial shows GSM connecting, MQTT reconnect | |
| 4 | Check console | Heartbeat shows connection: gsm | |
| 5 | Verify telemetry continues | Telemetry records arriving via GSM | |
| 6 | Power on WiFi router | Device detects WiFi, switches back | |
| 7 | Check console | Heartbeat shows connection: wifi | |
| 8 | Query telemetry history | No gaps during transition | |

## Scenario 3: SD Card Buffering

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Verify device online | Telemetry flowing normally | |
| 2 | Disable WiFi and GSM | Power off router + remove SIM | |
| 3 | Wait 10 minutes | Serial shows buffering to SD | |
| 4 | Check SD card (optional) | JSONL files with valid records | |
| 5 | Re-enable WiFi | Power on router | |
| 6 | Wait for drain | Serial shows SD queue draining | |
| 7 | Check console telemetry | Buffered records with source "sd" | |
| 8 | Verify order | Records in chronological order | |
| 9 | Verify live + drain | Live telemetry interleaved with drain | |

## Scenario 4: Crash Recovery

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Verify device online | Normal operation | |
| 2 | Trigger crash (test build or power cycle) | Device reboots | |
| 3 | Check serial | Watchdog reset reason logged | |
| 4 | Wait for reconnect | Device re-registers with console | |
| 5 | Check console | Boot count incremented | |
| 6 | Repeat crash 5 times | Factory reset triggered after 5th crash | |
| 7 | Check device | AP mode activated for reconfiguration | |

## Scenario 5: OTA Rollback

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Upload broken firmware to console | Firmware listed | |
| 2 | Deploy to test device | Device downloads and installs | |
| 3 | Wait for reboot | Device boots into broken firmware | |
| 4 | Wait for self-test | Self-test fails (MQTT can't connect) | |
| 5 | Automatic rollback | Device rolls back to previous firmware | |
| 6 | Wait for reconnect | Device online with old version | |
| 7 | Check console | Old firmware version shown | |

## Scenario 6: WiFi-Only Device

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Boot device without SIM module | GSM detected as unavailable | |
| 2 | Check serial | Transport priority forced to WiFi-only | |
| 3 | Configure via AP | GSM config section hidden/disabled | |
| 4 | Verify normal operation | All features work via WiFi | |
| 5 | Trigger OTA via WiFi | OTA completes successfully | |
| 6 | Disconnect WiFi | Data buffers to SD card | |

## Scenario 7: Multi-Device Fleet

| Step | Action | Expected Result | Pass |
|------|--------|-----------------|------|
| 1 | Register 5+ devices | All appear in console dashboard | |
| 2 | Create groups in console | Groups "Building A", "Building B" | |
| 3 | Assign devices to groups | Devices show group membership | |
| 4 | Deploy firmware to group | All devices in group update | |
| 5 | Send config to group | All devices apply config | |
| 6 | Power off one device | Console detects offline within 2.5 min | |
| 7 | Check alerts | Alert generated for offline device | |

---

## 72-Hour Stability Test

Deploy firmware to 3+ devices and monitor for 72 hours.

### Monitoring checklist (check every 12 hours):

| Metric | Hour 0 | Hour 12 | Hour 24 | Hour 36 | Hour 48 | Hour 60 | Hour 72 |
|--------|--------|---------|---------|---------|---------|---------|---------|
| min_free_heap (KB) | | | | | | | |
| Boot count | | | | | | | |
| Modbus error rate (%) | | | | | | | |
| MQTT disconnects | | | | | | | |
| Telemetry gaps | | | | | | | |
| Transport switches | | | | | | | |

### Induced failure tests during 72h run:

| Test | Time | Duration | Result | Recovery |
|------|------|----------|--------|----------|
| Kill WiFi | Hour 24 | 1 hour | GSM failover + SD buffer | |
| Kill MQTT broker | Hour 36 | 30 min | SD buffer + drain | |
| Power cycle device | Hour 48 | instant | Clean recovery | |

### Pass criteria:

- No memory leak: min_free_heap stable within 5%
- No unplanned reboots (boot count = 1)
- Zero data loss: all telemetry records accounted for
- All induced failures recovered automatically
- Console shows accurate device status throughout