a92b4277ae
Customer-stack measurements now flow to the Admin-stack central DB via
HTTPS POST, with firmware buffer-and-replay back-fills handled correctly.
Client side (push)
- monitoring.PowerMeasurements gains ReceivedAt (default NOW()) +
index. Push selects WHERE ReceivedAt > LastCursor, so back-dated
rows from offline-buffer replays are picked up automatically.
- app.FleetPushState table holds per-resource cursors + backoff state.
- FleetPushClient: HttpClient wrapper, X-Customer-Token header,
X-Batch-Type, X-Push-Cursor. 413 returns retry-after halving signal.
- FleetPushService: BackgroundService loop. Per tick: sites (full set),
devices (full set), measurements (cursor-driven up to 3 batches).
Exponential backoff per resource on failure (1m → 30m cap).
Honors 429 Retry-After. Only registered when RunMode=Client AND
FleetIngest__Enabled=true.
Admin side (ingest)
- /api/fleet/ingest: anonymous, X-Customer-Token authed against
fleet.Customers via SHA-256 indexed lookup. 401 on bad token; 400
on bad batch type.
- FleetIngestService dispatches by X-Batch-Type:
sites/devices → upsert by (CustomerId, Id) with ON CONFLICT UPDATE
measurements → bulk INSERT ON CONFLICT (Time, CustomerId, DeviceId)
DO NOTHING (idempotent under re-delivery).
- Updates fleet.Customers.FirstSeenAt/LastSeenAt on each successful batch.
- Writes fleet.IngestEvents audit row per batch (accepted, rejected,
bytes, client cursor, time-spread, error).
- FleetTimescaleBootstrapper runs after MigrateAsync in Admin mode:
CREATE EXTENSION timescaledb, create_hypertable on fleet.PowerMeasurements,
chunk interval 7 days, compression with segmentby=(CustomerId,DeviceId)
+ compress_orderby "Time" DESC, compression policy 7 days, hourly_per_device
continuous aggregate (realtime, materialized_only=false, 30-day start_offset
so back-fills get materialized on next refresh tick).
Wiring
- docker-compose.yml threads Application__RunMode + FleetIngest__* from
.env (defaults safely off) so a single dev host can run two stacks.
- .env.example documents the new vars under their own section.
Tests
- FleetIngestValidationTests (2 new). 53/53 passing.
Verified end-to-end on the dev host
- Client (portal-dev_portal, RunMode=Client, FleetIngest__Enabled=true)
pushes to Admin (portal-admin-test, RunMode=Admin, separate admin_fleet DB)
via container DNS.
- Customer registered on Admin (DEV0001), token captured, dropped into
Client .env, Client restarted, push service started on schedule.
- Ingested measurements (including a 2026-04-01 back-dated sample
simulating firmware replay) all land in fleet.PowerMeasurements with
the correct CustomerId.
- Customer.FirstSeenAt/LastSeenAt update, IngestEvents records every
batch (sites + devices per tick, measurements when cursor advances).
- Hypertable confirmed via timescaledb_information.hypertables;
hourly_per_device CA confirmed via timescaledb_information.continuous_aggregates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
79 lines
3.2 KiB
YAML
79 lines
3.2 KiB
YAML
# Local development stack.
|
|
# For production, see docker-compose.prod.yml (Traefik labels, no host ports, no anon Grafana).
|
|
services:
|
|
portal:
|
|
build: .
|
|
container_name: ${COMPOSE_PROJECT_NAME:-portal-dev}_portal
|
|
ports:
|
|
- "8080:8080"
|
|
environment:
|
|
- ASPNETCORE_ENVIRONMENT=${ASPNETCORE_ENVIRONMENT:-Development}
|
|
- Database__ConnectionString=Host=timescaledb;Port=5432;Database=${POSTGRES_DB:-power_monitoring};Username=${POSTGRES_USER:-power_user};Password=${POSTGRES_PASSWORD:-change_me_for_local_only}
|
|
- Database__AutoProvisionLocalTimescaleDb=false
|
|
# In the container the writable volume is /data/branding (Dockerfile chowns it).
|
|
# The appsettings.Development.json override of LogoStoragePath is for local `dotnet run`, not Docker.
|
|
- WhiteLabel__LogoStoragePath=/data/branding
|
|
- Authentication__DefaultAdminEmail=${Authentication__DefaultAdminEmail:-admin@example.com}
|
|
- Authentication__DefaultAdminPassword=${Authentication__DefaultAdminPassword:-ChangeMe123!}
|
|
- Grafana__BaseUrl=http://localhost:3001
|
|
- Grafana__InternalUrl=http://grafana:3000
|
|
# RunMode: Client (default) or Admin. Override in .env to test fleet aggregation locally.
|
|
- Application__RunMode=${Application__RunMode:-Client}
|
|
# Fleet ingest (Client mode): set Enabled=true + Url + Token to enable the push background service.
|
|
- FleetIngest__Enabled=${FleetIngest__Enabled:-false}
|
|
- FleetIngest__Url=${FleetIngest__Url:-}
|
|
- FleetIngest__Token=${FleetIngest__Token:-}
|
|
- FleetIngest__IntervalSeconds=${FleetIngest__IntervalSeconds:-60}
|
|
- FleetIngest__BatchSize=${FleetIngest__BatchSize:-5000}
|
|
depends_on:
|
|
timescaledb:
|
|
condition: service_healthy
|
|
volumes:
|
|
- portal-keys:/data/keys
|
|
- portal-branding:/data/branding
|
|
|
|
timescaledb:
|
|
image: timescale/timescaledb:2.17.2-pg16
|
|
container_name: ${COMPOSE_PROJECT_NAME:-portal-dev}_timescale
|
|
ports:
|
|
- "5433:5432"
|
|
environment:
|
|
- POSTGRES_DB=${POSTGRES_DB:-power_monitoring}
|
|
- POSTGRES_USER=${POSTGRES_USER:-power_user}
|
|
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-change_me_for_local_only}
|
|
volumes:
|
|
- timescale-data:/var/lib/postgresql/data
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-power_user} -d ${POSTGRES_DB:-power_monitoring}"]
|
|
interval: 5s
|
|
timeout: 5s
|
|
retries: 10
|
|
|
|
grafana:
|
|
image: grafana/grafana:11.4.0
|
|
container_name: ${COMPOSE_PROJECT_NAME:-portal-dev}_grafana
|
|
ports:
|
|
- "3001:3000"
|
|
environment:
|
|
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD:-admin}
|
|
- GF_SECURITY_ALLOW_EMBEDDING=true
|
|
- GF_AUTH_ANONYMOUS_ENABLED=true
|
|
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
|
|
- GF_USERS_ALLOW_SIGN_UP=false
|
|
- POSTGRES_DB=${POSTGRES_DB:-power_monitoring}
|
|
- POSTGRES_USER=${POSTGRES_USER:-power_user}
|
|
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-change_me_for_local_only}
|
|
volumes:
|
|
- grafana-data:/var/lib/grafana
|
|
- ./grafana/provisioning:/etc/grafana/provisioning:ro
|
|
- ./grafana/dashboards:/var/lib/grafana/dashboards:ro
|
|
depends_on:
|
|
timescaledb:
|
|
condition: service_healthy
|
|
|
|
volumes:
|
|
portal-keys:
|
|
portal-branding:
|
|
timescale-data:
|
|
grafana-data:
|