NightMare Networkv1.0.0-alpha.1

Search every page of the documentation. Or start with:

Docs / Modules / ESP32 platform lifecycle

ESP32 platform lifecycle

The active NightMare implementation targets:

ESP-IDF or Arduino
ESP32 / espressif32

Non-OTA modules use ESP-IDF, FreeRTOS, SPIFFS, and esp_mqtt_client directly. The optional OTA module uses ArduinoOTA and requires the Arduino framework.

The main project-facing lifecycle is:

startNightMareESP();
tickNightMareESP();

Main include#

Most applications can include:

#include <NightMare.h>

This exposes the enabled NightMare modules according to compile-time feature configuration.

NightMareConfig.h#

A consuming project may provide:

include/NightMareConfig.h

Features.h includes it automatically when present.

The project can define feature macros before NightMare's defaults are applied.

Example:

#pragma once

#define NM_FIRMWARE_VERSION "1.2.0"

#define NM_ENABLE_OTA 1
#define NM_ENABLE_HTTP 0
#define NM_ENABLE_WEBSOCKET 0

#define NM_SCHEDULER_OWN_TASK 1

#define NM_TIMEZONE "UTC0"
#define NM_NTP_SERVER_1 "pool.ntp.org"
#define NM_NTP_SERVER_2 "time.nist.gov"
#define NM_NTP_SERVER_3 "time.google.com"

#define NM_TELEMETRY_INTERVAL_MS 60000UL
#define NM_NETWORK_TELEMETRY_INTERVAL_MS 300000UL

Default feature flags#

Current defaults are:

NM_ENABLE_SETTINGS             1
NM_ENABLE_RESOURCES            1
NM_ENABLE_NETWORK              1
NM_ENABLE_CONSOLE              1
NM_ENABLE_WIFI                 1
NM_ENABLE_MQTT                 1
NM_ENABLE_TELEMETRY            1
NM_ENABLE_SCHEDULER            1
NM_ENABLE_JOBS                 1
NM_ENABLE_TIME_SYNC            1

NM_ENABLE_OTA                  0
NM_ENABLE_HTTP                 0
NM_ENABLE_WEBSOCKET            0
NM_ENABLE_LVGL                 0

NM_ENABLE_ACTION_PAYLOAD_ASSERTION     0
NM_ENABLE_REMOTE_RESOURCE_VERIFICATION 1

NM_CONSOLE_BUILTINS            1
NM_CONSOLE_SERIAL              0

The platform marker is currently:

NM_PLATFORM_ESP32 1

and is reserved for a future platform split.

Time defaults#

When time synchronization is enabled, the current defaults are:

NM_TIMEZONE      "UTC0"
NM_NTP_SERVER_1  "pool.ntp.org"
NM_NTP_SERVER_2  "time.nist.gov"
NM_NTP_SERVER_3  "time.google.com"

NM_TIMEZONE is the default POSIX timezone string. DeviceIdentity persists runtime changes under its private settings and applies the active value for local-time formatting. Epoch timestamps remain UTC-based.

Important configurable intervals#

Defaults:

NM_TELEMETRY_INTERVAL_MS             60000
NM_NETWORK_TELEMETRY_INTERVAL_MS   300000
NM_IDENTITY_CLEANUP_RETRY_MS       60000
NM_SYSTEM_REQUEST_RETRY_MS          1000
NM_SYSTEM_REQUEST_MAX_RETRY_MS    300000

Scheduler execution mode#

NM_SCHEDULER_OWN_TASK 1
    startNightMareESP() selects Scheduler TASK mode

NM_SCHEDULER_OWN_TASK 0
    startNightMareESP() selects Scheduler MANUAL mode

In MANUAL mode:

tickNightMareESP();

drives the Scheduler.

Resource housekeeping is cooperative regardless of Scheduler mode. With Resources enabled, tickNightMareESP() calls gResourcesManager.tick(), which performs at most one due manifest retry or inspects at most one Resource for advertisement refresh/retry or Remote freshness aging. It creates neither a Scheduler job nor a dedicated task.

Compile-time dependencies#

The current feature graph enforces:

MQTT
    -> NETWORK

NETWORK
    -> RESOURCES

TELEMETRY
    -> SCHEDULER
    -> NETWORK

JOBS
    -> SCHEDULER

SCHEDULER
    -> SETTINGS
    -> CONSOLE

CONSOLE
    -> SETTINGS

TIME_SYNC
    -> WIFI
    -> SETTINGS

WIFI
    -> SETTINGS

OTA
    -> WIFI
    -> SETTINGS

HTTP
    -> NETWORK
    -> CONSOLE

WEBSOCKET
    -> HTTP

CONSOLE_SERIAL
    -> CONSOLE

The cooperative deferred-publication processor checks NmConnection readiness, not MQTT readiness, so the standard lifecycle runs the same over MQTT and ESP-NOW.

These are current implementation dependencies, not necessarily permanent architectural requirements.

HardwareProfile#

Projects can optionally provide:

NightMareHardware.h

That file should expose the project's hardware profile through:

NMHardware::Profile projectProfile();

The model is built from Assembly, HardwareDefinition, Device, Terminal, Connector, ConnectorContact, EndpointRef, and Connection. Arrays are static pointer/count pairs suitable for firmware declarations. Profile contains the reusable definitions, one or more deployed root assemblies, the absolute host-assembly path, and deployment-level connections.

Endpoint paths inside definitions or assemblies are relative to that assembly; top-level endpoint paths are absolute from a deployment root. Slash is reserved as the path separator and cannot appear in an ID.

If no NightMareHardware.h exists, NightMare returns:

host assembly: main
roots:          [{ id: main, kind: Generic }]
definitions:    none
connections: none

The host assembly model feeds INFO/HARDWARE. The complete source configuration is retained canonically at <device>/hardware/msgpack; optional wire JSON is at <device>/hardware/json, and readable JSON remains available through HW.

validateHwConfig() checks references, IDs, definition cycles, connector-only assembly crossings, self-connections, duplicates, capacity, and canonical-net contradictions. buildTopologyGraph() creates endpoint nodes and physical edges; inferNets() computes connected components. See Hardware configuration v2 for the normative schema.

startNightMareESP()#

The standard startup helper performs common framework setup in this order.

1. Device identity#

gDeviceIdentity.begin();

2. Scheduler#

When enabled:

gScheduler.begin(
    NM_SCHEDULER_OWN_TASK
        ? SchedulerRunMode::TASK
        : SchedulerRunMode::MANUAL);

3. Pending identity cleanup retry#

When Scheduler + MQTT are enabled and cleanup is pending, NightMare installs:

_nm_identity_cleanup

as a recurring MANAGED callback job.

4. Telemetry#

When enabled:

Telemetry.start();

installs the periodic system/network publication jobs.

5. WiFi#

When enabled:

    NightMare::WiFiBegin();

starts the asynchronous WiFi path.

Resource registration comes first#

Application Resources should be declared before:

startNightMareESP();

Example:

void setup()
{
    power.onWrite = onPowerWrite;

    startNightMareESP();

    startApplicationHardware();
}

startNightMareESP() binds those declarations before MQTT participation and before old-identity cleanup may run.

tickNightMareESP()#

The framework cooperative tick currently handles:

one Resource housekeeping step
    when NM_ENABLE_RESOURCES

Time synchronization completion
    when NM_ENABLE_TIME_SYNC

Scheduler
    only when Scheduler mode is MANUAL

one pending `SystemRequest`

Serial console
    only when NM_ENABLE_CONSOLE && NM_CONSOLE_SERIAL

The optional serial console reads nonblocking ESP-IDF standard input and dispatches complete newline-terminated commands.

SNTP completion is intentionally dispatched here because the ESP callback runs on lwIP's task. NightMare defers the SystemFlag::TimeSynced transition and the application onTimeSync() callback to the normal cooperative context.

Task-driven/event-driven subsystems such as MQTT and WiFi are not otherwise polled here.

WiFi project credentials#

When WiFi is enabled, the current platform code includes a consuming-project header:

creds.h

and requires:

DEFAULT_SSID
DEFAULT_PASSWORD

The project, not the NightMare library, owns this credential file.

Direct ESP WiFi driver#

NightMare::WiFiBegin();

starts the radio (Network/WiFiRadio/, via WiFiRadioBegin()), loads the stored profile and starts the ESP-IDF station driver in Network/WiFiIP/NmWifiEsp.*, which uses esp_wifi, esp_netif, and ESP events directly and has no Arduino, PersistentSettings or identity dependency. Storage, the hostname and the first-connection services live in Network/WiFiIP/NmWifiService.*.

startNightMareESP() starts the radio on its own whenever NM_ENABLE_WIFI_RADIO is on, before (and independently of) WiFiBegin(), so an ESP-NOW device with NM_ENABLE_WIFI 0 gets a running radio and never joins an AP.

WiFiBegin() initializes PersistentSettings.

If either stored key is absent:

_ssid
_password

the current implementation writes defaults from:

DEFAULT_SSID
DEFAULT_PASSWORD

Then it starts an asynchronous connection using the stored values. If an attempt remains disconnected for 15 seconds, the asynchronous task retries at the next transmit-power level supported by the ESP32 driver. When one of those fallback levels connects, WiFiBegin() persists it with the rest of the WiFi profile. Its recovery task remains active after connection so a later loss uses the same retry sequence.

WiFi hostname and identity#

Both synchronous and asynchronous WiFi connection paths set the hostname from:

gDeviceIdentity.getDeviceName()

and lock the identity address before network participation.

This is one reason a later adoption may wait for reboot rather than changing the running name immediately.

Async WiFi monitor#

The driver uses ESP events for connection state and a monitor task for timed retries and transmit-power fallback:

wifi_monitor

with:

stack:     6144 bytes
priority:  1
core:      tskNO_AFFINITY

The event handlers run on the ESP default event loop task (sys_evt), whose stack has no room for a formatted log line or the connected chain. They only record what happened; the monitor task logs it (start, association channel, disconnect reason) and delivers the state callback -- which is why it needs the larger stack: on CONNECTED that callback starts OTA, SNTP and the network connection and saves to NVS.

Using no fixed core allows the same code to run on single-core ESP variants such as C3/C6/H2/S2.

A disconnected attempt is retried every 15 seconds while cycling through the driver's supported transmit-power levels. The task stays active after connecting and applies the same recovery sequence after a later disconnect. A fallback level that connects is persisted with the rest of the profile.

First WiFi connection#

On the first successful WiFi connection in a boot, the current implementation starts enabled network services:

OTA
SNTP time synchronization

in that order. WiFiIP also reports its state to an already-enabled MQTT service and to NmConnection. The router may select an active usable transport, but it never starts or stops a connectivity service.

Specifically:

initOTA();
NightMare::SelectConnection(configuredConnection);
startSntpTimeSync();

according to feature flags.

The initial concrete connection is the head of the failover order built from preferredConnection, whose build default is Remote MQTT. tickNightMareESP() also drives failover; see the Network module.

startSntpTimeSync() configures the ESP32 SNTP client and returns immediately. Completion is applied later through tickNightMareESP().

Later WiFi reconnects#

The WiFi layer tracks whether this is its first successful connection.

The first-connection initialization block is not rerun on every later WiFi reconnect.

Long-lived network services are expected to manage their own connection/reconnect lifecycle once started.

WiFi callback#

Projects can register:

WiFi_onConnected(callback);

with signature:

void callback(bool firstConnection);

The callback runs whenever the WiFi monitor detects a connection and tells the project whether this is the first successful connection handled by that module.

WiFi credential change#

NightMare::WiFiApplyProfile(profile);

does the following:

  1. calls WiFi_changeProfile(), which stops the recovery task and disconnects current WiFi,
  2. tries the new credentials synchronously for up to 15 seconds,
  3. if they fail, restores the previous credentials,
  4. if they succeed, writes the profile (SSID, password and TX power) to PersistentSettings and resumes the recovery task.

The private storage keys should not be manipulated directly by applications.

MQTT platform connection#

The ESP MQTT implementation uses:

esp_mqtt_client
FreeRTOS control queue/task

Local broker:

mqtt://...

Remote broker:

mqtts://...

The remote broker is verified with the project-provided:

ROOT_CA

See Network and MQTT.

Time synchronization#

Automatic time synchronization now uses the ESP32 SNTP client rather than a blocking HTTP request.

The canonical API is:

startSntpTimeSync();

It uses the process TZ value applied by DeviceIdentity, plus NM_NTP_SERVER_1..3.

Synchronization is asynchronous. The SNTP callback marks a pending event; tickNightMareESP() later calls processTimeSyncEvents() so the SystemFlag::TimeSynced transition and application callback do not run on lwIP's task.

See Time for the complete clock, timezone, formatting, and synchronization API.

MQTT-assisted time sync#

When MQTT connects and NightMare wall time is invalid, it can also publish:

Control/request

with payload:

time

and consume a Control/time JSON response.

That path ultimately calls:

manualSyncTime(timestamp);

Time-sync runtime state#

Successful time sync updates:

SystemState.set(SystemFlag::TimeSynced)

and invokes the optional callback installed by:

onTimeSync(callback);

OTA#

OTA is disabled by default:

NM_ENABLE_OTA 0

When enabled, it depends on WiFi + Settings and is initialized on the first successful WiFi connection.

OTA is a platform service, not part of the Resource protocol.

HTTP, WebSocket, and LVGL#

These modules remain optional:

NM_ENABLE_HTTP
NM_ENABLE_WEBSOCKET
NM_ENABLE_LVGL

An ESP-IDF build with NM_ENABLE_WEBSOCKET=1 also needs CONFIG_HTTPD_WS_SUPPORT=y in sdkconfig.defaults.

They are not part of the core Resource architecture and have not received the same freeze-level architecture pass.

New core design should not use those modules as precedent for Resource/network semantics.

Platform boundary today#

The current library already separates several concepts cleanly:

Resource model
DeviceIdentity state
Scheduler model
Telemetry model
NmConnection vs NmMqttConnection vs ESP MQTT driver

but the complete library is not yet portable beyond ESP32.

Current platform-specific dependencies include:

ESP hardware APIs
FreeRTOS
ESP-IDF SPIFFS
esp_wifi / esp_netif
esp_mqtt_client
ArduinoOTA
esp_sntp

A future platform split can move those implementations without requiring the Resource protocol itself to become platform-specific.

Last updated · Edit this page on GitHub