NightMare Networkv1.0.0-alpha.1

Search every page of the documentation. Or start with:

Docs / Getting started / Getting started

Getting started

Build a device that publishes a sensor, accepts writes to a setting and runs an Action, in three files. This is examples/basic from the repository; the rest of this guide then explains each piece and what else you can configure.

Quick start#

1. platformio.ini -- an Arduino-framework ESP32 project that depends on a release tag:

[env:esp32dev]
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.39/platform-espressif32.zip
framework = arduino
board = esp32dev
board_build.partitions = min_spiffs.csv
monitor_speed = 115200
monitor_filters = send_on_enter
; The serial console is off by default; on, it takes commands like ADOPT.
build_flags = -D NM_CONSOLE_SERIAL=1
lib_deps =
    https://github.com/vtmattedi/NightMareNetwork.git#v1.0.0-alpha.1

2. include/creds.h -- your Wi-Fi and the Local MQTT broker. The Remote fields only matter once you use a remote broker:

#pragma once
#define MQTT_CREDS_H

#define DEFAULT_SSID "your-wifi"
#define DEFAULT_PASSWORD "your-password"

#define LOCAL_MQTT_HOST "192.168.1.10"   // host only, no mqtt://
#define LOCAL_MQTT_PORT 1883

#define REMOTE_MQTT_URL "mqtt.example.com"
#define REMOTE_MQTT_PORT 8883
#define MQTT_USER "username"
#define MQTT_PASSWD "password"
#define ROOT_CA ""                       // the broker's CA PEM before using Remote MQTT

3. src/main.cpp -- declare what the device offers, bind handlers, start:

#include <Arduino.h>
#include <NightMare.h>

static ManagedSensor<uint32_t> uptimeSeconds("uptime_s");
static ManagedState<uint8_t> brightness("brightness");
static ManagedAction identify("identify");

static bool onBrightnessWrite(ManagedState<uint8_t> &, const uint8_t &requested)
{
    return requested <= 100; // false rejects the write; state stays unchanged
}

static ActionResult onIdentify(ManagedAction &, const std::string &payload)
{
    if (payload.length() != 0)
        return {false, "identify takes no arguments"};
    Serial.println("identify");
    return {true, "OK"};
}

static void publishUptime()
{
    uptimeSeconds.setValue(millis() / 1000UL);
}

static void preferLocalBroker(bool firstConnection)
{
    // The standard lifecycle starts Remote MQTT; use the local broker instead.
    if (firstConnection)
    {
        NightMare::Mqtt_enable(NightMare::MqttProfile::LOCAL);
        NightMare::SelectConnection(NightMare::ConnectionType::MQTT);
    }
}

void setup()
{
    Serial.begin(115200);
    brightness.onWrite = onBrightnessWrite;
    identify.onInvoke = onIdentify;
    uptimeSeconds.setValue(0);
    brightness.setValue(0);
    gScheduler.everyMonotonic("app.uptime", publishUptime, 1000);
    WiFi_onConnected(preferLocalBroker);
    startNightMareESP();
}

void loop()
{
    tickNightMareESP();
}

Build and flash it (pio run -t upload -t monitor). The device comes up under a name derived from its hardware, such as Esp32-nm-6ca172e0. Give it a name by typing on the serial monitor, or by publishing the same line to Esp32-nm-6ca172e0/console/in:

ADOPT desk-lamp

Then watch the broker (mosquitto_sub -h 192.168.1.10 -t 'desk-lamp/#' -v):

desk-lamp/status                       {"name":"desk-lamp",...,"online":true}
desk-lamp/manifest/msgpack             <the three Resources>
desk-lamp/resource/uptime_s/state      42
desk-lamp/resource/brightness/state    0

and drive it (mosquitto_pub -h 192.168.1.10 -t ... -m ...):

desk-lamp/resource/brightness/set      80      accepted: state becomes 80
desk-lamp/resource/brightness/set      180     rejected: state stays 80
desk-lamp/resource/identify/invoke             prints "identify"

That is the whole contract: no topic strings, subscriptions or reconnect handling in the firmware. The sections below take each piece in turn.

Requirements#

The current implementation targets:

framework: ESP-IDF or Arduino
platform:  espressif32

The library currently declares these dependencies:

ArduinoJson ^7.0.0

NM_ENABLE_OTA=1 requires an Arduino framework build because that optional module uses ArduinoOTA. The other modules use ESP-IDF APIs. Public text values use std::string, including Resource payloads and Config values. An existing LittleFS partition is reset when the new SPIFFS storage is first mounted. If a project includes ArduinoJson before NightMare headers, define ARDUINOJSON_ENABLE_STD_STRING=1 in its build flags.

With the default WiFi/MQTT feature set, a normal project needs:

include/creds.h

A project may optionally provide:

include/NightMareConfig.h
include/NightMareHardware.h

Add the library#

With PlatformIO, depend on a release tag of the repository:

lib_deps =
    https://github.com/vtmattedi/NightMareNetwork.git#v1.0.0-alpha.1

Pin a tag rather than the bare repository URL: without one, every build takes whatever is on the default branch at that moment. Use a commit hash instead of the tag only to try an unreleased fix. During development against a local checkout, a project can instead use a local/symlink dependency.

The public umbrella header is:

#include <NightMare.h>

or the compatibility umbrella:

#include <NightMareNetwork.h>

NightMareNetwork.h currently includes NightMare.h.

Configure features#

Create:

include/NightMareConfig.h

A useful explicit default configuration is:

#pragma once

#define NM_FIRMWARE_VERSION "1.0.0"

#define NM_ENABLE_SETTINGS 1
#define NM_ENABLE_RESOURCES 1
#define NM_ENABLE_NETWORK 1
#define NM_ENABLE_CONSOLE 1
#define NM_ENABLE_WIFI 1
#define NM_ENABLE_MQTT 1
#define NM_ENABLE_TELEMETRY 1
#define NM_ENABLE_JSON_WIRE 0
#define NM_ENABLE_SCHEDULER 1
#define NM_ENABLE_JOBS 1
#define NM_ENABLE_TIME_SYNC 1

#define NM_TIMEZONE "UTC0"
#define NM_NTP_SERVER_1 "pool.ntp.org"
#define NM_NTP_SERVER_2 "time.nist.gov"
#define NM_NTP_SERVER_3 "time.google.com"

#define NM_ENABLE_OTA 0
#define NM_ENABLE_HTTP 0
#define NM_ENABLE_WEBSOCKET 0
#define NM_ENABLE_LVGL 0

#define NM_ENABLE_ACTION_PAYLOAD_ASSERTION 0
#define NM_ENABLE_REMOTE_RESOURCE_VERIFICATION 1
#define NM_DEFAULT_MANIFEST_FORMAT msgpack

#define NM_TELEMETRY_INTERVAL_MS 60000UL
#define NM_NETWORK_TELEMETRY_INTERVAL_MS 300000UL
#define NM_IDENTITY_CLEANUP_RETRY_MS 60000UL
#define NM_SYSTEM_REQUEST_RETRY_MS 1000UL
#define NM_SYSTEM_REQUEST_MAX_RETRY_MS 300000UL

#define NM_SCHEDULER_OWN_TASK 1

#define NM_CONSOLE_BUILTINS 1
#define NM_CONSOLE_SERIAL 0

#define NM_LOG_LEVEL 0

Features.h supplies defaults for these macros, so NightMareConfig.h is optional. Providing it gives the consuming firmware one explicit place to record its feature choices and firmware version.

Configure WiFi and MQTT credentials#

With the default WiFi/MQTT features enabled, create:

include/creds.h

The current platform code expects:

#pragma once

#define MQTT_CREDS_H

#define DEFAULT_SSID "your-wifi"
#define DEFAULT_PASSWORD "your-password"

#define LOCAL_MQTT_HOST "192.168.1.10"
#define LOCAL_MQTT_PORT 1883

#define REMOTE_MQTT_URL "mqtt.example.com"
#define REMOTE_MQTT_PORT 8883

#define MQTT_USER "username"
#define MQTT_PASSWD "password"

static const char ROOT_CA[] = R"EOF(
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
)EOF";

Do not include mqtt:// in LOCAL_MQTT_HOST.

Do not include mqtts:// in REMOTE_MQTT_URL.

The connection builds those URI schemes itself.

Remote MQTT uses TLS and ROOT_CA.

Local MQTT currently uses plain:

mqtt://

Any MQTT broker works on either side. For a self-hosted Remote broker, Stackport-mqtt is the one the NightMare devices use: a TLS-only Mosquitto (port 8883, plus WebSocket for browsers) with a private CA, deployed through Stackport. Being TLS-only, it serves as the Remote broker; the Local broker stays a plain mqtt:// one on your network. Its hostnames are one deployment's values, so replace them with yours.

Optional hardware profile#

To make INFO report the host hardware and publish physical configuration, create:

include/NightMareHardware.h

Example:

#pragma once

#include <NightMare/HardwareProfile.h>

namespace NMHardware
{
inline Profile projectProfile()
{
    static const Terminal terminals[] = {
        {"GPIO9"},
        {"GND", CanonicalNet::Gnd},
    };
    static const Device devices[] = {
        {"mcu", terminals, 2, "ESP32-C3", "mcu", "ESP32-C3"},
    };
    static const ConnectorContact contacts[] = {{"SIGNAL"}, {"GND", CanonicalNet::Gnd}};
    static const Connector connectors[] = {
        {"j_switch", contacts, 2, "Wall switch", ConnectorKind::ScrewTerminal},
    };
    static const Connection internal[] = {
        {{"", EndpointKind::DeviceTerminal, "mcu", "GPIO9"},
         {"", EndpointKind::ConnectorContact, "j_switch", "SIGNAL"}},
        {{"", EndpointKind::DeviceTerminal, "mcu", "GND"},
         {"", EndpointKind::ConnectorContact, "j_switch", "GND"}},
    };
    static const HardwareDefinition definitions[] = {
        {"controller-v1", AssemblyKind::CustomBoard, "Controller",
         "esp32-c3-supermini:v1", nullptr,
         {nullptr, 0, devices, 1, connectors, 1, internal, 2}},
    };
    static const Assembly roots[] = {{"controller", "controller-v1"}};
    return {"controller", definitions, 1, roots, 1, nullptr, 0};
}
}

If this file is absent, NightMare reports:

host assembly: main
roots: [{ id: main, kind: generic }]

hostAssembly is the absolute ID path of the assembly running the firmware. Assemblies may reference reusable definitions. Devices expose terminals; connectors expose contacts. A physical connection that crosses an assembly boundary must connect two connector contacts. Electrical nets are inferred from the connection graph. See Hardware configuration v2.

Declare Resources#

Resources should normally have application lifetime.

A typical device can declare them globally:

#include <Arduino.h>  // Needed by the setup()/loop() example below.
#include <NightMare.h>

ManagedSensor<float> temperature("temperature");
ManagedState<bool> power("power");
ManagedAction identify("identify");

These mean:

temperature
    this device owns a read-only float Value

power
    this device owns a remotely writable bool Value

identify
    this device implements an Action

Implement a writable state#

A ManagedState<T> receives already-decoded values.

bool onPowerWrite(
    ManagedState<bool> &state,
    const bool &requested)
{
    digitalWrite(8, requested ? HIGH : LOW);

    // true means the requested value is accepted as authoritative state.
    return true;
}

Attach it before startup:

power.onWrite = onPowerWrite;

Returning false rejects the request and leaves Resource state unchanged.

Implement an Action#

A ManagedAction handler receives the canonical payload std::string and returns an ActionResult.

ActionResult onIdentify(
    ManagedAction &action,
    const std::string &payload)
{
    (void)action;

    if (payload.length() != 0)
        return {false, "identify takes no arguments"};

    Serial.println("I am here");
    return {true, "OK"};
}

Attach it:

identify.onInvoke = onIdentify;

For richer Actions, declare ActionArgMetadata and parse the payload in the handler.

Automatic Resource binding#

Each Resource enlists itself when constructed. startNightMareESP() binds the declared Resources in declaration order before networking starts. No explicit bindResource() calls are needed for Resources created before startup.

ResourcesManager does not own those objects. They must remain alive while bound.

Declaration before framework startup also matters for identity cleanup: retained state under an old adopted identity can only be removed for Resources that the current firmware has declared.

Give Managed Values an initial state#

A Managed Value has no authoritative state until it has been set.

Initialize values before networking if appropriate:

temperature.setValue(0.0f);
power.setValue(false);

Local Managed state does not depend on MQTT being connected.

Once MQTT connects, NightMare re-announces every Managed Value that has authoritative state.

Start NightMare#

A minimal setup is:

void setup()
{
    Serial.begin(115200);

    pinMode(8, OUTPUT);

    power.onWrite = onPowerWrite;
    identify.onInvoke = onIdentify;

    temperature.setValue(0.0f);
    power.setValue(false);

    startNightMareESP();
}

startNightMareESP() coordinates the enabled framework services.

With the normal feature set it:

initializes DeviceIdentity
restores bound Config values from persistent settings
restores Managed Value advertisement settings
restores Remote Resource source bindings
starts the Scheduler
installs pending identity-cleanup retry
starts periodic telemetry scheduling
starts NightMare::WiFiBegin()

MQTT is started by the first successful WiFi connection rather than directly by startNightMareESP().

Declare Configs before this call. Their constructor values are firmware defaults; after startup, valid persisted values have overwritten those defaults. Avoid consuming Config values as initialized settings before the restore phase completes.

Service the framework loop#

Call:

tickNightMareESP();

from loop():

void loop()
{
    tickNightMareESP();

    // Application cooperative work.
}

An ESP-IDF application uses app_main() instead. Declare Resources before startup, then call the same cooperative tick:

#include <NightMare.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

extern "C" void app_main()
{
    // Resources declared before this call are bound during startup.
    startNightMareESP();
    while (true)
    {
        tickNightMareESP();
        vTaskDelay(pdMS_TO_TICKS(10));
    }
}

With the default:

NM_SCHEDULER_OWN_TASK 1

the Scheduler owns its own FreeRTOS task.

tickNightMareESP() therefore does not also tick it.

If configured with:

NM_SCHEDULER_OWN_TASK 0

the same tickNightMareESP() call services the Scheduler cooperatively.

Publish sensor readings#

When the application obtains a new measurement:

void publishTemperature(float value)
{
    temperature.setValue(value);
}

For a bound ManagedSensor, NightMare publishes the encoded Value retained at:

<device>/resource/temperature/state

The application does not need to assemble that MQTT topic.

Observe another device#

Declare a RemoteSensor:

RemoteSensor<float> outsideTemperature("outside_temperature");

React to effective Value changes:

void onOutsideTemperature(
    NetValue<float> &resource,
    const float &value)
{
    Serial.printf("Outside: %.2f\n", value);
}

Then configure it before startup:

outsideTemperature.onUpdate = onOutsideTemperature;

outsideTemperature.setSource("weather-node", "temperature");

NightMare automatically subscribes to:

weather-node/manifest/msgpack
weather-node/resource/temperature/state

and reinstalls those subscriptions when the active transport reconnects.

Use a source selected at runtime#

A Remote Resource can be declared before it has a source:

RemoteSensor<float> selectedTemperature("selected_temperature");

void setup()
{
    startNightMareESP();
}

Later:

selectedTemperature.setSource(
    "weather-node",
    "temperature");

The manager replaces the old subscriptions, resets state learned from the previous source, saves the binding in /remoteresources.json, and republishes the consume manifest. startNightMareESP() restores the saved binding on the next boot after Resources are automatically bound and before networking starts.

Write another device's state#

Declare:

RemoteState<bool> bedroomPower("bedroom_power");

Configure its source:

bedroomPower.setSource("bedroom-ac", "power");

Request a change:

if (!bedroomPower.setValue(true))
{
    Serial.println("write was not accepted for connection");
}

RemoteState is optimistic by default.

Immediately after a successfully transported write:

bedroomPower.getValue();

shows the requested value for the optimistic window.

The owner's last reported truth remains available through:

bedroomPower.authoritativeValue();

Invoke a remote Action#

Declare and configure:

RemoteAction identifyRemote("bedroom_identify");

identifyRemote.setSource("bedroom-ac", "identify");

Invoke:

bool sent = identifyRemote.invoke();

true means the request was accepted for connection.

It is not a remote execution acknowledgement.

Ordinary Resource /invoke is fire-and-forget.

Add a Scheduler callback#

For recurring runtime work:

void sampleTemperature()
{
    // Read hardware.
    const float value = readTemperature();
    temperature.setValue(value);
}

void setup()
{
    // ...

    gScheduler.timer(
        "app.temperature",
        sampleTemperature,
        5000);

    startNightMareESP();
}

C++ callback jobs are MANAGED.

They cannot be removed by operator JOB CLEAR.

Scheduler ownership order#

A job may be added before gScheduler.begin() / startNightMareESP().

This is useful for application setup:

declare Resources
register callbacks/jobs
start framework

Direct gScheduler.tick() is also allowed for manual/cooperative use.

In the standard lifecycle, use tickNightMareESP() instead of manually ticking a Scheduler configured in TASK mode.

Operator commands#

With Console built-ins enabled, the command grammar includes:

PING
INFO ...
TIME
JOB ...
MQTT ...
NETWORK ...
CONFIG ...
> ...

Commands can arrive through:

serial console       when enabled
MQTT console
controlled MQTT / MQTTP
Scheduler std::string jobs

The > form routes command/control input to already-bound Resources. The character immediately after > is significant:

>list
>raw <topic> [payload]

> temperature
> power set true
> identify invoke {"mode":"blink"}

No space after > selects a ResourceManager operation such as list or raw.

A space after > selects a bound Resource by its unique local name. A bare Value reads its effective current Value; a bare Action invokes an empty payload. Explicit Resource verbs are get, set, invoke, source, advertise, and poll. The last two configure manager-owned advertisement policy and opted-in hardware polling metadata:

> temperature advertise -1
> temperature advertise 300000
> temperature poll 1000
> temperature poll reset

Advertisement is signed milliseconds: positive periodically reaffirms, 0 is event-driven, and negative suppresses publication. Suppression withdraws retained transport state without changing Resource availability or stopping local acquisition. poll requires the corresponding hardware capability flags.

>raw feeds an MQTT-shaped topic/payload through the Resource ingress path. For example:

>raw bedroom-ac/resource/power/set true

See Commands and Time.

What appears on MQTT#

For a device named:

living-room

the normal retained state may include:

living-room/status
living-room/info
living-room/telemetry/system
living-room/telemetry/network
living-room/telemetry/heartbeat
living-room/manifest/msgpack
living-room/resource/temperature/state
living-room/resource/power/state

Requests arrive at:

living-room/resource/power/set
living-room/resource/identify/invoke
living-room/console/in
living-room/console/controlled/<id>/in

See MQTT topics for the complete wire map.

Check the device identity#

The generated default network name is based on the board hardware:

Serial.println(gDeviceIdentity.getDeviceName().c_str());

Example:

Esp32-nm-6ca172e0

The same generated name is also available permanently as:

gDeviceIdentity.getHardwareSignature();

If the logical name is later adopted, the hardware signature remains unchanged.

The persisted POSIX timezone is available through:

gDeviceIdentity.getTimezone();

Use ADOPT <name> or CHANGE NAME <name> for identity migration, and TIMEZONE SET <posix-tz> or CHANGE TIMEZONE <posix-tz> to change local-time presentation through the command surface.

Coming from the older API#

Do not use the older Esp32Device, ResourceManager, NetEvent, or NetAction<T> examples as a guide for the current architecture. The active API is built around the global NightMare services plus Managed* / Remote* Resources.

Next steps#

Read these next:

Last updated · Edit this page on GitHub