Docs / Overview / NightMare Network
NightMare Network
NightMare Network is a small C++ framework and a set of MQTT conventions for ESP32 devices that need to participate in the same local network.
The project started from a recurring problem: the interesting part of an embedded project is usually the device itself, but every networked device ends up rebuilding the same surrounding machinery. It needs an identity, an MQTT address, reconnect behavior, retained state, discovery, remote control, scheduling, operator commands, telemetry, and enough conventions that another device or a backend can understand what it exposes.
NightMare makes those repeated concerns part of the framework.
The goal is not to turn every device into the same application. The goal is to make different applications participate in the network in the same way.
How it grew#
NightMare Network began as helpers reused across several firmware projects. As those projects needed to talk to their microcontrollers in the same way, the helpers became a common command resolver and MQTT addressing conventions. MQTT broker failover meant the device could keep using that conversation as network conditions changed.
MQTTP came next: a correlated request/response pattern over MQTT. It gave a caller its own reply topic while keeping the same command parser and project resolver. A command from an operator, another device, or a service could follow one grammar.
The current model makes application capabilities explicit. A device binds typed Managed and Remote Resources: Values for state, Actions for operations, and Events for occurrences. Resource manifests describe what a device offers; a separate consume manifest describes what it uses. A Remote Resource keeps a stable local name while its source binding can change. Config<T> declarations are bound into their own MessagePack declaration manifest and can be restored from persistent settings.
The C++ library now builds with ESP-IDF or Arduino. OTA is the optional Arduino-only module; the rest of the library does not require the Arduino framework.
What it is for#
NightMare is a fast starting point for a custom ESP32 device. Bind a Config<T> and its declaration appears in the Config manifest. Bind a ManagedSensor<T> and it appears in the Resource manifest; its readings have a standard state topic. The framework supplies the shared network behavior, while the application decides what the device senses, does, and exposes.
A device such as Watson, an acoustic detector, is the kind of problem this serves: its detection logic is specific to its hardware and purpose, but its configuration, events, commands, and network presence can use common tools. The framework gives that custom work a structure without prescribing its implementation.
Philosophy#
The central rule is:
Register once, participate automatically.
A Resource should not need custom MQTT code in every application. Once a Resource is declared and bound, the framework should know how to:
- publish its description,
- publish or consume its state,
- subscribe to the correct topics,
- restore those subscriptions after reconnect,
- route writes and invocations,
- and present the Resource consistently to other devices and tooling.
The same principle applies elsewhere. Device identity is owned in one place. Scheduler timing is implemented once. Telemetry has one owner. Common ESP32 startup is centralized instead of copied into every project.
This does not mean “hide everything.” NightMare deliberately keeps important semantics visible: who owns a value, whether a remote write is optimistic, whether a scheduled job survives reboot, and whether a message is retained are part of the programming model.
The network model#
The normal deployment model has a local MQTT broker inside a device cluster:
Device A ─┐
Device B ─┼── Local MQTT ── bridge ── Remote MQTT ── backend/services
Device C ─┘
The local broker is the cluster boundary. Devices inside the cluster can communicate without requiring the remote backend to be reachable.
The backend belongs on the remote side. It does not need to connect directly to each cluster's Local MQTT broker.
A cluster is a topology boundary. A namespace is a logical addressing boundary. They are intentionally different concepts. The current topic format is still device-rooted and does not yet include namespace as part of the MQTT address.
The core model#
A NightMare device is built from a few cooperating pieces:
DeviceIdentity
names the device and owns its local-time timezone
Resources
describe application capabilities and state
ResourcesManager
binds Resources to connection and routes Resource traffic
MQTT / NmMessageRouter
moves messages and dispatches NightMare protocol traffic
Scheduler
runs delayed or recurring command/callback work
Telemetry / INFO
exposes device description and runtime health
NightMareESP
starts common framework services and provides one cooperative tick point
Application code sits above those pieces. It declares Resources and implements what they mean.
Resources are the application contract#
NightMare models application-facing capabilities as Resources.
Values represent state. Actions represent operations.
Examples:
temperature
door_open
ac_power
target_temperature
set_timer
learn_ir_code
A Resource can be Managed by the current device or Remote, meaning it points at a Resource implemented by another device.
That distinction is permanent for the Resource object. Retargeting a Remote Resource changes what it points to; it does not turn it into a Managed Resource.
The network representation is intentionally small:
<device>/manifest/msgpack
<device>/manifest/json optional
<device>/manifest/consume/msgpack
<device>/manifest/consume/json optional
<device>/resource/<name>/state
<device>/resource/<name>/set
<device>/resource/<name>/invoke
The provider manifest describes Managed Resources. The separate consume
manifest is derived from valid bound Remote Resources. Retained /state
establishes availability and carries the owner's value. Remote freshness ages
separately from the last valid update, using the owner's advertised refresh
period.
Identity and presence#
A device identity exposes four related values:
- a logical device name used in MQTT topics,
- a stable hardware signature generated from the physical board,
- a raw machine-oriented device ID,
- a persisted POSIX timezone used for local-time presentation.
The logical name may be adopted. The hardware signature does not change when the device is renamed.
Presence is published separately from application state:
<device>/status
The retained status document carries the logical name, hardware signature, timezone, and presence. It has the same JSON shape for online state, graceful shutdown, and MQTT Last Will.
Resource availability and freshness do not depend on status. A device being online, a particular Resource being available, and its last update being fresh are three different facts.
Static information and telemetry#
NightMare separates information by lifecycle.
<device>/info
mostly static / boot-scoped description
<device>/hardware/msgpack
hardware configuration v2: assemblies, reusable definitions, devices,
connectors, and explicit physical connections; nets are inferred
<device>/telemetry/system
runtime system health
<device>/telemetry/network
network bookkeeping
<device>/telemetry/heartbeat
configurable, non-retained heartbeat
Application sensor/state data does not belong in telemetry. It belongs in Resources, where it already has ownership and freshness semantics.
Commands and Actions are different#
NightMare has an operator/framework command system and a Resource Action system.
Commands are used for things such as:
INFO
JOB
MQTT
NETWORK
CONFIG
REBOOT
Resource Actions represent application capabilities exposed by a device.
That separation is intentional. “Reconfigure the framework” and “tell this device to perform an application operation” are different kinds of work even when both can ultimately travel over MQTT.
Scheduling#
The Scheduler has one timing engine with several independent dimensions:
clock:
wall
monotonic
repeat:
one-shot
recurring
target:
command
callback
run mode:
TASK
MANUAL
scope:
MANAGED
USER
Framework/application C++ jobs are MANAGED. Jobs created through the operator JOB command are USER.
This boundary matters: an operator can list, delete, or clear USER jobs without accidentally deleting framework jobs such as telemetry publication.
Only wall-clock std::string command jobs persist across reboot. Callback pointers and monotonic deadlines are runtime state.
Common ESP32 lifecycle#
Applications normally register their resources and handlers, then call:
startNightMareESP();
The framework starts enabled common services such as identity, Scheduler, telemetry, WiFi/MQTT, and identity-cleanup retry.
The application loop calls:
tickNightMareESP();
for framework work that is intentionally cooperative. Components that already own a task or are event-driven are not duplicated there.
What NightMare is not#
NightMare is not a home automation platform, a universal optimizer for every subsystem it touches, or a framework for mass producing one fixed product. It is a set of practical tools for building a particular device with full control of its hardware and application behavior. It does not replace application logic, hardware drivers, or a general distributed database.
It also does not attempt to make every behavior transparent. Important limitations and trade-offs are documented explicitly, including the current ESP32 focus, the absence of namespace in topic addressing, the lack of a centralized runtime execution queue, and the deliberate use of plain function-pointer Scheduler callbacks.
The framework should remain small enough that those choices are understandable from the public model and the source.