From Simple to Efficient: Scaling Your Notecard Firmware Design

Every method of moving data between a host and the cloud sits somewhere on a continuum between simplicity and efficiency. At one end are approaches that get you from an idea to data in Notehub in an afternoon. At the other are approaches tuned for cost at scale: fewer bytes over the air, fewer events processed, less energy per reading, and less memory on the device. Notecard supports a number of approaches along that continuum, and the same note.add request introduced in the quickstart is the starting point for every step along the way.

You do not need to pick the efficient end on day one. Most successful deployments start simple, prove the product works, and then move along the line as volume grows and the bill for data and events becomes real. This guide describes four stages on that path, what each one costs you in development work, what constraint it lifts, and where in the docs to learn the details.

tip

Let AI write your firmware. Blues Expert MCP connects your AI coding assistant (Claude Code, GitHub Copilot, Cursor) directly to our API docs, providing live request validation and firmware best practices for Arduino, C, Zephyr, and Python. Install the Blues Expert MCP →

The Four Stages at a Glance

Stage	Approach	What you build	Constraint it lifts	What it earns you
1	Simple JSON messaging	Send and receive arbitrary JSON as Notes in Notefiles.	None yet. Notes are held in Notecard RAM: tens of bytes per Note, no more than 100 unsynced Notes per Notefile.	Proof of viability in hours. Immediate value.
2	Notefile templates	Declare the field names, types, and sizes of each Notefile up front.	Notes move to flash as fixed-length records. Hundreds of KB of storage, and the 100-Note limit goes away.	Products that generate more data, or sync less often, become feasible.
3	Templates with variable-length arrays and short binary payloads	Pack bursts of samples into a single Note; strip metadata with compact templates.	Same storage as stage 2, with far fewer bytes and events per sample.	Lower cellular data usage and fewer Notehub events per reading.
4	Live web requests with large binary payloads	Host manages a live connection and streams compressed binary through Notehub as a proxy.	Sized for large, occasional transfers. Not for isochronous media.	Media files, ML models, and datasets in the tens to hundreds of MB become viable.

Stage 1: Simple JSON Messaging

This is the pattern introduced in the quickstart. A note.add request takes any JSON object as its body, queues it in an outbound Notefile, and Notecard delivers it to Notehub on the next sync. Nothing about the structure has to be declared ahead of time, and the next Note can look completely different.

{
  "req": "note.add",
  "file": "sensors.qo",
  "body": {
    "temp": 21.4,
    "status": "ok",
    "door_open": true
  }
}

Two properties make this the right place to start. Store-and-forward is the default: Notes are saved on the Notecard first and synced later, so your host firmware never manages retries or buffering through a coverage gap. And the data is self-describing. Every event arriving in Notehub carries its own field names, so routes, JSONata transforms, and your cloud code can work with it immediately.

The trade-off is memory. Untemplated Notefiles are RAM-based and hold no more than 100 unsynced Notes each; the 101st note.add before a sync completes returns an error. Individual Notes should also stay under about 8 KB, and a binary payload on any Note is capped at 256 bytes. Every untemplated Note also carries its own field names and JSON punctuation. In Blues' measurements, that overhead is invisible when you sync one Note at a time (session overhead of about 1.2 KB dominates either way) and roughly doubles the cost of a sync carrying 100 Notes: 3.5 KB versus 1.6 KB templated. The more you batch, the more it matters.

Stage 1 is enough when your device reports infrequently, syncs at least as often as it fills the queue, and you are still learning what fields you actually need. Move on when you hit the 100-Note error, when your data usage in Notehub starts to matter, or when you need the same firmware to run over satellite or LoRa, both of which require templates.

Learn more

Adding Notes to Notefiles in Essential Requests
note.add API reference
The FAQ entry on how large a Note can be,
The measured figures in Data Usage Estimates.

Stage 2: Notefile Templates

If you can predict the shape of a Note, and that shape changes rarely, a template lets Notecard store and transmit it far more efficiently. A note.template request registers a schema against a .qo or .qos Notefile. The body you send is a set of hints: each key is a field you promise to send, and each value tells Notecard the type and width to reserve.

{
  "req": "note.template",
  "file": "sensors.qo",
  "body": {
    "temp": 14.1,
    "status": "-",
    "door_open": true,
    "cycles": 22
  }
}

Here 14.1 means a 4-byte float, "-" a variable-length string, true a boolean, and 22 a 2-byte unsigned integer. The full table of type hints is in Understanding Template Data Types. Once registered, Notecard stops storing JSON objects and writes fixed-length binary records to flash. Notehub reverses the process on arrival, so your routes and cloud code still receive ordinary JSON events. Nothing downstream changes.

The response to note.template includes a bytes field: the size of each Note as it will be transmitted, before compression. Use it. It is the fastest way to see what a design decision costs.

Subsequent note.add requests must match the template in field names and types. You cannot put an integer in a field declared as a string. Fields you omit are sent as empty values (see the omitempty behavior), which is a reason to keep templates tight rather than declaring every field you might someday want.

{
  "req": "note.add",
  "file": "sensors.qo",
  "body": {
    "temp": 21.4,
    "status": "ok",
    "door_open": true,
    "cycles": 47
  }
}

Issue the template at host boot, and again whenever an application configuration change alters the data shape (a different sensor count, for example). A template can be modified after Notes have been added to the Notefile; see Modifying a Template for how Notecard handles Notes already queued under the previous template, and Verifying a Template to confirm what is currently registered.

What you gain: the 100-Note limit no longer applies, storage per Note drops by an order of magnitude, and store-and-forward still works exactly as before. In Blues' published measurements on a cellular Notecard, a sync carrying 100 Notes dropped from 3.5 KB untemplated to 1.6 KB with a standard template. A sync carrying a single Note showed almost no difference, because TCP and TLS session overhead dominates at that size. Templates pay off when you batch, and batching is also what saves power.

What you give up: flexibility. You have to know your fields, their types, and their widths before you populate the Notefile. Templates apply to queue Notefiles (outbound .qo and .qos, and inbound queues on transports that require them), so .db Notefiles stay schemaless.

Learn more

Working with Note Templates
note.template API reference
The blog post Cut Your Data Usage in Half with Notefile Templates which walks through the byte math.

Stage 3: Variable-Length Arrays, Compact Templates, and Short Binary Payloads

Stage 2 makes each Note cheaper. Stage 3 reduces how many Notes you send at all, by packing several samples into one, and trims the metadata each Note carries.

Pack Bursts of Samples into One Note

Templated Notefiles support variable-length arrays. Declare an array field with a single integer or float hint, and later note.add requests can include any number of elements of that type.

{
  "req": "note.template",
  "file": "vibration.qo",
  "body": {
    "rms_g": 12.1,
    "samples": [12.1]
  }
}

The [12.1] is not a one-element array. It declares that samples holds 2-byte floats, as many as arrive:

{
  "req": "note.add",
  "file": "vibration.qo",
  "body": {
    "rms_g": 0.42,
    "samples": [0.11, 0.19, 0.31, 0.42, 0.28]
  }
}

For a device that samples every few seconds but only needs to report every few minutes, this turns dozens of Notes into one. That is fewer Notehub events, fewer bytes of framing, and better compression. Variable-length arrays are integer and float only; boolean and string arrays still need every element declared. See Using Arrays in Templates.

Strip Metadata with Compact Templates

By default a templated Note carries a creation timestamp, several location fields, and a timestamp for when the location was determined. Adding "format":"compact" to the template removes all of it. You can add back individual fields by naming them in the template body: _time, _lat, _lon, and _ltime.

{
  "req": "note.template",
  "file": "vibration.qo",
  "format": "compact",
  "port": 22,
  "body": {
    "_time": 14,
    "rms_g": 12.1,
    "samples": [12.1]
  }
}

In the measurements cited above, compact templates took the 100-Note sync from 1.6 KB down to 1.2 KB. Compact templates are required on any Notecard using satellite (NTN), where they also need a port between 1 and 100, and on Notecard for LoRa. Strings in compact templates are limited to 255 characters. Outside of those transports, weigh the savings against the debugging value of knowing when and where each Note was created. See Creating Compact Templates and Optimize Use of Compact Templates in the Satellite Best Practices guide.

Small Binary Payloads

When the data is already binary (a packed sensor frame, a small image thumbnail, an encrypted block), carry it in the Note's payload rather than base64-encoding it into a body field. Nothing needs to change in the template. Any note.add to a templated Notefile can include a payload of up to 256 bytes, with or without a body, and the same limit applies to untemplated Notes.

Pairing a payload with a variable-length array in the template lets you pack several binary segments into one Note and tell the cloud how to split them back apart. Declare a 1-byte unsigned integer array to hold the segment lengths:

{
  "req": "note.template",
  "file": "frames.qo",
  "body": {
    "payloadBytes": [21]
  }
}

Then send the packed bytes as the payload and the segment lengths in the body. Here three segments of 3, 7, and 4 bytes are packed into 14 bytes:

{
  "req": "note.add",
  "file": "frames.qo",
  "body": {
    "payloadBytes": [3, 7, 4]
  },
  "payload": "YWJjMTIzNDU2N0FCQ0Q="
}

Your service decodes the payload and slices it by the lengths in payloadBytes, recovering abc, 1234567, and ABCD. Although the payload appears as base64 in JSON, Notecard sends it over the air as binary, further compressed with Google's Snappy algorithm, so the base64 overhead you see in the request never reaches the network. See Using Templates with a Payload. For anything larger than 256 bytes, stage 4 is the approach.

Stage 4: Live Web Requests with Large Binary Payloads

Stages 1 through 3 all ride on the Notefile queue and inherit store-and-forward. Stage 4 leaves the queue behind. The host takes responsibility for the connection and moves large blocks of binary data through Notehub acting as a proxy to your cloud endpoint. This is the right approach for complex telemetry, Edge AI models, camera captures, and datasets in the tens to hundreds of MB. It is not designed for real-time or isochronous media.

Open a Live Connection

Web transactions require Notecard to be connected to Notehub. For periodic-mode devices, the recommended approach is to temporarily switch on a continuous connection with hub.set, do the work, and let the connection close after a period of inactivity:

{
  "req": "hub.set",
  "mode": "periodic",
  "on": true,
  "seconds": 300
}

Applications that need minimal latency in both directions can run in continuous mode full time, at a power cost. Avoid toggling between periodic and continuous in application code; use on and off instead, and steer clear of minimum and off modes unless you have a specific reason. Before issuing a web request, confirm the connection with hub.status and check that the response contains "connected": true.

{ "req": "hub.status" }

{ "status": "connected (session open) {connected}", "connected": true }

Learn more: hub.set and hub.status in the API reference, and Minimizing Latency for the full picture of continuous mode and the sync notification connection.

Send a Note Live, and Handle the Failure Yourself

The live argument on note.add bypasses saving the Note to flash. Paired with an open connection and "sync":true, it pushes the Note straight to Notehub. Check the response: if it contains an err field with {hub-not-connected}, the Note was not delivered and Notecard is not holding it for you.

{
  "req": "note.add",
  "file": "readings.qo",
  "live": true,
  "sync": true,
  "body": { "reading": "here's my string data", "values": [1, 2, 3, 4, 5] }
}

{ "err": "error adding note: not currently connected to notehub {hub-not-connected}" }

What you gain is a clear signal that a message was delivered, and freedom from the memory pressure of large Notefiles. What you take on is the storage and connectivity management that Notecard used to do for you. Decide up front what your application does when a live add fails. There are four reasonable answers.

Fall back to store-and-forward. Re-issue the note.add without live, so Notecard queues it in flash and delivers it on the next sync. Consider adding a max argument to cap how many such Notes can pile up, which matters for large Notes.
Drop it. Some data is ephemeral. If a reading cannot be delivered now, the next one supersedes it, and there is no point sending it late.
Wait and retry. When a cell signal fades or a WiFi router drops, Notecard may not yet know the connection is gone, and hub.status can report connected for a while after the link is dead. Loop on hub.status until it reports a connection, then retry. Always put a timeout on this loop so the host does not wait forever for a connection that is not coming.
Buffer on the host. If the host has spare RAM, flash, or a file system, hold the data locally and retry later.

Move Large Binary Blocks Through card.binary and web.post

For payloads too big for a Note, Notecard reserves a binary buffer (about 100 KB; the exact figure is the max field in a card.binary response). The pattern is: load the buffer from the host, send it with a single web.post to a Notehub proxy route, verify, clear the buffer, repeat. The SDKs (note-c, note-arduino, note-python) include helpers for the host-side transfer and MD5 verification.

{
  "req": "web.post",
  "route": "PostBinaryDataRoute",
  "binary": true,
  "verify": true,
  "content": "application/octet-stream"
}

{ "req": "card.binary", "delete": true }

Objects larger than the buffer are sent as a series of flushes and reassembled at your cloud endpoint. If you would rather Notehub do the reassembly, web.post also accepts base64 payload fragments that Notehub stitches together before invoking your route, at the cost of about 33% more bandwidth and host-managed offsets and checksums. Two things to know before you commit: the data in the binary buffer is not stored in a Notefile, so there is no store-and-forward, and Notehub removes any payload larger than 256 bytes from the stored event once it has been routed. Your endpoint is the system of record.

Learn more:

Sending and Receiving Large Binary Objects (including the fragment upload alternative)
Web Transactions for proxy route setup
card.binary API reference.

Choosing Your Stage

A few questions settle most cases.

Are you still changing what fields you send? Stay at stage 1. Templates reward stability, and you do not have it yet.

Have you hit the 100-Note error, or do you want to sync less often to save power? Move to stage 2. It costs one note.template request at boot and changes nothing in your cloud.

Does your device sample far more often than it needs to report, or is your Notehub event count or cellular data usage driving cost? Move to stage 3. Variable-length arrays are the single most under-used efficiency feature on the platform.

Are you targeting satellite or LoRa? Stage 3 with compact templates is mandatory, and your Note sizes must fit the network's packet limits.

Do you need to move a file, a model, or an image? That is stage 4, and only stage 4. Keep your telemetry on the Notefile queue at whatever stage suits it, and use the binary path for the big transfers alone.

The stages are not exclusive. A mature product commonly runs compact templated telemetry (stage 3) alongside an occasional stage 4 model download, with a stage 1 .db Notefile for configuration. Measure as you go: Measuring Data Usage shows how to read per-device consumption from Notehub, and Data Usage Estimates gives baseline numbers for each sync mode.