Understanding the Notecard Penalty Boxes
If you have a Notecard that is no longer communicating with Notehub or is repeatedly unable to connect to a cellular network, the Notecard may have been placed in a figurative "penalty box".
If lack of a reliable cellular connection is the reason your Notecard is not functional, you can get help troubleshooting your connection in this guide on Diagnosing Cellular Connectivity.
What is a Penalty Box?
A penalty box is a state in which the Notecard is isolated and not allowed to even attempt a connection to a cellular network. Repeating a network connection failure multiple times can unnecessarily drain an attached battery. Therefore, the penalty box is set up to preserve battery life while also occasionally "opening up" and allowing the Notecard to attempt a new connection at certain intervals.
If you have enabled the capture of trace logs on a Notecard, you may see log events that refer to the penalty box, such as:
{"text":"penalty: entering penalty box: connect delayed (89 min remaining): can't open session to notehub: socket open PPP: ppp connection timeout? {service}{extended-service-failure}"}
{"text":"penalty: removed from penalty box: penalty period expired"}
In this guide we will look at the reasons why a Notecard may be placed in a penalty box and under which conditions a Notecard may be removed from a penalty box.
The failure of a Notecard to sync with Notehub does not necessarily mean it is in a penalty box. It could also be due the use of an invalid ProductUID or incorrect Notehub host name. These can be diagnosed in the response to a hub.sync.status command. For example:
{"req":"hub.sync.status"}
{"alert":true,"status":"connect delayed (60 min remaining): can't open session to discovery service: socket: cannot find host: blah.notefile.net {service}{host-unreachable} {service}{extended-service-failure}","time":1638397092,"completed":24}
Types of Penalty Boxes
There are four types of penalty boxes that a Notecard may enter after repeated failures:
- Insufficient Power Supply
- Network Registration Failure
- Inability to Connect to Notehub
- Inability to Acquire GPS Satellite Fix
Each penalty box is explained below including commands on how to manually remove a Notecard from a penalty box (when available).
Insufficient Power Supply
If the Notecard reboots due to insufficient current supply while the modem is
trying to register the Notecard on a cellular network, it will enter a penalty
box. If the reboot is due to a card.restart
, card.restore
, or a DFU-induced
restart, the Notecard will not enter a penalty box.
The length of time the Notecard is in the penalty box is 1/2 of the outbound
sync interval or 120 minutes (whichever is greater). For each subsequent failure
during modem registration, the penalty box interval grows by 120 minutes, up to
a maximum of three days.
The Notecard will be released from the penalty box upon a successful registration on a cellular network, or if the host issues any of the following:
- A hub.sync request with the
allow:true
parameter. - A card.wireless.penalty
request with the
reset:true
parameter. - A hub.log request with the
sync:true
parameter. - Any change sent in a hub.set request.
Network Registration Failure
If the cellular modem on the Notecard cannot register with a network within 120 seconds (or within 15 minutes if using NB-IoT), the Notecard will enter a network registration penalty box.
The length of time the Notecard is in the penalty box is extended by each subsequent network registration failure, by a minimum of 15 minutes. For each failure the previous delay is multiplied by 1.25 + 15 minutes, up to a maximum of 3 days. For example, the first set of retry intervals (in minutes) are: 15, 34, 58, 86, 123, 169, 227, and 298.
As of Notecard firmware v5.3.1, if the Notecard is connected over USB, three short 3-8 minute retries will occur before it is placed into a network registration failure penalty box.
The Notecard will be released from the penalty box upon a successful registration on a cellular network, or if the host issues any of the following:
- A hub.sync request with the
allow:true
parameter. - A card.wireless.penalty
request with the
reset:true
parameter. - A hub.log request with the
sync:true
parameter. - Any change sent in a hub.set request.
Overriding Network Registration Penalty Box Settings
The misuse of this feature may result in the cellular carrier preventing the Notecard from future connections because it's effectively "spamming" the network. The cellular carrier may blacklist devices that it thinks are attempting to connect too frequently.
The default settings that determine the duration of the Network Registration Penalty Box are calculated based on the number of consecutive registration failures and can be overridden at runtime by using the card.wireless.penalty API:
{
"req": "card.wireless.penalty",
"set": true,
"rate": <PenaltyMultiplierRate(float)>,
"add": <PenaltyAddMins>,
"max": <PenaltyMaxMins>,
"min": <PenaltyMinMins>
}
If overriding the default settings with this API request, all of the above fields MUST be specified.
When using this API request, the first retry interval is PenaltyMinMins
. For
each successive failure, the prior delay is multiplied by
PenaltyMultiplierRate
plus another PenaltyAddMins
is added, up to a maximum
of PenaltyMaxMins
.
For example, by sending the following request, the retry intervals would be 5, 20, 50, 110, 230, 470, and 720 minutes:
{"req":"card.wireless.penalty","set":true,"rate":2.0,"add":10,"max":720,"min":5}
Likewise, the default settings for a Network Registration Penalty Box can be
also overridden with the
environment variable
_wireless_penalty
:
_wireless_penalty = <PenaltyMultiplierRate(float)>,<PenaltyAddMins>,<PenaltyMinMins>,<PenaltyMaxMins>
Inability to Connect to Notehub
If the cellular modem can successfully register with the network, but is unable to connect to Notehub, the Notecard may be placed in a "Notehub connection failure" penalty box.
The Notecard will enter this penalty box if connection failures to Notehub
happen more than 3 consecutive times (if the Notecard is battery-powered) or 6
consecutive times (if the Notecard is USB/line-powered). The Notecard will
remain in the penalty box for 1/2 of the configured outbound
sync interval or
60 minutes, whichever is greater. Note that this penalty box will only come into
play if there are no other network registration or power-related errors.
The Notecard will be released from the penalty box upon a successful connection to Notehub, or if the host issues any of the following:
- A hub.sync request with the
allow:true
parameter. - A card.wireless.penalty
request with the
reset:true
parameter. - A hub.log request with the
sync:true
parameter. - Any change sent in a hub.set request.
Inability to Acquire GPS Satellite Fix
If the Notecard is configured to acquire its location via GNSS (GPS) but is unable to make a position fix, it may be placed in a penalty box (provided all of the following conditions are met):
- The device is in motion (as determined by the on-board accelerometer).
- At least one GPS satellite is visible.
- A GPS position fix is NOT made within 15 minutes.
If the GPS is not able to track any satellites at all, it is powered off after 90 seconds of searching and not placed in a penalty box (or it will be removed from the penalty box if it was already in one).
The length of time the Notecard is in the penalty box is 30 minutes multiplied by the number of consecutive GPS position fix failures (up to a maximum of 12 hours). This maximum duration of 12 hours would be reached after approximately 5 days of continuous GPS position fix failures.
The Notecard will be released from the penalty box if either:
- The GPS makes a valid position fix.
- The GPS is unable to track any satellites for 90 seconds.