Skip to content

Hold pushes until the CMD_GET_CONTACTS response has finished - #3403

Open
l33tdawg wants to merge 1 commit into
meshcore-dev:devfrom
l33tdawg:fix/contacts-response-not-atomic-dev
Open

l33tdawg wants to merge 1 commit into
meshcore-dev:devfrom
l33tdawg:fix/contacts-response-not-atomic-dev

Conversation

@l33tdawg

@l33tdawg l33tdawg commented Sep 13, 2026

Copy link
Copy Markdown

What happens

CMD_GET_CONTACTS is answered by a ContactsIterator that MyMesh::checkSerialInterface() advances one frame per loop() pass:

} else if (_iter_started              // check if our ContactsIterator is 'running'
           && !_serial->isWriteBusy()) {

Everything else in loop() runs first — BaseChatMesh::loop() and the packet callbacks — and those write push frames straight to the same serial interface (onDiscoveredContact 0x80/0x8A, onContactPathUpdated 0x81, processAck 0x82, queueMessage / onChannelMessageRecv / onChannelDataRecv 0x83, onContactOverwrite 0x8F, onContactsFull 0x90, logRxRaw, onContactResponse, onRawDataRecv, onTraceRecv, …).

So a client that asks for contacts and reads until the response ends can receive this:

> 02 <count LE32>          RESP_CODE_CONTACTS_START
> 03 <contact 148 bytes>   RESP_CODE_CONTACT
> 80 <pubkey>              PUSH_CODE_ADVERT              <-- inside the response
> 03 <contact 148 bytes>   RESP_CODE_CONTACT
...
> 04 <lastmod LE32>        RESP_CODE_END_OF_CONTACTS

Two things are wrong with that. A push code in the middle of a response cannot be told apart from a corrupt reply by a client that is reading a response it asked for, and PUSH_CODE_CONTACT_DELETED (0x8F) / PUSH_CODE_CONTACTS_FULL (0x90) specifically mean the table being read changed while it was being read — which is the one thing a snapshot read must not have to guess about.

For context on how it was found, and to be upfront: a host driver of ours reads the response strictly and fails closed on a frame that belongs to no response, so it was seeing this about 25 times out of 30 reads. That strictness is defensible but you could reasonably call it a client bug. I am opening this because the alternative — every client has to know the full push-code table and decide for itself which codes invalidate a read in progress — seems worse than making the response contiguous, which costs the firmware very little.

The fix

Async pushes now go through a new MyMesh::writePushFrame():

  • not iterating → written directly, exactly as before;
  • iterating → copied into a small bounded FIFO;
  • once the response has ended → flushed one per loop() pass, in order, using the same isWriteBusy() pacing.

Once anything is held, further pushes keep queueing until it drains, so pushes are delivered in the order they were raised.

Responses to commands the client sent itself (handleCmdFrame, writeOKFrame, writeErrFrame) are deliberately untouched — deferring those could delay something a client is waiting on. The FIFO is MAX_DEFERRED_PUSHES (default 8) × MAX_FRAME_SIZE, overridable with -DMAX_DEFERRED_PUSHES=n; a full FIFO drops the push, which is what the BLE serial interface already does when its send queue is full.

Cost

Measured with pio run on the same tree before and after:

environment RAM before RAM after flash before flash after
RAK_4631_companion_radio_ble 149,912 (63.7%) 151,328 (64.3%) 475,960 477,240
heltec_v4_companion_radio_usb 103,488 104,904 650,237 650,469

+1,416 bytes of RAM in both cases (8 × 176 + 8 length bytes). On the nRF52 that is most of the cost; lowering MAX_DEFERRED_PUSHES lowers it linearly if that matters on the smallest boards.

Testing

Compiled for heltec_v4_companion_radio_usb (ESP32-S3) and RAK_4631_companion_radio_ble (nRF52), both SUCCESS.

Not hardware tested. I have a pair of test radios on this firmware family and can flash them and re-run the read loop that found this, if you would rather see that before merging.

A CMD_GET_CONTACTS reply is streamed one frame per loop() pass, and the packet
callbacks push frames (advert, path updated, message waiting, send confirmed,
contact deleted, contacts full, raw/trace/log data) from that same loop() in
between. A client reading the response therefore sees a push frame between
RESP_CODE_CONTACTS_START and RESP_CODE_END_OF_CONTACTS, where it belongs to no
response it asked for and cannot be told apart from a corrupt reply; for
CONTACT_DELETED and CONTACTS_FULL it also describes a change to the very table
being read, which invalidates the snapshot mid-read.

Hold async pushes in a small bounded FIFO while the iterator is running, and
flush them in order once the response has ended. Responses to commands the
client sent itself are untouched, so nothing a client waits on is delayed.
@l33tdawg l33tdawg changed the title Hold pushes until the CMD_GET_CONTACTS response has finished 🤖🤖 Hold pushes until the CMD_GET_CONTACTS response has finished Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant