Conversation
A CMD_GET_CONTACTS reply is streamed one frame per loop() pass, and the packet callbacks push frames (advert, path updated, message waiting, send confirmed, contact deleted, contacts full, raw/trace/log data) from that same loop() in between. A client reading the response therefore sees a push frame between RESP_CODE_CONTACTS_START and RESP_CODE_END_OF_CONTACTS, where it belongs to no response it asked for and cannot be told apart from a corrupt reply; for CONTACT_DELETED and CONTACTS_FULL it also describes a change to the very table being read, which invalidates the snapshot mid-read. Hold async pushes in a small bounded FIFO while the iterator is running, and flush them in order once the response has ended. Responses to commands the client sent itself are untouched, so nothing a client waits on is delayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happens
CMD_GET_CONTACTSis answered by aContactsIteratorthatMyMesh::checkSerialInterface()advances one frame perloop()pass:Everything else in
loop()runs first —BaseChatMesh::loop()and the packet callbacks — and those write push frames straight to the same serial interface (onDiscoveredContact0x80/0x8A,onContactPathUpdated0x81,processAck0x82,queueMessage/onChannelMessageRecv/onChannelDataRecv0x83,onContactOverwrite0x8F,onContactsFull0x90,logRxRaw,onContactResponse,onRawDataRecv,onTraceRecv, …).So a client that asks for contacts and reads until the response ends can receive this:
Two things are wrong with that. A push code in the middle of a response cannot be told apart from a corrupt reply by a client that is reading a response it asked for, and
PUSH_CODE_CONTACT_DELETED(0x8F) /PUSH_CODE_CONTACTS_FULL(0x90) specifically mean the table being read changed while it was being read — which is the one thing a snapshot read must not have to guess about.For context on how it was found, and to be upfront: a host driver of ours reads the response strictly and fails closed on a frame that belongs to no response, so it was seeing this about 25 times out of 30 reads. That strictness is defensible but you could reasonably call it a client bug. I am opening this because the alternative — every client has to know the full push-code table and decide for itself which codes invalidate a read in progress — seems worse than making the response contiguous, which costs the firmware very little.
The fix
Async pushes now go through a new
MyMesh::writePushFrame():loop()pass, in order, using the sameisWriteBusy()pacing.Once anything is held, further pushes keep queueing until it drains, so pushes are delivered in the order they were raised.
Responses to commands the client sent itself (
handleCmdFrame,writeOKFrame,writeErrFrame) are deliberately untouched — deferring those could delay something a client is waiting on. The FIFO isMAX_DEFERRED_PUSHES(default 8) ×MAX_FRAME_SIZE, overridable with-DMAX_DEFERRED_PUSHES=n; a full FIFO drops the push, which is what the BLE serial interface already does when its send queue is full.Cost
Measured with
pio runon the same tree before and after:RAK_4631_companion_radio_bleheltec_v4_companion_radio_usb+1,416 bytes of RAM in both cases (8 × 176 + 8 length bytes). On the nRF52 that is most of the cost; lowering
MAX_DEFERRED_PUSHESlowers it linearly if that matters on the smallest boards.Testing
Compiled for
heltec_v4_companion_radio_usb(ESP32-S3) andRAK_4631_companion_radio_ble(nRF52), bothSUCCESS.Not hardware tested. I have a pair of test radios on this firmware family and can flash them and re-run the read loop that found this, if you would rather see that before merging.