Skip to content

T-4705 Do not ship log records emitted by the flush worker itself - #46

Merged
PetrHeinz merged 3 commits into
masterfrom
claude/t-4705-flusher-log-feedback-loop
Sep 24, 2026
Merged

PetrHeinz merged 3 commits into
masterfrom
claude/t-4705-flusher-log-feedback-loop

Conversation

@PetrHeinz

@PetrHeinz PetrHeinz commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

The Django setup in the docs puts the handler on the root logger. With that logger at DEBUG, every upload feeds the handler again: urllib3 logs each request it makes at DEBUG, that record lands in the handler's queue, the next flush uploads it, urllib3 logs that request as well, and so on. Reproduced against master with a local endpoint: a single logger.info() produced 9 uploads in 2 seconds, and the process never exited, because the worker drains the queue at shutdown and every drain refills it. In practice manage.py commands and the runserver reloader hang, while the same handler on an app logger works, since urllib3's records never reach it.

The first commit adds a test that attaches the handler to a logger the uploader itself logs to and expects flush() to return; it is expected to fail CI. The second commit made emit() drop records logged from a flush worker thread altogether. The third commit replaces that with shipping them opportunistically:

  • records logged from a flush worker are queued as TransportFrames, a marker subclass of dict that msgpack packs unchanged
  • a batch is uploaded only if it contains at least one regular record; a batch made solely of transport frames is discarded, since uploading it would just log again
  • the worker's own put never blocks, so it cannot wedge on its own full queue with drop_extra_events=False

So during runtime urllib3's two lines about an upload ride along with the next batch, while at flush() or exit, where a drain necessarily ends on a transport-only batch, they are dropped. Records from application threads are unaffected.

🤖 Generated with Claude Code

PetrHeinz and others added 3 commits September 23, 2026 16:30
…d's own log records

With the handler on a DEBUG root logger, urllib3's per-request record is queued, uploaded, logged again, and so on, so flush() never returns and the interpreter cannot exit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d of dropping them

Records logged from a flush worker are queued as TransportFrames; a batch made solely of them is discarded rather than uploaded, so the upload's own urllib3 lines ride along with the next real batch and can never trigger an upload by themselves.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@PetrHeinz
PetrHeinz marked this pull request as ready for review September 24, 2026 14:10
@PetrHeinz
PetrHeinz merged commit dee5880 into master Sep 24, 2026
14 checks passed
@PetrHeinz
PetrHeinz deleted the claude/t-4705-flusher-log-feedback-loop branch September 24, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant