CAMEL-24382: camel-azure-cosmosdb - process change-feed batches synchronously so events are not lost on failure - #25636
Open
oscerd wants to merge 1 commit into
Open
Conversation
…ronously so events are not lost on failure (apache#25617) CosmosDbConsumer.onEventListener dispatched the exchange fire-and-forget (getAsyncProcessor().process(exchange, EmptyAsyncCallback.get())) and returned immediately. The Azure SDK ChangeFeedProcessor uses a synchronous handleChanges handler and advances (checkpoints) the lease as soon as that handler returns - i.e. before the route processed the batch. On a route failure the lease had already advanced, so the batch was never redelivered and the change feed silently lost data (at-most-once). Process the batch synchronously and rethrow on failure, so the lease is only checkpointed after successful processing and a failed batch is redelivered on the next feed poll (at-least-once). The processing is extracted into a package-private processBatch(Exchange) seam covered by CosmosDbConsumerTest (failure rethrows, success returns normally). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andrea Cosentino <ancosen@gmail.com> (cherry picked from commit 4e94d72)
Contributor
|
🌟 Thank you for your contribution to the Apache Camel project! 🌟 🐫 Apache Camel Committers, please review the following items:
|
Contributor
|
🧪 CI tested the following changed modules:
🔬 Scalpel shadow comparison — Scalpel: 9 tested, 29 compile-only — current: 9 all testedMaveniverse Scalpel detected 38 affected modules (current approach: 9).
|
davsclaus
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #25617 (CAMEL-24382) to
camel-4.22.x.The bug
The
camel-azure-cosmosdbchange-feed consumer processed each batch with a fire-and-forget asynccallback (
getAsyncProcessor().process(exchange, EmptyAsyncCallback.get())). Azure'sChangeFeedProcessoradvances (checkpoints) the lease as soon as the event handler returns, so thelease could move past a batch before processing finished — and if processing then failed, those
events were silently lost.
The fix
Process each batch synchronously and rethrow on failure as a
RuntimeCamelException, so thehandler only returns normally once the exchange has completed. On failure the lease is not advanced
and the batch is redelivered on the next feed poll (at-least-once) instead of being dropped.
Backport notes
4e94d720ec40); no code adaptation was required(
camel-4.22.xalready usescamel-test-junit6, likemain).install -DskipTestsis green andCosmosDbConsumerTestpasses(2 tests), with no generated-file drift.
Claude Code on behalf of @oscerd