fix(core): retry invalid structured model responses - #1433
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Hi @BlueCatPro, thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you. |
sherryfox
left a comment
There was a problem hiding this comment.
Hey @BlueCatPro, thanks for this — and for the issue write-up that preceded it.
I left one parity question about what gets stored so please take a look 🙏
| + "\nRecall the set_model_response function correctly, fix the errors, and" | ||
| + " call it again with all required fields using the correct types."); | ||
| } | ||
| toolContext.actions().setSetModelResponse(args); |
There was a problem hiding this comment.
Python records the validated and dumped object here, not the input:
validated_response = self.output_schema.model_validate(args)
result = validated_response.model_dump(exclude_none=True)
...
tool_context.actions.set_model_response = resultSchemaUtils.validateMapOnSchema returns void, so there's nothing to store here but args. Confirmed on this branch: {"a":"x","b":null} passes validation unchanged and lands in setSetModelResponse as-is, where Python would emit {"a":"x"}.
Is the difference intended?
There was a problem hiding this comment.
Thanks for catching this, and sorry I missed this distinction. The difference was not intended.
I’ve updated SetModelResponseTool to create a copy after successful validation, omit null-valued fields to match the concrete model_dump(exclude_none=True) behavior you identified, and use the same result both as the tool return value and the value stored in EventActions.
6a562f2 to
fa32b78
Compare
| + "\nRecall the set_model_response function correctly, fix the errors, and" | ||
| + " call it again with all required fields using the correct types."); | ||
| } | ||
| // Match Python's model_dump(exclude_none=True) for Java's map-shaped response. |
There was a problem hiding this comment.
Thanks — this is what I meant, and using the same map for the return value and the actions is right.
One thing the loop does not cover: pydantic's exclude_none=True is recursive, and this filters only the top level. Same input both sides:
python {'name':'x','addr':{'city':'NYC','zip':None}} -> {'name':'x','addr':{'city':'NYC'}}
here {name=x, addr={city=NYC, zip=null}}
Narrow in practice — it needs a nested nullable property that the model actually sends as null, since SchemaUtils.matchType rejects a null on a non-nullable one before you get here. But the comment above says "Match Python's model_dump(exclude_none=True)" and the test is named ..._excludesNullFromReturnedAndRecordedResponse, so either the filter should recurse or those two should say top-level only.
There was a problem hiding this comment.
Ah, you’re right — my first pass missed nested nulls. Sorry about that. Just pushed a recursive fix + tests for nested objects and objects inside arrays.
| if (Objects.equals(funcResponse.name().orElse(""), SetModelResponseTool.NAME)) { | ||
| Object response = funcResponse.response(); | ||
| // The tool returns the args map directly. | ||
| Optional<Object> validatedResponse = functionResponseEvent.actions().setModelResponse(); |
There was a problem hiding this comment.
This matches _output_schema_processor.py exactly — reading the recorded value rather than the raw function response is what stops feedback being promoted, so no change asked for here.
Worth a test though: an event whose set_model_response never succeeded should yield Optional.empty() from getStructuredModelResponse, so the "feedback is never the final output" property is pinned rather than implied.
There was a problem hiding this comment.
Thanks for the suggestion. getStructuredModelResponse_withValidationFeedback_returnsEmpty covers this case by constructing validation feedback without a recorded setModelResponse and verifying that getStructuredModelResponse returns Optional.empty().
fa32b78 to
fddb96e
Compare
|
Thanks for the contribution, and welcome! This is a port of the Python set_model_response retry behavior but actual implementation looks to have a bug. SetModelResponseTool.java — excludeNullFields (the two overloads) |
Link to Issue or Description of Change
1. Link to an existing issue:
Problem:
When
set_model_responsearguments fail output-schema validation, the validation exception prevents the model from receiving actionable feedback and retrying with a corrected structured response.Solution:
set_model_responseso the model can retry.EventActions.Testing Plan
Unit Tests:
Tests executed:
EventActionsTestOutputSchemaTestSetModelResponseToolTestResult:
BUILD SUCCESSManual End-to-End (E2E) Tests:
Not run against a live model. A Runner-level regression test using the scripted
TestLlmverifies that validation feedback from the invalid call is included in the retry request and that only the corrected, validated response becomes the final structured output.Checklist
CONTRIBUTING.mddocument.Additional context
This is my first open-source contribution and my first contribution to adk-java. I may have missed some repository conventions, so I would appreciate any guidance and am happy to make follow-up changes. Thank you for your time and review!