feat(harness): add a Gemini CLI harness (tap, turn, and init templates) - #516
feat(harness): add a Gemini CLI harness (tap, turn, and init templates)#516michaelxu2288 wants to merge 1 commit into
Conversation
Adds Gemini CLI as a framework harness alongside Claude Code and Codex: - convert_gemini_cli_to_agentex_events: maps the CLI's stream-json events (init, message deltas, tool_use, tool_result, error, result; schema per packages/core/src/output/types.ts in google-gemini/gemini-cli) onto the canonical StreamTaskMessage* stream. Assistant deltas open one text slot that closes on the next tool event, the result, or end of stream, so every Start has a Done; tool requests and results pair by tool_id. - GeminiCliTurn: HarnessTurn wrapper exposing session_id and model from the init event and normalising result.stats into TurnUsage. - Both exported from agentex.lib.adk. - agentex init templates sync-gemini-cli, default-gemini-cli and temporal-gemini-cli (registered in TemplateType, file map and menus), cloned from the Claude Code templates: prompt passed via -p with stdin closed (the CLI reads stdin to EOF in headless mode), optional GEMINI_MODEL, GEMINI_API_KEY credential, npm install -g @google/gemini-cli in the Dockerfile. Turns are independent prompts: the CLI's --resume takes latest/index, not a session id. - Tests: tap (text deltas, whole messages, tools, errors, callbacks, source close on cancel), turn (usage mapping, protocol), harness end to end through UnifiedEmitter with span derivation; template suite covers the three new templates. Offline tests only; a live smoke run needs a Gemini API key. Claude-Session: https://claude.ai/code/session_01HCVKnA7LeJZ44nxZz1uzF3
| elif evt_type == "result": | ||
| done = _close_text() | ||
| if done is not None: | ||
| yield done | ||
| if on_result is not None: | ||
| await on_result(evt) |
There was a problem hiding this comment.
Terminal failures appear successful
A terminal Gemini result can have status: "error", but this branch closes the text stream and invokes the normal result callback without checking the status or error payload. UnifiedEmitter therefore treats authentication, turn-limit, and other terminal failures as successful completion, potentially returning missing or partial output instead of surfacing the error. Standalone error events are also logged without entering the canonical stream.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/lib/adk/_modules/_gemini_cli_sync.py
Line: 262-267
Comment:
**Terminal failures appear successful**
A terminal Gemini `result` can have `status: "error"`, but this branch closes the text stream and invokes the normal result callback without checking the status or error payload. `UnifiedEmitter` therefore treats authentication, turn-limit, and other terminal failures as successful completion, potentially returning missing or partial output instead of surfacing the error. Standalone error events are also logged without entering the canonical stream.
**Knowledge Base Used:**
- [Agent framework integrations](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/agent-framework-integrations.md)
- [Harness delivery and metrics](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/harness-and-metrics.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| if proc.returncode is None: | ||
| try: | ||
| proc.terminate() | ||
| except ProcessLookupError: | ||
| pass | ||
| await proc.wait() |
There was a problem hiding this comment.
Cancellation can hang indefinitely
If the Gemini CLI or an active tool subprocess does not exit after SIGTERM, this unbounded proc.wait() blocks generator cleanup indefinitely. As a result, request cancellation or Temporal activity cancellation can remain hung. The same cleanup pattern also appears in the sync template and temporal-gemini-cli/project/activities.py.j2; each path needs a bounded graceful shutdown followed by forced termination.
Knowledge Base Used: Temporal execution
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/lib/cli/templates/default-gemini-cli/project/acp.py.j2
Line: 121-126
Comment:
**Cancellation can hang indefinitely**
If the Gemini CLI or an active tool subprocess does not exit after SIGTERM, this unbounded `proc.wait()` blocks generator cleanup indefinitely. As a result, request cancellation or Temporal activity cancellation can remain hung. The same cleanup pattern also appears in the sync template and `temporal-gemini-cli/project/activities.py.j2`; each path needs a bounded graceful shutdown followed by forced termination.
**Knowledge Base Used:** [Temporal execution](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/scale-agentex-python/-/docs/temporal-execution.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
What
Adds Gemini CLI as a framework harness, mirroring the Claude Code and Codex ones:
convert_gemini_cli_to_agentex_events(src/agentex/lib/adk/_modules/_gemini_cli_sync.py): maps the CLI'sstream-jsonevents (init,messagedeltas,tool_use,tool_result,error,result; schema frompackages/core/src/output/types.tsin google-gemini/gemini-cli) onto the canonicalStreamTaskMessage*stream. Assistant deltas open one text slot that closes on the next tool event, theresult, or end of stream, so everyStarthas aDone; tool requests and results pair bytool_id; afinallycloses the source iterator on cancellation like the other taps.GeminiCliTurn(_gemini_cli_turn.py):HarnessTurnwrapper exposingsession_idandmodelfrominitand normalisingresult.statsintoTurnUsage(tokens, cached, duration, tool calls; cost is not reported by the CLI).agentex.lib.adk.agentex inittemplatessync-gemini-cli,default-gemini-cli,temporal-gemini-cli, registered inTemplateType, the file map and the menus. Prompt passed via-pwith stdin closed (in headless mode the CLI reads stdin to EOF and appends it), optionalGEMINI_MODEL,GEMINI_API_KEYcredential,npm install -g @google/gemini-cliin the Dockerfile. Turns are independent prompts because the CLI's--resumetakeslatest/index rather than a session id; the Temporal template keeps the reportedsession_idfor observability only and says so.Companion docs page: scaleapi/scale-agentex#429.
Test
tests/lib/adk/test_gemini_cli_sync.py(12),tests/lib/adk/test_gemini_cli_turn.py(12),tests/lib/core/harness/test_harness_gemini_cli_sync.py(5): pass.tests/lib/cli/test_init_templates.py: 29 passed, including the three new templates; all rendered projects byte-compile.geminibinary (needs a Gemini API key). Happy to add a recorded live fixture if a maintainer can share one, or run it myself once I have a key.https://claude.ai/code/session_01HCVKnA7LeJZ44nxZz1uzF3