Add agentic issue investigation workflow (ported from azure-sdk-for-net) - #48830
ROHIT SINGHAL (rohitsinghal4u) wants to merge 5 commits into
Conversation
Ports the Azure SDK for .NET Agentic Issue Investigation workflow to azure-sdk-for-python, adapted for Python conventions (PyPI package metadata, azure-<service>-<package> naming, sdk/<service>/TROUBLESHOOTING.md lookups). - Adds .github/workflows/issue-investigation.md + compiled lock.yml - Updates issue-triage.md to add Step 7: Dispatch Agentic Investigation, auto-chaining into the new investigation workflow after a clean triage handoff, plus the dispatch-workflow safe-output declaration Same decision gates as .NET: handoff validation, version currency, duplicate detection, insufficient context, working-as-designed/service-side, actionable SDK issue (with exclusions), and an overarching confidence gate. For testing/validation in this fork only.
|
Azure Pipelines: Successfully started running 1 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds an automated, evidence-gated investigation stage after issue triage.
Changes:
- Adds issue investigation and decision rules.
- Dispatches investigations after clean triage.
- Adds compiled workflow definitions.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
.github/workflows/issue-investigation.md |
Defines investigation behavior and outputs. |
.github/workflows/issue-investigation.lock.yml |
Compiles the investigation workflow. |
.github/workflows/issue-triage.md |
Dispatches eligible issues for investigation. |
.github/workflows/issue-triage.lock.yml |
Compiles updated triage orchestration. |
Suppressed comments (1)
.github/workflows/issue-investigation.md:61
- The assignment sink is also repository-wide, so a compromised or mistaken investigation can assign Copilot to an issue other than the dispatched one. Restrict the safe output to the supplied issue number.
target: "*"
| add-comment: | ||
| max: 1 | ||
| target: "*" | ||
| close-issue: | ||
| max: 1 | ||
| target: "*" | ||
| state-reason: not_planned |
| @@ -0,0 +1,1615 @@ | |||
| # gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"6e6951229826a679722ac1d0aff27abbb93023c69ee48f305f84fbd7db84b7d7","body_hash":"afbdd168ecf6ce4443c835d0e20065bf40c3e6f80b199fd551b1424c85f2ea7d","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}} | |||
|
|
||
| ## Support Policy Expectation | ||
|
|
||
| By policy, Azure SDK support is only available for the latest package version. Version currency is a mandatory decision point, not just background guidance. |
| Do not assign Copilot, even if the above are met, when the issue requires any of the following. Use `noop` or a targeted Insufficient Context request instead: | ||
| - Public API design or compatibility decisions (new members, signature changes, breaking changes). | ||
| - Security- or privacy-sensitive changes. | ||
| - Changes with data-loss or reliability risk. | ||
| - Service-contract or protocol-level changes. | ||
| - Broad refactoring spanning multiple files or components. | ||
| - Unclear code or documentation ownership. | ||
| - Investigation that depends on live-service behavior that cannot be verified from repository context alone. | ||
|
|
||
| If any exclusion applies, or the fix area cannot be stated specifically, do not assign Copilot -- call `noop` or request the missing information instead. |
| # gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"119d6ba5ab3dfaf997287dd0f857fd8dd20ffe78e7b649777a0adc83c0d94f75","body_hash":"386be54d7373e4c92f43d88ce7ffa6526da1a0d2c390c2c1877ac5614f435946","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.63"}} | ||
| # gh-aw-manifest: {"version":1,"secrets":["GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/cache/save","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} | ||
| # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md | ||
| # gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"05b306844fe731bbc51742814306b173e76912201a7e90c5783c248d61a8abd5","body_hash":"dc8a877d119de2565e69737741e0463fd731dbe25d0159ee257cf166e3793683","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}} |
| In `sdk/storage/azure-storage-blob/azure/storage/blob/_list_blobs_helper.py`, `BlobPropertiesPaged._get_next_cb` returns a response whose `continuation_token` is `None` on the final page, but `PageIterator.__next__` does not check for `None` before calling `_get_next` again, so the underlying transport raises `StopIteration` directly instead of the iterator's own clean termination path. The bug is present in current code on the latest stable release. | ||
|
|
||
| ### 🛠️ Suggested Fix | ||
|
|
||
| Add an explicit `continuation_token is None` check in `PageIterator.__next__` before issuing the next page request, raising the standard `StopIteration` from the iterator itself rather than letting it propagate from the transport call. A regression test that exhausts a multi-page `list_blobs` result set verifies the fix. |
The prior actionable-path example claimed a StopIteration/continuation_token bug in azure-core PageIterator/_list_blobs_helper.py that does not actually exist -- current source (paging.py:79-80) already guards against it. Presenting a false root cause as a worked example risked priming the investigator to reproduce it on a superficially similar real report. Replaced with a verified-real example: an etag operand-order bug in azure-appconfiguration's set_configuration_setting (confirmed present in current source by direct inspection), which silently ignores an explicit etag kwarg whenever the ConfigurationSetting object already has one set.
|
Thanks for the thorough review — addressing each finding: 1. Fabricated worked example (issue-investigation.md:310) — FIXED. Verified: 2 & 5. Stale gh-aw compiler version (v0.81.6) on both lock files. Confirmed valid — this repo's other workflows are on v0.87.1+, and the older compiler's Copilot CLI invocation pattern doesn't copy the binary to a mounted path first. I attempted to upgrade my local 3. Version Currency wording ("only available for the latest package version"). This is intentional and inherited verbatim from the already-merged 4. Exclusion routing ambiguity (noop vs. "Requires a human"). Confirmed this same tension exists verbatim in the current 6. Wildcard |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
.github/workflows/issue-investigation.md:129
- This is stricter than the Azure SDK support policy: an Active major version is fully supported, while customers are only encouraged to use the latest compatible release. The Version Currency rule consequently short-circuits valid reports from any older minor/patch and tells supported customers that support only applies to latest. Reframe currency as diagnostic guidance and reserve support-based deferral for lifecycle/compatibility cases rather than every non-latest version.
## Support Policy Expectation
By policy, Azure SDK support is only available for the latest package version. Version currency is a mandatory decision point, not just background guidance.
.github/workflows/issue-investigation.md:224
- For confirmed SDK-side issues that hit an exclusion, this rule directs the agent to
noop, but the Comment Format explicitly definesRequires a human. Analysis provided belowfor those same cases. As written, complex but well-evidenced SDK issues can silently disappear instead of giving maintainers the prepared analysis. Emit the human-review outcome for confirmed SDK-side exclusions, while retaining targeted requests/noop only when ownership or required evidence is unresolved.
Do not assign Copilot, even if the above are met, when the issue requires any of the following. Use `noop` or a targeted Insufficient Context request instead:
.github/workflows/issue-triage.lock.yml:1
- This regenerated lockfile is also pinned to gh-aw v0.81.6 and retains the older direct
/usr/local/bin/copilotinvocation. Recompileissue-triage.mdwith the current gh-aw release so it uses the repository's mounted Copilot CLI pattern and commit the resulting lockfile; generated YAML should not be patched manually.
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"05b306844fe731bbc51742814306b173e76912201a7e90c5783c248d61a8abd5","body_hash":"dc8a877d119de2565e69737741e0463fd731dbe25d0159ee257cf166e3793683","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}}
|
|
||
| ## Required Handoff Validation | ||
|
|
||
| Retrieve the issue with `get_issue`. Inspect labels and label colors. |
There was a problem hiding this comment.
🟡 Changes recommended
The workflows contain unsafe wildcard targets, an incorrect support-policy gate, contradictory outcome instructions, and outdated generated locks.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (5)
.github/workflows/issue-investigation.md:224
- This makes the documented
Requires a humanoutcome unreachable: exclusions are told to emitnoop, while lines 260 and 279 say those same exclusions should produce the human-analysis comment. Confirmed SDK-side issues requiring API, security, reliability, or broad design work will therefore receive no investigation result. Route confirmed exclusions to the human outcome and reservenoopfor cases lacking enough evidence.
Do not assign Copilot, even if the above are met, when the issue requires any of the following. Use `noop` or a targeted Insufficient Context request instead:
.github/workflows/issue-investigation.md:42
- The workflow already has one trusted target in
github.event.inputs.issue_number, buttarget: "*"allows model output influenced by untrusted issue text to close any issue in the repository. Restrict this destructive safe output to that input (and preferably requirecustomer-reported) so the safe-output boundary contains a misdirected action.
target: "*"
.github/workflows/issue-triage.lock.yml:1
- This regenerated lock also remains on gh-aw v0.81.6 and invokes
/usr/local/bin/copilotrather than the mounted${RUNNER_TEMP}/gh-aw/bin/copilotpattern emitted by the repository's v0.87.1 workflow. Recompileissue-triagewith the repository-standard current gh-aw before merging so the newly chained triage run does not fail at agent startup.
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"05b306844fe731bbc51742814306b173e76912201a7e90c5783c248d61a8abd5","body_hash":"dc8a877d119de2565e69737741e0463fd731dbe25d0159ee257cf166e3793683","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}}
.github/workflows/issue-investigation.md:129
- The Azure SDK support policy does not limit support to the latest point release: Active major versions are fully supported, while the latest compatible release is only encouraged because it receives fixes. Making this a mandatory early exit will incorrectly reject valid reports against supported older minor/patch releases. Base the gate on lifecycle status and known fixes instead of exact version currency.
By policy, Azure SDK support is only available for the latest package version. Version currency is a mandatory decision point, not just background guidance.
.github/workflows/issue-investigation.md:39
- This wildcard lets untrusted issue content redirect the investigation comment to an unrelated issue. Scope
add-commentto${{ github.event.inputs.issue_number }}so output can only affect the issue selected by the dispatcher.
target: "*"
- Files reviewed: 3/4 changed files
- Comments generated: 1
- Review effort level: Balanced
| name: copilot | ||
| allowed: [copilot] | ||
| max: 1 | ||
| target: "*" |
- Fix stale tool reference: prompt said get_issue, the pinned GitHub MCP server (v1.4.0) exposes issue_read with no alias for the old name. Earlier runs worked because the model self-corrected, but that's fragile. - Scope add-comment/close-issue/assign-to-agent target to the dispatched issue_number instead of wildcard *, so a misdirected or injected output can no longer affect an unrelated issue. Verified in compiled output. - Fix a real contradiction: the Actionable SDK Issue exclusions said to noop, while the Comment Format section said the same condition should produce the 'Requires a human' outcome, making that outcome unreachable. Now routes well-evidenced exclusions to the human outcome and reserves noop for cases lacking evidence.
…rohitsinghal4u/azure-sdk-for-python into add-agentic-issue-investigation
|
Update after a second automated review pass surfaced 3 more real findings — fixed all 3:
Still open, honestly:
|
There was a problem hiding this comment.
🟡 Changes recommended
The support-policy gate is inaccurate, closed issues are not excluded, and both lockfiles require recompilation with current gh-aw.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
.github/workflows/issue-investigation.md:96
- The handoff gate never checks that the issue is still open. Because the investigation is queued after triage, an author or maintainer can close the issue before this run starts; that closed issue still passes these label checks and can receive a new automated comment or Copilot-assignment attempt. Require the retrieved issue state to be open so stale/manual dispatches no-op.
.github/workflows/issue-investigation.md:129
- This version gate is based on an incorrect support-policy premise. The Azure SDK lifecycle policy says an Active major version is fully supported and only encourages customers to use the latest compatible minor/patch release; it does not make every older point release unsupported. As written, the workflow will stop investigation and send supported customers away whenever their point release is not the newest. Please rewrite this section and the Version Currency rule to treat lifecycle status as the support boundary, while recommending an upgrade when the issue may already be fixed.
By policy, Azure SDK support is only available for the latest package version. Version currency is a mandatory decision point, not just background guidance.
.github/workflows/issue-triage.lock.yml:1
- This lockfile was regenerated with gh-aw v0.81.6, so it retains the old hard-coded
/usr/local/bin/copilotsandbox invocation (line 938). The repository's v0.87.1-generated workflow instead resolves Copilot fromPATH, copies it into${RUNNER_TEMP}/gh-aw/bin, mounts that directory, and invokes the copied binary (mgmt-sdk-pr-review.lock.yml:772-809); without that pattern, the installed CLI may not be available in the sandbox. Recompile this workflow with the repository's current gh-aw version before merging.
# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"05b306844fe731bbc51742814306b173e76912201a7e90c5783c248d61a8abd5","body_hash":"dc8a877d119de2565e69737741e0463fd731dbe25d0159ee257cf166e3793683","compiler_version":"v0.81.6","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65"}}
- Files reviewed: 3/4 changed files
- Comments generated: 1
- Review effort level: Balanced
| fi | ||
| # shellcheck disable=SC1003,SC2086 | ||
| sudo -E awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_TOOL_CACHE_MOUNT:+--mount "$GH_AW_TOOL_CACHE_MOUNT"} ${GH_AW_DOCKER_HOST:+--docker-host "$GH_AW_DOCKER_HOST"} ${GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS} --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --skip-pull \ | ||
| -- /bin/bash -c 'set +o histexpand; export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && : "${RUNNER_TOOL_CACHE:?RUNNER_TOOL_CACHE must be set}"; GH_AW_TOOL_CACHE="$RUNNER_TOOL_CACHE"; export PATH="$(find "$GH_AW_TOOL_CACHE" -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; GH_AW_NPM_GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"; if [ -n "$GH_AW_NPM_GLOBAL_ROOT" ]; then export NODE_PATH="${GH_AW_NPM_GLOBAL_ROOT}${NODE_PATH:+:${NODE_PATH}}"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-tool github --allow-tool safeoutputs --allow-tool web_fetch --allow-tool write --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log |
Summary
Ports the Agentic Issue Investigation workflow from
azure-sdk-for-net(see Azure/azure-sdk-for-net#60514) toazure-sdk-for-python, adapted for this repository's conventions.After
issue-triagelabels and routes a new customer-reported issue, this workflow runs a deeper, evidence-gated investigation: it checks version currency against PyPI, rules out duplicates, flags missing repro context, distinguishes SDK-side bugs from service-side/working-as-designed behavior, and — only when a fix is bounded and well-evidenced — recommends Copilot with a concrete root cause and suggested fix. It is designed to abstain (noop) rather than guess whenever evidence is incomplete.Changes
.github/workflows/issue-investigation.md(+ compiled.lock.yml).github/workflows/issue-triage.md(+ compiled.lock.yml) — addsdispatch-workflowsafe-output and Step 7: Dispatch Agentic Investigation, auto-chaining into the new workflow once an issue is cleanly triaged (same conditions used on.NET: exactly one service label#e99695, one category label#ffeb77,customer-reportedpresent, none ofneeds-triage/needs-team-triage/issue-addressed/needs-author-feedback)Adaptations from .NET
.NET-style package naming →azure-<service>-<package>naming (e.g.azure-keyvault-secrets)sdk/<service>/TROUBLESHOOTING.mdandsdk/<service>/<package>/TROUBLESHOOTING.mdfollow the same convention already in this repo, so those lookups carry over unchangedknown-behaviors.mdreferences dropped — not an established convention in this repo (unlike.NET)Validation performed
gh aw compile: 0 errors, 0 warningsgh aw lint(actionlint): 0 issuesRecommended for Copilot automated fix)More information needed from the author)Reproduce on the latest version)Known limitation (same as .NET)
Direct Copilot assignment (
assign_to_agent) is best-effort and will typically skip on this repository: GitHub's Copilot coding-agent assignment API only accepts a user-to-server identity (PAT / OAuth app token / GitHub App user-to-server token), and the token available to this job is server-to-server. This is documented in the frontmatter comment on theassign-to-agentblock. Until a user-to-server credential is wired intoGH_AW_AGENT_TOKEN, the workflow recommends Copilot in its comment and a maintainer completes the assignment.Note on compiled lock files
Compiled locally with
gh-awv0.81.6. I intentionally left this repo's other four agentic workflows untouched even though recompiling triggered drift on them locally (different pinned runtime version) — onlyissue-investigationandissue-triageare included here. Happy to re-compile with whatevergh-awversion this repo currently standardizes on if that differs.GitHub's automated PR review caught real issues, most now fixed (see PR comments for the full list). One remains and needs maintainer action:
gh-awv0.81.6, but this repo's other agentic workflows are on v0.87.1+, which uses a safer Copilot CLI invocation pattern (copies the binary to a mounted path before invoking it). I couldn't upgrade my local compiler to verify/fix this myself (the v0.87.10 download was blocked by Windows Defender as a false positive in my environment). Please recompileissue-investigation.lock.ymlandissue-triage.lock.ymlwith a currentgh-awbefore merging.