Skip to content

fix(ci): open a pull request for a clean backport instead of pushing it - #8378

Closed
mengw15 wants to merge 1 commit into
apache:mainfrom
mengw15:fix/8377-backport-clean-path-opens-pr
Closed

mengw15 wants to merge 1 commit into
apache:mainfrom
mengw15:fix/8377-backport-clean-path-opens-pr

Conversation

@mengw15

@mengw15 mengw15 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Direct Backport Push cherry-picked a cleanly-applying fix onto the release branch and pushed it. Since release/* came under the Merge Queue ruleset, every one of those pushes is rejected — GH013: must go through a pull request, through the merge queue, three required status checks expected — and the job retries five times and fails. The last one that landed was 2026-07-24. Five fixes are on main and not on release/v1.2 because of it (#8377).

The cleaner the backport, the more certainly it was lost: the conflict path opens a pull request and kept working, so almost everything that reached a release branch in the last six weeks arrived that way.

This routes both outcomes through a pull request. That is also what ASF policy asks for — an automated service must not push to a branch subject to official release without prior authorization from Infrastructure — so a backport now travels the way every other change to a release branch travels.

The two are not the same pull request:

conflicted (unchanged) clean (new)
state draft, for its author to finish ready for review
auto-merge off armed (squash)
what is left resolve conflicts, mark ready the release manager's approval

A clean backport needs nobody's hands: it cherry-picked without conflicts, the backported tree built green before the original merged, and that manager already approved the original for this branch. The body and status comment say exactly that, so the approval is a confirmation rather than a second review.

Why a clean one is closed and reopened. Both open as github-actions[bot], and GitHub suppresses workflow runs for anything GITHUB_TOKEN does — so neither starts with any checks. A draft can afford that, since CI fires when its author pushes a resolution. A clean one cannot: nobody is going to push anything, its three required contexts would never appear, and auto-merge would wait on them forever, looking like progress while standing still. Closing and reopening it under AUTO_MERGE_TOKEN emits pull_request: reopened, which Required Checks, Check License Headers and Validate PR title all subscribe to, and leaves the bot as the pull request's author.

Reopening is the half that must not be lost, since a backport left closed is a fix silently dropped. It retries, and if it still fails it says so on the original PR and fails the job rather than leaving a closed pull request nobody is watching. If auto-merge cannot be armed afterwards, the backport PR gets a comment correcting the claim its own body already made, so a reviewer does not approve and then wait for a merge that never comes.

push_entries is now always empty, which leaves the push-backports job unreachable. Removing it is deliberately left to a separate change, so that this one is a behaviour change and that one is a pure deletion.

Any related issues, documentation, discussions?

Closes #8377.

How was this PR tested?

The routing was driven locally: with both targets clean, one clean and one conflicted, and both conflicted, push_entries comes out empty in every case while the clean targets carry clean: "true" and the conflicted ones clean: "false". Mislabelling a clean target turns that run red, so the check is not vacuous. The workflow parses as YAML, every embedded github-script body passes node --check, and release_branches.py still parses the annotated config.

Not verifiable off GitHub, and worth watching on the first clean backport after this lands: that the close/reopen actually starts the three required checks, and that auto-merge is accepted on the reopened pull request. Both failure modes are handled rather than assumed — a failed reopen reports on the original PR and fails the job, a failed arming comments on the backport PR — but neither path has executed in production.

The evidence for the diagnosis itself is in #8377: run 33706287941 shows the GH013 rejection, and 9a989b4bd on release/v1.2 is the last push that succeeded.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-5)

@github-actions github-actions Bot added fix ci changes related to CI labels Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Yicong-Huang
    You can notify them by mentioning @Yicong-Huang in a comment.

@mengw15
mengw15 requested a balanced review from Copilot September 3, 2026 07:57
@codecov-commenter

codecov-commenter commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.05%. Comparing base (16da22b) to head (e47d213).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8378      +/-   ##
============================================
- Coverage     94.05%   94.05%   -0.01%     
  Complexity     4772     4772              
============================================
  Files          1196     1196              
  Lines         48753    48753              
  Branches       5916     5916              
============================================
- Hits          45856    45853       -3     
- Misses         1449     1451       +2     
- Partials       1448     1449       +1     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø) Carriedforward from 16da22b
agent-service 99.32% <ø> (ø) Carriedforward from 16da22b
amber 89.89% <ø> (ø) Carriedforward from 16da22b
computing-unit-managing-service 73.67% <ø> (ø) Carriedforward from 16da22b
config-service 87.12% <ø> (ø) Carriedforward from 16da22b
file-service 87.91% <ø> (ø) Carriedforward from 16da22b
frontend 96.85% <ø> (ø) Carriedforward from 16da22b
notebook-migration-service 79.31% <ø> (ø)
pyamber 98.41% <ø> (-0.07%) ⬇️
workflow-compiling-service 77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Clean backports can be misclassified, reported as failures, and processed out of commit order.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Routes clean and conflicted release backports through pull requests instead of protected-branch pushes.

Changes:

  • Opens clean backports ready for review with auto-merge.
  • Retains draft PRs for conflicted backports.
  • Updates backport documentation and status messaging.
File summaries
File Description
.github/workflows/direct-backport-push.yml Implements PR-based backports and auto-merge.
.github/workflows/backport-publish.yml Updates clean-backport check messaging.
.github/release-branches.yml Documents the revised workflow.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/direct-backport-push.yml Outdated
Comment thread .github/workflows/direct-backport-push.yml
Comment thread .github/workflows/direct-backport-push.yml
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 16da22b benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 414 0.253 23,873/30,133/30,133 us 🔴 -13.4% / 🔴 +88.4%
🔴 bs=100 sw=10 sl=64 913 0.557 106,896/156,971/156,971 us 🔴 +18.5% / 🔴 +41.8%
🔴 bs=1000 sw=10 sl=64 1,094 0.668 914,785/1,031,165/1,031,165 us 🔴 +6.0% / 🟢 +10.3%
Baseline details

Latest main 16da22b from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 414 tuples/sec 478 tuples/sec 750.25 tuples/sec -13.4% -44.8%
bs=10 sw=10 sl=64 MB/s 0.253 MB/s 0.292 MB/s 0.458 MB/s -13.4% -44.7%
bs=10 sw=10 sl=64 p50 23,873 us 21,575 us 13,041 us +10.7% +83.1%
bs=10 sw=10 sl=64 p95 30,133 us 29,286 us 15,996 us +2.9% +88.4%
bs=10 sw=10 sl=64 p99 30,133 us 29,286 us 19,395 us +2.9% +55.4%
bs=100 sw=10 sl=64 throughput 913 tuples/sec 942 tuples/sec 961.14 tuples/sec -3.1% -5.0%
bs=100 sw=10 sl=64 MB/s 0.557 MB/s 0.575 MB/s 0.587 MB/s -3.1% -5.1%
bs=100 sw=10 sl=64 p50 106,896 us 104,401 us 103,974 us +2.4% +2.8%
bs=100 sw=10 sl=64 p95 156,971 us 132,458 us 110,662 us +18.5% +41.8%
bs=100 sw=10 sl=64 p99 156,971 us 132,458 us 120,690 us +18.5% +30.1%
bs=1000 sw=10 sl=64 throughput 1,094 tuples/sec 1,113 tuples/sec 991.91 tuples/sec -1.7% +10.3%
bs=1000 sw=10 sl=64 MB/s 0.668 MB/s 0.679 MB/s 0.605 MB/s -1.6% +10.3%
bs=1000 sw=10 sl=64 p50 914,785 us 897,273 us 1,013,755 us +2.0% -9.8%
bs=1000 sw=10 sl=64 p95 1,031,165 us 972,782 us 1,056,115 us +6.0% -2.4%
bs=1000 sw=10 sl=64 p99 1,031,165 us 972,782 us 1,082,840 us +6.0% -4.8%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,483.43,200,128000,414,0.253,23872.56,30133.48,30133.48
1,100,10,64,20,2191.40,2000,1280000,913,0.557,106895.63,156971.37,156971.37
2,1000,10,64,20,18273.47,20000,12800000,1094,0.668,914784.88,1031164.86,1031164.86

@mengw15
mengw15 force-pushed the fix/8377-backport-clean-path-opens-pr branch from d708ff0 to ba751cb Compare September 3, 2026 08:06
@mengw15
mengw15 requested a balanced review from Copilot September 3, 2026 08:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Reruns can preserve an incorrect draft state, and token/status handling can falsely report stalled or closed backports as successful.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread .github/workflows/direct-backport-push.yml
Comment thread .github/workflows/direct-backport-push.yml Outdated
Comment thread .github/workflows/direct-backport-push.yml Outdated
Direct Backport Push cherry-picked a cleanly-applying fix onto the
release branch and pushed it. Since `release/*` came under the Merge
Queue ruleset every one of those pushes has been rejected with GH013 —
must go through a pull request, through the merge queue, three required
checks expected — and the job has retried five times and failed. The last
one that landed was on 2026-07-24. Five fixes sit on main and not on
release/v1.2 because of it, tracked in apache#8377.

The cleaner the backport, the more certainly it was lost: the conflict
path opens a pull request and kept working, so almost everything that
reached a release branch in the last six weeks arrived that way.

Route both outcomes through a pull request. That is also what ASF policy
asks for — an automated service must not push to a branch subject to
official release without prior authorization from Infrastructure — so a
backport now travels the way every other change to a release branch
travels.

The two are not the same pull request. A conflicted one still opens as a
draft for its author to finish. A clean one needs nobody's hands, so it
opens ready for review with auto-merge armed and the release manager's
approval as the only step left: it cherry-picked without conflicts, the
backported tree built green before the original merged, and that manager
already approved the original for this branch. The body and the status
comment say exactly that, so the approval is a confirmation rather than a
second review.

Both are opened as github-actions[bot]. GitHub suppresses workflow runs
for anything GITHUB_TOKEN does, so neither starts with any checks — which
a draft can afford, since CI fires when its author pushes a resolution,
and a clean one cannot: nobody is going to push anything, its three
required contexts would never appear, and auto-merge would wait on them
forever. So a clean one is closed and reopened under AUTO_MERGE_TOKEN,
which emits `pull_request: reopened` — subscribed to by all three
required workflows — while leaving the bot as the pull request's author.

Reopening is the half that must not be lost, since a backport left closed
is a fix silently dropped. It retries, and if it still fails it says so on
the original PR and fails the job rather than leaving a closed pull
request nobody is watching. If auto-merge cannot be armed afterwards, the
backport PR gets a comment correcting the claim its own body already
made, so a reviewer does not approve and wait for a merge that never
comes.

`push_entries` is now always empty, leaving the push-backports job
unreachable. Removing it is a separate change, so that this one is a
behaviour change and that one is a pure deletion.
@mengw15
mengw15 force-pushed the fix/8377-backport-clean-path-opens-pr branch from ba751cb to e47d213 Compare September 3, 2026 08:21
@mengw15 mengw15 closed this Sep 3, 2026
renovate-bot pushed a commit to renovate-bot/apache-_-texera that referenced this pull request Sep 11, 2026
…8379)

### What changes were proposed in this PR?

The Merge Queue ruleset requires every change into `release/*` to arrive
as a PR with one approving review, green required checks, and a pass
through the merge queue. Right for people — but it also blocks
`direct-backport-push.yml`, whose fast path pushes clean cherry-picks;
every such push has been rejected since 2026-07-24, and five backports
were silently lost (apache#8377).

This splits the ruleset in two, rule-for-rule identical: `Merge Queue`
keeps `~DEFAULT_BRANCH`, and a new `Merge Queue (release)` carries the
three release branches plus a `bypass_actors` entry for the GitHub
Actions app (`actor_id: 15368`). The split exists because a bypass is
ruleset-wide — kept in one ruleset, it would let workflows push `main`
too.

Scope, stated precisely: the bypass exempts actions performed as the
Actions app — any workflow's `GITHUB_TOKEN`, not just the backport
workflow, since rulesets cannot scope a bypass to one workflow. People
and PATs still face every rule on every branch; `main` gets no bypass;
force pushes and branch deletion stay blocked for everyone, Actions
included, by `Default Branch Protection`.

Ordering inside the file is load-bearing: asfyaml applies rulesets in
file order, so `Merge Queue (release)` is created before `Merge Queue`
stops covering the release branches. If GitHub rejects the new ruleset,
the apply aborts with today's protections fully intact — no failure path
leaves the release branches uncovered.

The bypass alone would not revive the fast path: since apache#4676 the push
job checked out with `AUTO_MERGE_TOKEN`, so GitHub evaluated its pushes
as that PAT's owner — every pre-ruleset direct push shows a person as
the pusher — and an Actions-app bypass would not cover them. The push
job now uses the default `GITHUB_TOKEN`, which the bypass does cover,
and dispatches `Required Checks` on the pushed branch explicitly, since
a `GITHUB_TOKEN` push starts no push-triggered runs while
`workflow_dispatch` is the documented exception that always creates one.
The conflict path keeps the PAT: it pushes unprotected `backport/*`
branches, where the opened PR's CI must still trigger.

### Any related issues, documentation, discussions?

Closes apache#8377. apache#8378 took the PR-plus-auto-merge route to the same
problem and is closed in favor of trying the bypass first. What lands on
a release branch through this path is still only a cherry-pick of a
commit that passed main's full CI and, once apache#8096 lands, its release
manager's approving review.

### How was this PR tested?

`.asf.yaml` and the workflows parse, and the structural check is now
committed instead of run once: `.github/scripts/test_asf_rulesets.sh`
(picked up by build.yml's glob-discovered infra tests) asserts the two
rulesets' `rules` blocks stay deep-equal and that `.asf.yaml` and every
workflow parse under a duplicate-key-strict loader, with PyYAML pinned
in `amber/dev-requirements.txt` — the file the infra job installs; every
failure path (duplicate key, rules drift, bypass on main, bypass
tampered, ruleset reorder, missing PyYAML) was verified red before
trusting the green. asfyaml treats a ruleset carrying
`target`/`rules`/`bypass_actors` as a raw payload and forwards it
verbatim (`_RAW_RULESET_KEYS` in `feature/github/rulesets.py`; its
upstream tests assert the POST payload carries `bypass_actors`).

What cannot be proven before merge is GitHub accepting the Actions app
as a bypass actor on this org: the same payload on a personal repository
is rejected with "Actor GitHub Actions integration must be part of the
ruleset source or owner organization", and no ASF repository uses an
Integration bypass actor yet — hence the fail-safe ordering above. After
Infra applies the merged file, `GET /repos/apache/texera/rulesets`
should list `Merge Queue (release)`; if it does not, the apply failed
closed and nothing changed. The next clean backport is the end-to-end
test.

### Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-fable-5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci changes related to CI fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backport fast path has been dead since 2026-07-24: direct pushes to release/* are rejected by the ruleset

3 participants