Skip to content

NO-JIRA: scripts: prefer locally built RPMs over product repos in configure-vm.sh - #7258

Closed
mkowalski wants to merge 1 commit into
openshift:mainfrom
mkowalski:fix-local-rpm-repo-priority-main
Closed

mkowalski wants to merge 1 commit into
openshift:mainfrom
mkowalski:fix-local-rpm-repo-priority-main

Conversation

@mkowalski

@mkowalski mkowalski commented Aug 27, 2026

Copy link
Copy Markdown

What

Set priority=1 on the microshift-local repository used by configure-vm.sh so dnf always installs the locally built RPMs instead of any microshift RPMs published in enabled product repositories.

Same change as #7257 (release-4.21), applied to this branch since the script is identical here and equally exposed.

Why

On release-4.21, all conformance CI jobs permafailed because dnf resolved microshift from rhocp-4.21-for-rhel-9-x86_64-rpms (released NEVRA 4.21.29-... sorts higher than the locally built nightly version) instead of the source-built RPMs in the --repofrompath local repo. The released RPM pins an outdated ovn-kubernetes image (missing the OCPBUGS-105440 fix), so CI kept hitting an already-fixed bug. See #7257 for the full investigation.

The same silent version-ordering hazard exists on every branch once a matching product repo publishes a higher-versioned microshift RPM. With repository priority 1 on the local repo, dnf prefers packages from it regardless of version ordering, so CI tests what it built.

How was this tested

bash -n and shellcheck on the modified script (no new findings). Behavioral verification comes from CI on this PR; not verified locally.


This PR description was generated using AI. Please verify before acting on it.
Assisted-By: Claude Fable 5

Summary by CodeRabbit

  • Bug Fixes
    • Local MicroShift installations now prioritize the locally generated repository over published repositories.
    • Repository GPG checking remains disabled for local development environments.

CI installs MicroShift by building RPMs from source and installing them
via a --repofrompath local repository. Once microshift RPMs became
available in the rhocp-4.21 product repository with a release version
(e.g. 4.21.29-...) that sorts higher than the locally built nightly
version, dnf silently resolved 'microshift' from the product repository
instead of the local one.

As a result, CI jobs tested the released RPM (pinning an outdated
ovn-kubernetes image affected by OCPBUGS-105440) instead of the RPMs
built from the checked-out source, causing all release-4.21 conformance
jobs to permafail with 'failed to get ofport of -ex' even after the fix
merged.

Set the highest repository priority on the local repository so dnf
always prefers the locally built RPMs regardless of version ordering.

Assisted-By: Claude Fable 5
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 27, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@mkowalski: This pull request explicitly references no jira issue.

Details

In response to this:

What

Set priority=1 on the microshift-local repository used by configure-vm.sh so dnf always installs the locally built RPMs instead of any microshift RPMs published in enabled product repositories.

Same change as #7257 (release-4.21), applied to this branch since the script is identical here and equally exposed.

Why

On release-4.21, all conformance CI jobs permafailed because dnf resolved microshift from rhocp-4.21-for-rhel-9-x86_64-rpms (released NEVRA 4.21.29-... sorts higher than the locally built nightly version) instead of the source-built RPMs in the --repofrompath local repo. The released RPM pins an outdated ovn-kubernetes image (missing the OCPBUGS-105440 fix), so CI kept hitting an already-fixed bug. See #7257 for the full investigation.

The same silent version-ordering hazard exists on every branch once a matching product repo publishes a higher-versioned microshift RPM. With repository priority 1 on the local repo, dnf prefers packages from it regardless of version ordering, so CI tests what it built.

How was this tested

bash -n and shellcheck on the modified script (no new findings). Behavioral verification comes from CI on this PR; not verified locally.


This PR description was generated using AI. Please verify before acting on it.
Assisted-By: Claude Fable 5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: c514e548-7fcd-4f40-9d2e-71a705f447ea

📥 Commits

Reviewing files that changed from the base of the PR and between 6fec122 and 820d173.

📒 Files selected for processing (1)
  • scripts/devenv-builder/configure-vm.sh

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.


Walkthrough

The VM configuration now assigns priority 1 to the local MicroShift RPM repository while keeping GPG checking disabled.

Changes

Local repository priority

Layer / File(s) Summary
Configure local repository priority
scripts/devenv-builder/configure-vm.sh
The local MicroShift repository configuration now sets microshift-local.priority=1 and keeps GPG checking disabled.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 820d1

The change is limited to configure-vm.sh and makes repository selection prefer locally built RPMs; no actionable correctness or deployment risk remains, so it is merge-ready after normal checks.

Suggested labels: ready-for-human-review

Suggested reviewers: eslutsky, pmtk

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: configure-vm.sh now prefers locally built RPMs over product repositories.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds the DNF repository priority setting and changes no Ginkgo tests or test titles. The changed file contains no…
Test Structure And Quality ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds the DNF repository priority option and comments; it adds no Ginkgo test code, It blocks, cluster operation…
Microshift Test Compatibility ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The patch adds a DNF repository priority setting and adds no Ginkgo e2e tests or other test declarations. The MicroShift tes…
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh (+6/-1). The diff adds a DNF repository priority setting and comments; it adds no Ginkgo tests or test declarations. Theref…
Topology-Aware Scheduling Compatibility ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds --setopt=microshift-local.priority=1 to a DNF install command and updates comments. It does not add or mod…
Ote Binary Stdout Contract ✅ Passed PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The added code is a comment and --setopt=microshift-local.priority=1; the existing gpgcheck=0 option remains. No OTE b…
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The check is not applicable. The PR changes only scripts/devenv-builder/configure-vm.sh and adds a DNF repository priority option plus comments. The verified diff adds no Ginkgo tests, test files, I…
No-Weak-Crypto ✅ Passed PASS. The exact diff changes only comments and adds --setopt=microshift-local.priority=1 to a DNF install command. The changed lines introduce no MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB mode, custo…
Container-Privileges ✅ Passed The pull request changes only scripts/devenv-builder/configure-vm.sh. The added behavior is --setopt=microshift-local.priority=1 in a DNF command. It does not add a privileged container, host name…
No-Sensitive-Data-In-Logs ✅ Passed PASS: The pull request changes only the DNF repository priority and adds explanatory comments. The new argument is the non-sensitive value microshift-local.priority=1. It does not add logging or exp…
Full details: Stable And Deterministic Test Names

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds the DNF repository priority setting and changes no Ginkgo tests or test titles. The changed file contains no It(), Describe(), Context(), or When() calls, so the stable and deterministic test-name check is not applicable.

Full details: Test Structure And Quality

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds the DNF repository priority option and comments; it adds no Ginkgo test code, It blocks, cluster operations, waits, or assertions. The custom Ginkgo test-quality check is therefore not applicable.

Full details: Microshift Test Compatibility

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The patch adds a DNF repository priority setting and adds no Ginkgo e2e tests or other test declarations. The MicroShift test compatibility check is therefore not applicable.

Full details: Single Node Openshift (Sno) Test Compatibility

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh (+6/-1). The diff adds a DNF repository priority setting and comments; it adds no Ginkgo tests or test declarations. Therefore the SNO test compatibility check is not applicable.

Full details: Topology-Aware Scheduling Compatibility

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The diff adds --setopt=microshift-local.priority=1 to a DNF install command and updates comments. It does not add or modify deployment manifests, operator code, controllers, replicas, affinity, topology spread, node selectors, tolerations, or PDBs. The topology-aware scheduling check is therefore not applicable.

Full details: Ote Binary Stdout Contract

Explanation

PASS: The pull request changes only scripts/devenv-builder/configure-vm.sh. The added code is a comment and --setopt=microshift-local.priority=1; the existing gpgcheck=0 option remains. No OTE binary code or process-level stdout write was added or changed.

Full details: Ipv6 And Disconnected Network Test Compatibility

Explanation

The check is not applicable. The PR changes only scripts/devenv-builder/configure-vm.sh and adds a DNF repository priority option plus comments. The verified diff adds no Ginkgo tests, test files, IPv4 assumptions, or new external connectivity requirements.

Full details: No-Weak-Crypto

Explanation

PASS. The exact diff changes only comments and adds --setopt=microshift-local.priority=1 to a DNF install command. The changed lines introduce no MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB mode, custom cryptography, or non-constant-time secret/token comparison. Existing pull-secret handling and gpgcheck=0 are unchanged.

Full details: Container-Privileges

Explanation

The pull request changes only scripts/devenv-builder/configure-vm.sh. The added behavior is --setopt=microshift-local.priority=1 in a DNF command. It does not add a privileged container, host namespace settings, SYS_ADMIN, allowPrivilegeEscalation, or a new root execution path. Existing sudo usage is unchanged and is outside the changed lines.

Full details: No-Sensitive-Data-In-Logs

Explanation

PASS: The pull request changes only the DNF repository priority and adds explanatory comments. The new argument is the non-sensitive value microshift-local.priority=1. It does not add logging or expose passwords, tokens, API keys, PII, session IDs, hostnames, or customer data. The existing set -x in scripts/dnf_retry.sh may log the DNF command, but the changed command contains only repository metadata and a local path.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from jerpeter1 and jogeo August 27, 2026 08:29
@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mkowalski
Once this PR has been reviewed and has the lgtm label, please assign jerpeter1 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai coderabbitai Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Aug 27, 2026
@redhat-chai-bot

Copy link
Copy Markdown
Contributor

This fix is correct and addresses the root cause of the 4.21 nightly permafail (and a latent issue on all branches).

The problem: The dnf install command makes the microshift-local repo available via --repofrompath, but without a priority setting, dnf silently prefers microshift-4.21.29 from rhocp-4.21-for-rhel-9-x86_64-rpms because its NEVRA sorts higher than the locally-built nightly RPMs. The released 4.21.29 RPM pins the pre-fix ovn-kubernetes-microshift image (built before OCPBUGS-105440 was fixed), so CI tests always run with the broken OVN image regardless of what the nightly payload contains.

The fix: --setopt=microshift-local.priority=1 (highest dnf priority) ensures the locally-built RPMs win over any released version. This is minimally invasive — it doesn't disable the RHOCP repos (preserving access to other dependencies) and uses the standard dnf mechanism for repo preference.

Evidence: Confirmed by examining the openshift-microshift-infra-rpm-install-aws step artifact logs on PR #7242, which show every MicroShift package installed from rhocp-4.21-for-rhel-9-x86_64-rpms instead of microshift-local.

Tracked in USHIFT-7489.


AI-generated. Review for accuracy.

@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

@mkowalski: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-tests-bootc-arm-el9 820d173 link true /test e2e-aws-tests-bootc-arm-el9
ci/prow/e2e-aws-tests-bootc-arm-el10 820d173 link true /test e2e-aws-tests-bootc-arm-el10
ci/prow/e2e-aws-tests 820d173 link true /test e2e-aws-tests
ci/prow/e2e-aws-tests-bootc-el9 820d173 link true /test e2e-aws-tests-bootc-el9
ci/prow/e2e-aws-tests-arm 820d173 link true /test e2e-aws-tests-arm
ci/prow/e2e-aws-tests-bootc-el10 820d173 link true /test e2e-aws-tests-bootc-el10

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@pacevedom

Copy link
Copy Markdown
Contributor

Replaced by #7262
/close

@openshift-ci openshift-ci Bot closed this Aug 27, 2026
@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

@pacevedom: Closed this PR.

Details

In response to this:

Replaced by #7262
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants