Skip to content

NO-JIRA: scripts: prefer locally built RPMs over product repos in configure-vm.sh - #7259

Closed
mkowalski wants to merge 1 commit into
openshift:release-5.0from
mkowalski:fix-local-rpm-repo-priority-release-5.0
Closed

mkowalski wants to merge 1 commit into
openshift:release-5.0from
mkowalski:fix-local-rpm-repo-priority-release-5.0

Conversation

@mkowalski

Copy link
Copy Markdown

What

Set priority=1 on the microshift-local repository used by configure-vm.sh so dnf always installs the locally built RPMs instead of any microshift RPMs published in enabled product repositories.

Same change as #7257 (release-4.21), applied to this branch since the script is identical here and equally exposed.

Why

On release-4.21, all conformance CI jobs permafailed because dnf resolved microshift from rhocp-4.21-for-rhel-9-x86_64-rpms (released NEVRA 4.21.29-... sorts higher than the locally built nightly version) instead of the source-built RPMs in the --repofrompath local repo. The released RPM pins an outdated ovn-kubernetes image (missing the OCPBUGS-105440 fix), so CI kept hitting an already-fixed bug. See #7257 for the full investigation.

The same silent version-ordering hazard exists on every branch once a matching product repo publishes a higher-versioned microshift RPM. With repository priority 1 on the local repo, dnf prefers packages from it regardless of version ordering, so CI tests what it built.

How was this tested

bash -n and shellcheck on the modified script (no new findings). Behavioral verification comes from CI on this PR; not verified locally.


This PR description was generated using AI. Please verify before acting on it.
Assisted-By: Claude Fable 5

CI installs MicroShift by building RPMs from source and installing them
via a --repofrompath local repository. Once microshift RPMs became
available in the rhocp-4.21 product repository with a release version
(e.g. 4.21.29-...) that sorts higher than the locally built nightly
version, dnf silently resolved 'microshift' from the product repository
instead of the local one.

As a result, CI jobs tested the released RPM (pinning an outdated
ovn-kubernetes image affected by OCPBUGS-105440) instead of the RPMs
built from the checked-out source, causing all release-4.21 conformance
jobs to permafail with 'failed to get ofport of -ex' even after the fix
merged.

Set the highest repository priority on the local repository so dnf
always prefers the locally built RPMs regardless of version ordering.

Assisted-By: Claude Fable 5
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 27, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@mkowalski: This pull request explicitly references no jira issue.

Details

In response to this:

What

Set priority=1 on the microshift-local repository used by configure-vm.sh so dnf always installs the locally built RPMs instead of any microshift RPMs published in enabled product repositories.

Same change as #7257 (release-4.21), applied to this branch since the script is identical here and equally exposed.

Why

On release-4.21, all conformance CI jobs permafailed because dnf resolved microshift from rhocp-4.21-for-rhel-9-x86_64-rpms (released NEVRA 4.21.29-... sorts higher than the locally built nightly version) instead of the source-built RPMs in the --repofrompath local repo. The released RPM pins an outdated ovn-kubernetes image (missing the OCPBUGS-105440 fix), so CI kept hitting an already-fixed bug. See #7257 for the full investigation.

The same silent version-ordering hazard exists on every branch once a matching product repo publishes a higher-versioned microshift RPM. With repository priority 1 on the local repo, dnf prefers packages from it regardless of version ordering, so CI tests what it built.

How was this tested

bash -n and shellcheck on the modified script (no new findings). Behavioral verification comes from CI on this PR; not verified locally.


This PR description was generated using AI. Please verify before acting on it.
Assisted-By: Claude Fable 5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 307ce90d-577c-4913-af24-53c2c8f722fd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from agullon and copejon August 27, 2026 08:29
@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mkowalski
Once this PR has been reviewed and has the lgtm label, please assign kasturinarra for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@redhat-chai-bot

Copy link
Copy Markdown
Contributor

This fix is correct and addresses the root cause of the 4.21 nightly permafail (and a latent issue on all branches).

The problem: The dnf install command makes the microshift-local repo available via --repofrompath, but without a priority setting, dnf silently prefers released RPMs from rhocp-4.y repos when their NEVRA sorts higher than the locally-built nightly RPMs. This causes CI tests to run with stale released images instead of the locally-built ones.

The fix: --setopt=microshift-local.priority=1 (highest dnf priority) ensures the locally-built RPMs always win. Minimally invasive — uses the standard dnf mechanism for repo preference without disabling other repos.

Tracked in USHIFT-7489.


AI-generated. Review for accuracy.

@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

@mkowalski: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-tests-arm ea91cc6 link true /test e2e-aws-tests-arm
ci/prow/e2e-aws-tests-bootc-el9 ea91cc6 link true /test e2e-aws-tests-bootc-el9
ci/prow/e2e-aws-tests-bootc-arm-el9 ea91cc6 link true /test e2e-aws-tests-bootc-arm-el9
ci/prow/e2e-aws-tests ea91cc6 link true /test e2e-aws-tests
ci/prow/e2e-aws-tests-bootc-arm-el10 ea91cc6 link true /test e2e-aws-tests-bootc-arm-el10
ci/prow/e2e-aws-tests-bootc-el10 ea91cc6 link true /test e2e-aws-tests-bootc-el10

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@pacevedom

Copy link
Copy Markdown
Contributor

Replaced by #7264
/close

@openshift-ci openshift-ci Bot closed this Aug 27, 2026
@openshift-ci

openshift-ci Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

@pacevedom: Closed this PR.

Details

In response to this:

Replaced by #7264
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants