The generated ram-air polar fixture is not reproducible, so every test asserting numbers on it draws a fresh geometry each run

[1-Bort-1]

ram_air_matrix_wing (test/test_data_utils.jl:26) builds its wing by slicing data/ram_air_kite/ram_air_kite_body.obj and sweeping NeuralFoil over it into test/generated/ (gitignored, so CI regenerates it every run). That fixture is not reproducible across machines, and not even across two attempts of the same job.

norm(jac_fwd) from the @info "POLAR_MATRICES linearize jacobian norms" line in test/solver/test_forwarddiff.jl, run 34401975496 on #285 — same commit, same fixture config (n_panels=8, n_sections=4, alpha_range = deg2rad.(-5:5:15), delta_range = deg2rad.(-3:3:3)):

where	norm_fwd
attempt 1, Julia 1.12 ubuntu with code coverage	2.1143274312638014
attempt 3, Julia 1.12 macOS aarch64 / 1.12 windows x64 / 1.11 ubuntu coverage	2.114573062599269
attempt 3, Julia 1.11 ubuntu / 1.12 ubuntu / 1.12 ubuntu coverage	2.165929233230906
agent box, Debian x64	2.2058861369294172

A 4.3% spread, and the same job — Julia 1.12 ubuntu with code coverage — gives 2.1143274312638014 on attempt 1 and 2.165929233230906 on attempt 3.

It is the fixture and not the code. The INVISCID sibling in the same file, which builds its wing in Julia and touches no generated file, reads norm_fwd = 11.110746084832735, norm_fd = 11.110746104003324 on this box and the same 17 digits in that failing coverage job. So VSM's solver, geometry pipeline, ForwardDiff and FiniteDiff paths are bit-reproducible, coverage build included; what moves is what obj_to_yaml writes.

Two things are ruled out:

• Not the %.4f polar rounding (write_polar_matrix_csv, src/airfoil_aero/airfoil_solvers/common.jl:62). Jittering every Cl/Cd/Cm entry by ±1 unit in the last written decimal, 15 draws, moves norm_fwd by ±0.02% — two orders below the observed spread.
• Not run-to-run on one box. Regenerating three times here with BLAS pinned to one thread gives byte-identical geometry.yaml, polars/*.csv and airfoils/*.dat. Only the full-precision pressure/*_cf.csv differ, by a few ulps, and those do not feed cl/cd/cm.

Scale of it in physical terms: scaling every Cl by 5% moves norm_fwd by 2.7% and the panels' converged alphas by 0.09°, so a 4.3% spread is of order 0.1–0.2° of panel alpha — enough to move a panel across the polar table's 5° alpha grid.

That is what makes #287 a lottery: each run draws a different set of panel alphas, and a draw that lands one within the finite-difference stencil of a table knot fails the AD-vs-FD check. #287 fixes that check's own error budget; it does not make the fixture stop moving, and every other test asserting a number on this geometry is drawing from the same hat.

What I would look at first: whether the difference is in geometry.yaml's section coordinates or in the polar values, by having a run print a hash of each generated file. The coordinates go through yaml_scalar(x::Real) = string(round(Float64(x); digits=3)) (src/airfoil_aero/geometry_yaml.jl:22), 1 mm on a 0.168 m tip chord, which is ~0.3° of tip incidence per rounding unit — so a last-digit flip there is not small. Whether a fixed fixture should be committed, or the generation made deterministic, is the decision this needs.

Found while working #287, which changes only test/solver/test_forwarddiff.jl and leaves this alone.

[1-Bort-1]

Queued as VortexStepMethod.jl-291.

You asked to see a plan first, so it will stop and post one before writing code.

[1-Bort-1]

Objective

Make every test that asserts a number on the ram-air geometry read the same bytes on
every machine, by committing the fixture instead of redrawing it each run.

Approach

The generation side is already exhausted. ae11d48 pinned BLAS to one thread around the
conversion, which is why three regenerations here are byte-identical; what is left is
machine-level and cannot be pinned — NeuralFoil is Float32 GEMM through BLAS
(src/airfoil_aero/neuralfoil.jl:194), the Kulfan fit is a LAPACK least-squares solve
(src/airfoil_aero/kulfan.jl:147), and the slicer rides on libm. The pipeline then turns
a ulp into a discrete change: argmin(abs.(arclen .- t)) picks which station a section
lands on (src/obj_adapter/obj_slice.jl:471), and if TE_thickness < 0
(src/airfoil_aero/kulfan.jl:150) re-solves a different system on the sign of a fitted
coefficient. That is the 4.3%. Bit-identical generation across machines would mean
replacing BLAS and LAPACK with fixed-order arithmetic in src/ — a production change made
for a test's benefit, and it would still not cover libm.

So: generate the default config once here with the BLAS pin, commit it as
test/data/ram_air_matrix/ (29 files, 940 KB, measured — all of it read back, since
Wing(yaml) loads the Cp/cf tables at src/yaml_geometry.jl:290), and have
ram_air_matrix_dir return that directory. The three configs the suite currently draws
collapse to that one where the tests allow it, which also takes 25.55 s per config off
every CI job.

Nothing loses coverage: obj_to_yaml is already exercised live into mktempdir() with
structural assertions by three testsets in test/obj_adapter/test_obj_adapter.jl:49,60,63,
and those stay untouched.

The question for you is that CLEAN_CODE.md §9 calls committing generated files never
clean. I read that as aimed at build artefacts rather than test input data — the repo
already tracks data/TUDELFT_V3_KITE/, which is the same yaml-plus-per-section-polar
bundle — but it reads the other way, so it is yours to settle. If you would rather keep
nothing generated in git, the alternative is to move the two numeric tests onto a wing
built in Julia, like the INVISCID sibling that reads the same 17 digits everywhere; that
costs the end-to-end obj→yaml→Wing path its numeric check and a larger rewrite of
test/body_aerodynamics/test_results.jl.

Files I expect to touch

• test/data/ram_air_matrix/** — new, the committed bundle
• test/test_data_utils.jl — ram_air_matrix_dir returns the tracked directory; the
  conversion, the hash-keyed path and the BLAS pin go
• test/body_aerodynamics/test_results.jl, test/wake/test_wake.jl,
  test/test_refinement_validation.jl, test/plotting/test_plotting.jl,
  test/obj_adapter/test_obj_adapter.jl — drop the per-config keywords, keep n_panels
• .gitignore — the now-dead test/generated/ line
• CHANGELOG.md

Not test/solver/test_forwarddiff.jl: #292 owns lines 53-89. If collapsing the configs
turns out to need its call site as well, this branch goes on top with agent stack 292.

---

Remove the agent:needs-plan-approval label to start.
Task VortexStepMethod.jl-291.

[1-Bart-1]

Isn't this already fixed on main?

[1-Bort-1]

Objective

None. The change I proposed last turn is withdrawn, and I am recommending #291 be closed
rather than built.

Approach

Bart asked whether this is already fixed on main. The fixture is not — ram_air_matrix_dir
(test/test_data_utils.jl:42) is unchanged and still redraws into gitignored
test/generated/ every run. But the premise the issue rests on is: that every test
asserting a number on this geometry is near enough its bound for a redraw to matter.

That is testable, and I had not tested it. Multiplying every Cl in the fixture's
polars/*.csv by 1.05 and 0.95 swings norm_fwd from 2.144 to 2.265, bracketing the
whole observed CI spread (2.1143 … 2.2059). Across those three fixtures, on 50dcb12:

Cl factor	POLAR rel_err (bound 1e-4)	results scale=1 (bound 5e-3)	results scale=2 (bound 1e-1)
×0.95	5.271e-8	2.405e-12	4.183e-4
×1.00	4.801e-8	3.947e-12	4.271e-4
×1.05	4.303e-8	8.772e-12	4.354e-4

Three to nine orders of margin everywhere, and a 10% Cl swing moves the tightest of them
by 4%. #292 (0da841d) closed the one check that was actually flaking; the other numeric
consumer was never close. So the residual is that norm_fwd is not comparable between
machines and that each CI job pays ~25 s per config to redraw — neither of which earns
940 KB of committed generated files against CLEAN_CODE.md §9.

Files I expect to touch

None. The deliverable is the answer on the thread; the branch stays empty and no PR opens.

---

Remove the agent:needs-plan-approval label to start.
Task VortexStepMethod.jl-291.