Skip to content

Compose spec fragments into a whole model via given - #959

Draft
FabianHofmann wants to merge 2 commits into
spec-whole-modelfrom
spec-given
Draft

FabianHofmann wants to merge 2 commits into
spec-whole-modelfrom
spec-given

Conversation

@FabianHofmann

Copy link
Copy Markdown
Collaborator

Closes # (if applicable).

TODO (human): write your intent by hand here.

Note

The following content was generated by AI.

Stacked on #958. This PR adds math-spec's given: key through composition, not binding.

Changes proposed in this Pull Request

  • Compose fragments. Re-export math_spec.merge and math_spec.override as linopy.spec.merge / linopy.spec.override. A fragment declares under given: the variables and constraints it reads but does not build; merge folds those reads into the fragment that introduces them, override lays a patch over a base. Both return a whole-model dict that Model.from_spec builds as usual.
  • Refuse a residual given. build_into (linopy/spec/accessor.py) raises ValueError when a spec still reads a given: name after lowering. The guard sits right after normalize_spec and before any data is attached, so a fragment fails with a clear message instead of a downstream attach or coverage error. This keeps the whole-model-only stance of Build whole models from math-spec (whole-model-only) #958.
  • Tests. New test/test_spec_composition.py covers merge/override folding, building and solving a composed model, refusal of a lone fragment and of a merge with no introducer, and a netcdf round-trip.
  • Docs. Release-notes bullet, doc/api.rst composition note plus spec.merge / spec.override in the API listing.

Warning

math-spec pin is not release-ready. pyproject.toml pins math-spec to the feat/given branch head SHA behind a TODO, because no released alpha ships merge / override / Program.given yet. An alpha tag must be cut from feat/given and the pin updated before merge. #958 must merge first.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.

Re-export math_spec.merge/override from linopy.spec and refuse a spec
that still reads a name under given: as a fragment. Guard sits in
build_into after normalize_spec, before any data is attached.
@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy time — v1 / legacy
peak v1/legacy time v1/legacy
Full table (time + peak, mean)
benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05107 (1.09)   │   15.03 (1.00) 
 (v1)        0.04668 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05923 (1.11)   │   12.04 (1.00) 
 (v1)        0.05331 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02209 (1.08)   │   15.20 (1.0) 
 (v1)        0.02053 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.03461 (1.0)   │   44.93 (1.0) 
 (v1)       0.03738 (1.08)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02542 (1.07)   │   11.51 (1.0) 
 (v1)        0.02365 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05756 (1.07)   │   24.34 (1.06) 
 (v1)        0.05394 (1.0)   │    23.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0667 (1.07)   │   16.12 (1.00) 
 (v1)       0.06218 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.01391 (1.07)   │   752.18 (1.10) 
 (v1)        0.01296 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0136 (1.08)   │   3.12 (1.33) 
 (v1)       0.01254 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05184 (1.38)   │   126.16 (1.44) 
 (v1)        0.03766 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05134 (1.59)   │   126.16 (1.44) 
 (v1)        0.03232 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05161 (1.39)   │   126.16 (1.44) 
 (v1)        0.03722 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03132 (1.01)   │    715.12 (1.0) 
 (v1)        0.03099 (1.0)   │   787.73 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03054 (1.10)   │   4.54 (1.27) 
 (v1)        0.02782 (1.0)   │    3.57 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.2159 (1.04)   │   704.12 (1.09) 
 (v1)        0.2068 (1.0)   │    643.85 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.2275 (1.05)   │   18.34 (1.0) 
 (v1)        0.2171 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.2245 (1.04)   │   9.54 (1.0) 
 (v1)        0.2156 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04325 (1.09)   │   3.77 (1.12) 
 (v1)        0.03985 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04341 (1.08)   │   216.59 (1.10) 
 (v1)        0.04004 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0223 (1.08)   │   938.49 (1.0) 
 (v1)       0.02064 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02383 (1.09)   │   9.66 (1.0) 
 (v1)        0.02195 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)    0.023 (1.08)   │   5.32 (1.0) 
 (v1)       0.02137 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01275 (1.01)   │   1.47 (1.0) 
 (v1)        0.01261 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01264 (1.02)   │   1.47 (1.0) 
 (v1)        0.01242 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01263 (1.00)   │   1.47 (1.0) 
 (v1)         0.0126 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1082 (1.05)   │   946.85 (1.06) 
 (v1)        0.1035 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                time (s)         peak (KiB) 
 name               mean   │           mean 
────────────────────────────────────────────
 (legacy)   0.106 (1.04)   │   12.01 (1.00) 
 (v1)       0.1021 (1.0)   │    11.99 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.02771 (1.0)   │   147.70 (1.06) 
 (v1)       0.02976 (1.07)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.02754 (1.0)   │   2.60 (1.09) 
 (v1)       0.02961 (1.07)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02285 (1.09)   │   696.75 (1.03) 
 (v1)         0.0209 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06813 (1.06)   │   137.97 (1.0) 
 (v1)        0.06432 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03832 (1.05)   │   69.22 (1.0) 
 (v1)        0.03654 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02646 (1.10)   │   402.33 (1.00) 
 (v1)        0.02395 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.02598 (1.0)   │   3.19 (1.19) 
 (v1)       0.02706 (1.04)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.02885 (1.08)   │   29.00 (1.54) 
 (v1)        0.02667 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03788 (1.10)   │   37.95 (1.43) 
 (v1)        0.03441 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.05454 (1.0)   │    410.93 (1.0) 
 (v1)       0.05591 (1.03)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)     0.0604 (1.0)   │     9.94 (1.0) 
 (v1)       0.06292 (1.04)   │   10.22 (1.03) 

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.

@FBumann

FBumann commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

@FabianHofmann Im confused by the pr description

The power of given: as i imagine it is, that users can add a fragment to any linopy model, not just ones buildt with a spec.

Linopy itself just needs to ensure the contract is ok (given variables exist and dims match)?

Or does it not work as imago ed? Or is it just the next step?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants