Report SBD's trimmed determinants as SCIResult.carryover - #32
Draft
Jim Garrison (garrison) wants to merge 2 commits into
Draft
Jim Garrison (garrison) wants to merge 2 commits into
Jim Garrison (garrison) wants to merge 2 commits into
Conversation
SBD selects a subset of its subspace whenever carryover_type is nonzero, and returns it in carryover_adet and carryover_bdet. This wrapper discarded it and returned the eigenvector instead, which it reads back from a file SBD dumps. The PR at Qiskit/qiskit-addon-sqd#369 introduces the ability for a solver to report those determinants directly, as SCIResult.carryover, leaving SCIResult.sci_state as None. That removes the disk round-trip: the eigenvector is len(strings_a) * len(strings_b) doubles, which is the one quantity a large distributed calculation cannot afford to move between processes. The trim SQD schedule that PR adds wants exactly the weight-ranked selection SBD already computed, so the eigenvector is not merely expensive there, it is unnecessary. The new report_carryover argument selects between the two. It defaults to None, meaning report the carryover when SBD selects one and return an eigenvector when it does not. Since #28 defaults carryover_type to 0, that makes this opt-in: a caller sets carryover_type to 1, 2 or 3 through sbd_config. Pass False to keep the eigenvector even when SBD trims, for a caller that wants the amplitudes for its own analysis. Passing True with carryover_type=0 raises rather than silently returning an empty carryover. The comment on that default is updated: it said SBD's carryover is "NOT consumed on this path", which was true when written and is now the description of a default rather than of the wrapper. The reasoning behind the default is unchanged -- most callers ignore SBD's selection and should not pay to compute it. Two ordering facts, since #29 established that these should be stated rather than assumed: - The carryover comes back in descending weight order. CarryOverAdet and CarryOverBdet (upstream chemistry/tpb/rdmat.h) sort by the diagonal reduced density matrix and keep the top entries; neither calls sort_bitarray, so this is neither canonical order nor deduplicated. That is the order a trim wants, and the addon applies its own shape constraints to whatever a policy returns, so nothing here depends on either property. - The eigenvector path is untouched, including #29's labelling of amplitudes with the canonically ordered, deduplicated lists SBD was given. The two paths therefore return differently ordered strings, which the docstrings now say. Ranks other than the control process return a well-formed empty carryover, since the configuration recovery loop reads results on rank 0 only. Not mergeable until SCIResult.carryover is released: qiskit-addon-sqd#369 is still open, so the qiskit-addon-sqd pin in pyproject.toml cannot yet be raised to a version that has it. The pin needs bumping once that lands and ships. Assisted-by: Claude Opus 5 (1M context)
Nothing exercised report_carryover, and nothing exercised the amplitude handling either, so the suite could not distinguish the two paths solve_sci chooses between. That matters more than it looks: the choice is invisible in an energy, which is identical either way, so a wrong path shows up only as a wrong subspace one iteration later. Ten tests over the vendored h2o data, using 40 alpha determinants so a ratio leaves a proper subset to inspect: - which output is returned, over the (carryover_type, report_carryover) matrix, asserting that exactly one of sci_state and carryover is populated - report_carryover=True with carryover_type=0 raises rather than returning an empty carryover - the carryover holds int(ratio * len(strings)) determinants per spin sector - with the eigenvector requested, it spans the full input subspace even when carryover_type is nonzero -- the property #19 fixed - the carryover is the highest-weight determinants in descending weight order - the energy does not depend on which output was requested The weight-order test is the one worth reading. It compares SBD's CarryOverAdet selection against ranking the amplitudes SBD would otherwise have returned: two independent routes, agreeing on the set and on the order. It also asserts the result is *not* ascending, since a canonically ordered list could match by coincidence if the weights happened to rank the determinants that way. That pins the ordering the docstrings promise, which differs from every other determinant list in the wrapper because CarryOverAdet never calls sort_bitarray. Verified non-vacuous: with python/sbd_solver.py reverted to origin/main, all ten fail; with the change, all ten pass. Full suite 12 passed, 7 skipped, plus the 2-rank MPI test green, on the cpu backend against system OpenMPI 5.0.9. Assisted-by: Claude Opus 5 (1M context)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SBD selects a subset of its subspace whenever
carryover_typeis nonzero, returning it incarryover_adet/carryover_bdet. This wrapper discarded that and returned theeigenvector instead, read back from a file SBD dumps to disk.
Qiskit/qiskit-addon-sqd#369 introduces the ability for a solver to
report those determinants directly, as
SCIResult.carryover, leavingSCIResult.sci_stateas
None. This takes that path.Why
The eigenvector is
len(strings_a) * len(strings_b)doubles — the one quantity a largedistributed calculation cannot afford to move between processes. Writing it to disk and
reading it back is pure overhead when the caller only wants a weight-ranked determinant
selection, which is exactly what the trim SQD schedule in that PR wants and what SBD has
already computed.
Interface
report_carryoverselects between the two paths:None(default)carryover_typeis nonzero; return an eigenvector otherwiseTrueValueErrorifcarryover_type=0, since SBD then selects nothingFalseSince #28 defaults
carryover_typeto 0, this is opt-in: a caller setscarryover_typeto 1, 2 or 3 throughsbd_config.Revisiting a comment from #28
#28 set
carryover_type = 0with the rationale that SBD's carryover is "NOT consumed onthis path". That was true when written; with this change it describes a default rather than
the wrapper. The comment is updated accordingly. The default value is unchanged — the
reasoning behind it still holds, since most callers ignore SBD's selection and should not
pay to compute it.
Two ordering facts
#29 established that these should be stated rather than assumed:
CarryOverAdet/CarryOverBdet(upstreamchemistry/tpb/rdmat.h:102) sort by the diagonal reduceddensity matrix and keep the top entries; neither calls
sort_bitarray, so the result isneither canonically ordered nor deduplicated. That is the order a trim wants, and the
addon applies its own shape constraints to whatever a policy returns, so nothing here
depends on either property.
canonically ordered, deduplicated lists SBD was given.
The two paths therefore return differently ordered strings. The docstrings now say so.
Ranks other than the control process return a well-formed empty carryover, since the
configuration recovery loop reads results on rank 0 only.
Tests
test/test_report_carryover.pyadds ten tests over the vendored h2o data. Nothingpreviously exercised
report_carryover, and nothing exercised the amplitude handlingeither, so the suite could not tell the two paths apart — which matters because the choice
is invisible in an energy, identical either way, and shows up only as a wrong subspace one
iteration later.
(carryover_type, report_carryover)matrix, assertingexactly one of
sci_state/carryoveris populatedreport_carryover=Truewithcarryover_type=0raises rather than returning emptyint(ratio * len(strings))determinants per spin sectorcarryover_typeis nonzero (theproperty possible bug in returning wavefunction amplitudes #19 fixed)
The weight-order test is the one worth reading: it compares SBD's
CarryOverAdetselectionagainst ranking the amplitudes SBD would otherwise have returned. Two independent routes,
agreeing on the set and the order. It also asserts the result is not ascending, since
a canonically ordered list could match by coincidence.
Verified non-vacuous: with
python/sbd_solver.pyreverted toorigin/main, all tenfail; with the change, all ten pass.
Verification
Built against system OpenMPI 5.0.9 with a matching from-source
mpi4py, cpu backend:pytest test/— 12 passed, 7 skippedmpirun ... pytest --only-mpi— 1 passed, 18 skippedreport_carryovermatrix confirmed by hand before the tests were written:carryover_type=1+ default reportscarryover=(4,4)from 40 inputs atratio=0.1,while
report_carryover=Falsereturns the full(40,40)eigenvector, same energy(
-85.29400075) in every combinationStill blocked on the pin
SCIResult.carryoveris unreleased, so theqiskit-addon-sqdpin inpyproject.tomlcannot be raised yet. Testing here used an editable install of the
qiskit-addon-sqd#369 branch. This stays a draft until that lands and ships.
This PR was generated by Claude Opus 5 under my guidance.