Skip to content

Report SBD's trimmed determinants as SCIResult.carryover - #32

Draft
Jim Garrison (garrison) wants to merge 2 commits into
mainfrom
solver-supplied-carryover
Draft

Jim Garrison (garrison) wants to merge 2 commits into
mainfrom
solver-supplied-carryover

Conversation

@garrison

@garrison Jim Garrison (garrison) commented Sep 16, 2026

Copy link
Copy Markdown
Member

SBD selects a subset of its subspace whenever carryover_type is nonzero, returning it in
carryover_adet / carryover_bdet. This wrapper discarded that and returned the
eigenvector instead, read back from a file SBD dumps to disk.

Qiskit/qiskit-addon-sqd#369 introduces the ability for a solver to
report those determinants directly, as SCIResult.carryover, leaving SCIResult.sci_state
as None. This takes that path.

Why

The eigenvector is len(strings_a) * len(strings_b) doubles — the one quantity a large
distributed calculation cannot afford to move between processes. Writing it to disk and
reading it back is pure overhead when the caller only wants a weight-ranked determinant
selection, which is exactly what the trim SQD schedule in that PR wants and what SBD has
already computed.

Interface

report_carryover selects between the two paths:

value behavior
None (default) report the carryover when carryover_type is nonzero; return an eigenvector otherwise
True report the carryover; raises ValueError if carryover_type=0, since SBD then selects nothing
False keep the eigenvector even when SBD trims — for a caller that wants amplitudes for its own analysis

Since #28 defaults carryover_type to 0, this is opt-in: a caller sets
carryover_type to 1, 2 or 3 through sbd_config.

Revisiting a comment from #28

#28 set carryover_type = 0 with the rationale that SBD's carryover is "NOT consumed on
this path". That was true when written; with this change it describes a default rather than
the wrapper. The comment is updated accordingly. The default value is unchanged — the
reasoning behind it still holds, since most callers ignore SBD's selection and should not
pay to compute it.

Two ordering facts

#29 established that these should be stated rather than assumed:

  • The carryover comes back in descending weight order. CarryOverAdet /
    CarryOverBdet (upstream chemistry/tpb/rdmat.h:102) sort by the diagonal reduced
    density matrix and keep the top entries; neither calls sort_bitarray, so the result is
    neither canonically ordered nor deduplicated. That is the order a trim wants, and the
    addon applies its own shape constraints to whatever a policy returns, so nothing here
    depends on either property.
  • The eigenvector path is untouched, including Label amplitudes with the subspace that was diagonalized, not the input #29's labelling of amplitudes with the
    canonically ordered, deduplicated lists SBD was given.

The two paths therefore return differently ordered strings. The docstrings now say so.

Ranks other than the control process return a well-formed empty carryover, since the
configuration recovery loop reads results on rank 0 only.

Tests

test/test_report_carryover.py adds ten tests over the vendored h2o data. Nothing
previously exercised report_carryover, and nothing exercised the amplitude handling
either, so the suite could not tell the two paths apart — which matters because the choice
is invisible in an energy, identical either way, and shows up only as a wrong subspace one
iteration later.

  • which output is returned, over the (carryover_type, report_carryover) matrix, asserting
    exactly one of sci_state / carryover is populated
  • report_carryover=True with carryover_type=0 raises rather than returning empty
  • the carryover holds int(ratio * len(strings)) determinants per spin sector
  • the eigenvector spans the full input subspace even when carryover_type is nonzero (the
    property possible bug in returning wavefunction amplitudes #19 fixed)
  • the carryover is the highest-weight determinants in descending weight order
  • the energy does not depend on which output was requested

The weight-order test is the one worth reading: it compares SBD's CarryOverAdet selection
against ranking the amplitudes SBD would otherwise have returned. Two independent routes,
agreeing on the set and the order. It also asserts the result is not ascending, since
a canonically ordered list could match by coincidence.

Verified non-vacuous: with python/sbd_solver.py reverted to origin/main, all ten
fail; with the change, all ten pass.

Verification

Built against system OpenMPI 5.0.9 with a matching from-source mpi4py, cpu backend:

  • pytest test/12 passed, 7 skipped
  • 2-rank mpirun ... pytest --only-mpi1 passed, 18 skipped
  • the report_carryover matrix confirmed by hand before the tests were written:
    carryover_type=1 + default reports carryover=(4,4) from 40 inputs at ratio=0.1,
    while report_carryover=False returns the full (40,40) eigenvector, same energy
    (-85.29400075) in every combination

Still blocked on the pin

SCIResult.carryover is unreleased, so the qiskit-addon-sqd pin in pyproject.toml
cannot be raised yet. Testing here used an editable install of the
qiskit-addon-sqd#369 branch. This stays a draft until that lands and ships.


This PR was generated by Claude Opus 5 under my guidance.

SBD selects a subset of its subspace whenever carryover_type is nonzero, and
returns it in carryover_adet and carryover_bdet. This wrapper discarded it and
returned the eigenvector instead, which it reads back from a file SBD dumps.

The PR at Qiskit/qiskit-addon-sqd#369 introduces the
ability for a solver to report those determinants directly, as
SCIResult.carryover, leaving SCIResult.sci_state as None. That removes the disk
round-trip: the eigenvector is len(strings_a) * len(strings_b) doubles, which is
the one quantity a large distributed calculation cannot afford to move between
processes. The trim SQD schedule that PR adds wants exactly the weight-ranked
selection SBD already computed, so the eigenvector is not merely expensive there,
it is unnecessary.

The new report_carryover argument selects between the two. It defaults to None,
meaning report the carryover when SBD selects one and return an eigenvector when
it does not. Since #28 defaults carryover_type to 0, that makes this opt-in: a
caller sets carryover_type to 1, 2 or 3 through sbd_config. Pass False to keep the
eigenvector even when SBD trims, for a caller that wants the amplitudes for its
own analysis. Passing True with carryover_type=0 raises rather than silently
returning an empty carryover.

The comment on that default is updated: it said SBD's carryover is "NOT consumed
on this path", which was true when written and is now the description of a default
rather than of the wrapper. The reasoning behind the default is unchanged -- most
callers ignore SBD's selection and should not pay to compute it.

Two ordering facts, since #29 established that these should be stated rather than
assumed:

- The carryover comes back in descending weight order. CarryOverAdet and
  CarryOverBdet (upstream chemistry/tpb/rdmat.h) sort by the diagonal reduced
  density matrix and keep the top entries; neither calls sort_bitarray, so this is
  neither canonical order nor deduplicated. That is the order a trim wants, and
  the addon applies its own shape constraints to whatever a policy returns, so
  nothing here depends on either property.
- The eigenvector path is untouched, including #29's labelling of amplitudes with
  the canonically ordered, deduplicated lists SBD was given. The two paths
  therefore return differently ordered strings, which the docstrings now say.

Ranks other than the control process return a well-formed empty carryover, since
the configuration recovery loop reads results on rank 0 only.

Not mergeable until SCIResult.carryover is released: qiskit-addon-sqd#369 is still
open, so the qiskit-addon-sqd pin in pyproject.toml cannot yet be raised to a
version that has it. The pin needs bumping once that lands and ships.

Assisted-by: Claude Opus 5 (1M context)
Nothing exercised report_carryover, and nothing exercised the amplitude handling
either, so the suite could not distinguish the two paths solve_sci chooses between.
That matters more than it looks: the choice is invisible in an energy, which is
identical either way, so a wrong path shows up only as a wrong subspace one
iteration later.

Ten tests over the vendored h2o data, using 40 alpha determinants so a ratio leaves
a proper subset to inspect:

- which output is returned, over the (carryover_type, report_carryover) matrix,
  asserting that exactly one of sci_state and carryover is populated
- report_carryover=True with carryover_type=0 raises rather than returning an empty
  carryover
- the carryover holds int(ratio * len(strings)) determinants per spin sector
- with the eigenvector requested, it spans the full input subspace even when
  carryover_type is nonzero -- the property #19 fixed
- the carryover is the highest-weight determinants in descending weight order
- the energy does not depend on which output was requested

The weight-order test is the one worth reading. It compares SBD's CarryOverAdet
selection against ranking the amplitudes SBD would otherwise have returned: two
independent routes, agreeing on the set and on the order. It also asserts the result
is *not* ascending, since a canonically ordered list could match by coincidence if
the weights happened to rank the determinants that way. That pins the ordering the
docstrings promise, which differs from every other determinant list in the wrapper
because CarryOverAdet never calls sort_bitarray.

Verified non-vacuous: with python/sbd_solver.py reverted to origin/main, all ten
fail; with the change, all ten pass. Full suite 12 passed, 7 skipped, plus the
2-rank MPI test green, on the cpu backend against system OpenMPI 5.0.9.

Assisted-by: Claude Opus 5 (1M context)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant