Skip to content

test(verify): give every operator the same table to read - #8360

Open
kz930 wants to merge 5 commits into
apache:mainfrom
kz930:feat/verify-canonical-table
Open

kz930 wants to merge 5 commits into
apache:mainfrom
kz930:feat/verify-canonical-table

Conversation

@kz930

@kz930 kz930 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

One table, thirty-four columns and fifteen rows, holding a column of every
type an operator can ask for and the awkward values each type admits: an empty
cell, a zero, a negative, a string that looks like a number, a date at a
boundary.

An operator that needs something the table cannot hold — a pair of tables
sharing a key, a numeric projection — names a projection of it rather than a
fixture of its own, so what every operator reads stays one thing.

Any related issues, documentation, discussions?

Part of #8325, 8 of 27; that issue lists the set in order.

Closes #8412, the task this change is the whole of.

How was this PR tested?

The tests in this change cover it. The whole set is exercised together once the last piece lands: every operator run through the engine and through its generated script, and the two answers compared.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

One table, thirty-four columns and fifteen rows, holding a column of every
type an operator can ask for and the awkward values each type admits: an empty
cell, a zero, a negative, a string that looks like a number, a date at a
boundary.

An operator that needs something the table cannot hold — a pair of tables
sharing a key, a numeric projection — names a projection of it rather than a
fixture of its own, so what every operator reads stays one thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added feature platform Non-amber Scala service paths labels Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@codecov-commenter

codecov-commenter commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.74%. Comparing base (bf356f5) to head (3f1c611).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8360      +/-   ##
============================================
+ Coverage     93.50%   95.74%   +2.24%     
+ Complexity     4882      477    -4405     
============================================
  Files          1220      590     -630     
  Lines         50788    33952   -16836     
  Branches       6262     4250    -2012     
============================================
- Hits          47489    32508   -14981     
+ Misses         1746      856     -890     
+ Partials       1553      588     -965     
Flag Coverage Δ *Carryforward flag
access-control-service 71.78% <ø> (ø)
agent-service 99.32% <ø> (ø) Carriedforward from 5042d96
amber 90.82% <ø> (+1.71%) ⬆️ Carriedforward from 5042d96
computing-unit-managing-service 77.14% <ø> (ø)
config-service 87.25% <ø> (ø)
file-service 81.53% <ø> (ø)
frontend 96.72% <ø> (+0.03%) ⬆️ Carriedforward from 5042d96
notebook-migration-service 83.73% <ø> (ø)
pyamber 98.47% <ø> (ø) Carriedforward from 5042d96
workflow-compiling-service 74.09% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared fixture and its tests look good.

@kz930
kz930 marked this pull request as draft September 4, 2026 17:12
An operator writes a column name into the code it emits, and a name holding the
characters that end a Python literal has to be escaped on the way. Renaming
seven of this table's columns puts that question to a real run rather than to a
reader.

They are renamed rather than added, so the table keeps its width and the
arrangements an operator needs: three components summing to 100, a label with a
prediction over it. One of the seven carries a single quote and a newline too,
the two that end a '...' literal and a # comment. HostileColumn indexes them by
type, for the configuration generator to reach.

A paragraph of scaladoc that sat above the wrong member moves back to the
projection it describes, and the comments naming these columns now name them as
what they are rather than as identifiers that no longer resolve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kz930
kz930 force-pushed the feat/verify-canonical-table branch from 6d002e3 to 7eb6d7c Compare September 9, 2026 06:50
kz930 and others added 2 commits September 10, 2026 13:13
The table every operator reads carried doubles, strings, integers, a
long and timestamps, and no boolean at all. An operator that renders a
column as text answers differently for one: the engine writes the
field's toString and gets "true", where Python's str() writes "True".
Nothing in the shared table could ask the question.

The column is `score >= 3.0` rather than free-standing, so the spec can
state what it holds: six false and nine true, with both values inside
either port's window.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A port can carry no rows at all: an upstream filter that matches nothing
still ends its channel. That is a different table from one with holes in
it, since a column with no values has no minimum, no quantile and no
inferable type, so it needs a writer of its own. The shaping the gap
variant already did moves behind that writer so both get the same
sidecars.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kz930

kz930 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@carloea2 this moved after your approval: seven fixture columns were renamed to hostile spellings, the canonical table gained a boolean column, and a fixture can now write the same columns with no rows under them. Would you take another look when you have a moment?

@kz930
kz930 marked this pull request as ready for review September 15, 2026 07:49

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The canonical table now covers empty data, booleans, and difficult column names. Looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Give every operator the same table to read

3 participants