test_base_table_column_addition_mv flakes: MV schema-notification lag exceeds retry budget

[mykaul]

Summary

test_base_table_column_addition_mv (tests/integration/standard/test_metadata.py:2334) intermittently fails in CI. It surfaced most recently as a seemingly-unrelated CI failure on PR #786 (a _query() callback micro-optimization) — investigation confirmed #786's own change is correct (the closure captures host/connection/pool via default arguments, so there is no late-binding bug), and the failure is this pre-existing test flake, unrelated to that PR's diff.

Root cause

The test already contains a self-documented workaround for exactly this class of issue:

# This is a workaround for mv notifications being separate from base table schema responses.
# This maybe fixed with future protocol changes
for i in range(10):
    mv_alltime = self.cluster.metadata.keyspaces[self.keyspace_name].views["alltimehigh"]
    if("fouls" in mv_alltime.columns):
        break
    time.sleep(.2)

assert "fouls" in mv_alltime.columns

Materialized-view schema-change notifications arrive as a separate control-connection event from the base-table schema response, so there's an inherent lag between ALTER TABLE ... ADD fouls completing and the MV's own metadata reflecting the new column. The existing retry loop caps out at 10 × 0.2s = 2s total wait, which is apparently not always enough under CI load/contention, so the final assert "fouls" in mv_alltime.columns occasionally fails.

Impact

Intermittent, non-deterministic CI failures in test_base_table_column_addition_mv, which can mask unrelated changes' CI results (as happened with PR #786) and erode trust in CI signal.

Suggested fix direction

• Increase the retry budget/backoff (the current 2s cap is arbitrary and was likely sized for lighter CI load).
• Consider waiting on an actual schema-agreement/event signal instead of a fixed-count polling sleep, if the driver exposes one for MV metadata updates.
• Apply the same treatment to any sibling tests with the identical workaround pattern (e.g. test_base_table_type_alter_mv immediately below it, and other MV-metadata tests in the same file), if they share the same fragile assumption.

🤖 Generated with Claude Code