From e3f9503431967db60aa372af56937db11a168e3d Mon Sep 17 00:00:00 2001 From: Makisuo Date: Mon, 21 Sep 2026 21:09:39 +0200 Subject: [PATCH 1/4] chore(db): drop the migration privilege and preflight scripts The prd deploy applies migrations, so the standalone helpers around the old `migrate:prod` path are dead: the PUBLIC privilege scaffolding, the session role pin, and the drizzle 1.0 bookkeeping preflight (prod passed it on 2026-09-19). The credential broker stays for the restore drill, the raw-SQL audit and the backfills. The privilege invariant moves to the roles themselves: every runtime role is a member of `postgres`, which the docs now state with the query that checks it. --- .github/workflows/deploy-pr-preview.yml | 2 +- CLAUDE.md | 5 +- docs/persistence.md | 32 +-- packages/db/package.json | 3 - packages/db/scripts/ensure-privileges.test.ts | 28 --- packages/db/scripts/ensure-privileges.ts | 151 ----------- packages/db/scripts/migrations-preflight.ts | 234 ------------------ packages/db/scripts/planetscale-connection.ts | 60 +---- .../planetscale-migrations-preflight.ts | 21 -- packages/db/scripts/reset-preview-branch.ts | 2 +- 10 files changed, 16 insertions(+), 522 deletions(-) delete mode 100644 packages/db/scripts/ensure-privileges.test.ts delete mode 100644 packages/db/scripts/ensure-privileges.ts delete mode 100644 packages/db/scripts/migrations-preflight.ts delete mode 100644 packages/db/scripts/planetscale-migrations-preflight.ts diff --git a/.github/workflows/deploy-pr-preview.yml b/.github/workflows/deploy-pr-preview.yml index 139ee9956d..2c2860d86d 100644 --- a/.github/workflows/deploy-pr-preview.yml +++ b/.github/workflows/deploy-pr-preview.yml @@ -145,7 +145,7 @@ jobs: # # To restore per-PR databases: flip that resolver back to "managed" for # `pr` and re-add the steps this comment replaced (PlanetScale CLI setup, - # `planetscale-pr-branch.ts up`, `db:ensure-privileges`, `db:migrate`, + # `planetscale-pr-branch.ts up`, `db:migrate`, # `db:normalize-preview`, `electric-pr-branch.ts up`, and the matching # `planetscale-pr-branch.ts down` teardown). Every script is still in the # repo, dormant. diff --git a/CLAUDE.md b/CLAUDE.md index 9bce93693b..18fc7b7fc0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -183,8 +183,9 @@ database), reached from Workers via the Hyperdrive binding `MAPLE_DB`. - Migrations: `bun run --cwd packages/db db:generate`. **The prd deploy applies them**: the PlanetScale `main` branch is an alchemy `Planetscale.PostgresBranch` in `alchemy.run.ts` with `migrations` pointed at `packages/db/drizzle`; never run `drizzle-kit migrate` against prd. It - migrates as a temporary role, so a migration creating a table must `GRANT` it `TO PUBLIC` itself - (the ingest gateway reads only through PUBLIC). PGlite applies them at layer build. + migrates as a temporary role dropped with `postgres` as successor, so every runtime role must be a + member of `postgres` (`pg_has_role(rolname, 'postgres', 'member')`) to read what it creates. PGlite + applies them at layer build. - **PR preview deploys are label-gated** (2026-08, cost — re-enabled by `fd00bcd412`). A PR gets a preview only while it carries the `preview` label; `deploy-pr-preview.yml` triggers on `opened, reopened, synchronize, labeled, unlabeled, closed` and tears the stack down the moment diff --git a/docs/persistence.md b/docs/persistence.md index c673bddca5..5ab2f826d7 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -92,32 +92,18 @@ The prd deploy applies migrations: `alchemy.run.ts` declares the PlanetScale `ma `Planetscale.PostgresBranch` with `migrations` pointed at `packages/db/drizzle`, and the api, ai and alerting Workers carry its name in their env so they upload after it. Bookkeeping is alchemy's `__alchemy_migrations`; `drizzle.__drizzle_migrations` was copied in once and is frozen, so never run -`drizzle-kit migrate` against prd. The deploy migrates as a temporary role, not `postgres`, so the -branch's default privileges do not cover the tables it creates: a migration that creates one grants -it `TO PUBLIC` itself. The deploy reads `PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN` / -`PLANETSCALE_ORGANIZATION` from Infisical prod; `bun dev` leaves the PlanetScale provider out. - -The first v1 migrate on a database migrated by drizzle 0.x upgrades `drizzle.__drizzle_migrations` -in place (adds `name` and `applied_at`), matching every existing row to a local folder by -`created_at` truncated to the second, then by hash, and **refusing the whole run if any row matches -nothing**. A row like that is a migration that was applied and later renumbered or re-timestamped, -or one applied from a branch that never merged. Check before migrating. The report prints a -DELETE for a superseded row and an UPDATE for a renumbered row whose SQL is byte-identical; a row -whose SQL changed after it ran gets a `git diff` instead, because relabelling it would record -statements this database never saw as applied. - -The report also lists every local migration no row matches, because the v1 migrator applies all -of them where the 0.x migrator only applied those newer than the newest recorded timestamp. A -migration whose DDL reached the schema without a row (a `db:push`, a run that died after its -transaction committed) used to be skipped silently and now fails on the objects that already -exist. Compare each pending folder's first statement with the schema; record the ones already -applied with the INSERT the report prints rather than replaying them: +`drizzle-kit migrate` against prd. The deploy migrates as a temporary role that is dropped with +`postgres` as its successor, so the tables it creates end up owned by `postgres` with no other grants. +Every runtime role must therefore be a member of `postgres`: mint credentials with +`--inherited-roles postgres`, and this must return no rows: -```bash -bun run --cwd packages/db db:migrate:preflight # DATABASE_URL, defaults to the docker Postgres -bun run --cwd packages/db ps:migrations-preflight main # a PlanetScale branch, read-only +```sql +SELECT rolname FROM pg_roles WHERE rolcanlogin AND NOT pg_has_role(rolname, 'postgres', 'member') ``` +The deploy reads `PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN` / +`PLANETSCALE_ORGANIZATION` from Infisical prod; `bun dev` leaves the PlanetScale provider out. + PGlite applies the same bundled migrations while its layer is built. The test harness caches a fresh migrated PGlite snapshot and restores it per test, so integration tests exercise the PostgreSQL schema without a shared server. diff --git a/packages/db/package.json b/packages/db/package.json index 5b9c6c479b..98400d504b 100644 --- a/packages/db/package.json +++ b/packages/db/package.json @@ -15,14 +15,11 @@ "test": "vitest run --passWithNoTests", "db:generate": "drizzle-kit generate --config ./drizzle.config.ts", "db:migrate": "drizzle-kit migrate --config ./drizzle.config.ts", - "db:migrate:preflight": "bun scripts/migrations-preflight.ts", "db:push": "drizzle-kit push --config ./drizzle.config.ts", "db:studio": "drizzle-kit studio --config ./drizzle.config.ts", - "db:ensure-privileges": "bun scripts/ensure-privileges.ts", "db:reset-preview": "bun scripts/reset-preview-branch.ts", "db:audit-raw-sql": "bun scripts/audit-raw-sql.ts", "db:normalize-preview": "bun scripts/normalize-preview-ownership.ts", - "ps:migrations-preflight": "bun scripts/planetscale-migrations-preflight.ts", "db:restore-test": "bun scripts/restore-test.ts", "db:backfill:dashboards-v3": "bun scripts/backfill-dashboard-datasource-v3.ts" }, diff --git a/packages/db/scripts/ensure-privileges.test.ts b/packages/db/scripts/ensure-privileges.test.ts deleted file mode 100644 index 337b3d1e63..0000000000 --- a/packages/db/scripts/ensure-privileges.test.ts +++ /dev/null @@ -1,28 +0,0 @@ -import { describe, expect, it } from "vitest" -import { defaultPrivilegeStatements, sweepStatements } from "./ensure-privileges" - -describe("defaultPrivilegeStatements", () => { - it("keys table and sequence defaults to the given creating role", () => { - const statements = defaultPrivilegeStatements("postgres") - expect(statements).toHaveLength(2) - for (const statement of statements) { - expect(statement).toContain('ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA public') - expect(statement).toContain("TO PUBLIC") - } - }) - - // The standalone path keys defaults per candidate creating role (login role - // AND postgres) — the helper must therefore be callable per role, with the - // dotted PlanetScale login role quoted as one identifier. - it("quotes a dotted PlanetScale login role", () => { - const statements = defaultPrivilegeStatements("pscale_api_abc.def") - expect(statements[0]).toContain('FOR ROLE "pscale_api_abc.def"') - }) -}) - -describe("sweepStatements", () => { - it("backfills existing tables and sequences to PUBLIC", () => { - expect(sweepStatements.some((s) => s.includes("ON ALL TABLES IN SCHEMA public"))).toBe(true) - expect(sweepStatements.some((s) => s.includes("ON ALL SEQUENCES IN SCHEMA public"))).toBe(true) - }) -}) diff --git a/packages/db/scripts/ensure-privileges.ts b/packages/db/scripts/ensure-privileges.ts deleted file mode 100644 index 7d2ba0742f..0000000000 --- a/packages/db/scripts/ensure-privileges.ts +++ /dev/null @@ -1,151 +0,0 @@ -#!/usr/bin/env bun -/** - * Make a PlanetScale Postgres branch grant correct privileges to new tables BY - * ITSELF, so a plain `drizzle-kit migrate` can never again ship a table the - * fleet cannot read. - * - * Run this BEFORE migrate (order matters — `ALTER DEFAULT PRIVILEGES` only - * applies to objects created after it): - * - * DATABASE_URL="$MAPLE_PG_URL" bun packages/db/scripts/ensure-privileges.ts - * - * The deploy migrates as a temporary role these defaults never key to: a - * migration that creates a table must GRANT it to PUBLIC itself. - * - * ── Why PUBLIC, and why no runtime-role name ────────────────────────────── - * Prod has four `pscale_api_*` login roles. Three are members of `postgres` - * with rolinherit and so can read a postgres-owned table through inheritance; - * `pscale_api_rg068pnctlxw` — the ingest gateway, which connects via PSBouncer - * on 6432 rather than Hyperdrive — is NOT a member and reads only through - * PUBLIC. That asymmetry is the whole shape of the 2026-07-29 outage: a new - * table with owner-only privileges broke the gateway alone while the API and - * alerting workers kept serving, so it hid for 21.7h. - * - * PUBLIC covers every consumer without naming any of them, which is what lets - * this run with no configuration. The old approach needed MAPLE_PG_RUNTIME_ROLE - * naming one specific role, and granting that role was never what kept the - * fleet up. - * - * ── Why default privileges rather than a post-migrate grant sweep ───────── - * A sweep has to be remembered on every path that ever runs DDL. Default - * privileges are a property of the branch: once set, every subsequent - * `CREATE TABLE` is born correct, including tables created by a rebuild - * migration that DROPped its grants. The sweep below is kept only to heal - * objects that predate this — it is idempotent and cheap. - * - * Default privileges are keyed to the CREATING role, and the standalone path - * cannot know which identity a later `drizzle-kit migrate` process will create - * objects as — its own `SET ROLE postgres` is session-scoped, and only the - * brokered prod connection persists the pin (`pinSessionRoleToPostgres` in - * planetscale-connection.ts). So defaults are keyed to BOTH candidates: the - * login role (while the session still is it), then `postgres` where membership - * allows the switch. Whichever one migrate's connections end up creating as, - * its defaults fire. - */ -import * as Predicate from "effect/Predicate" -import postgres from "postgres" -import { fail } from "./planetscale-connection" - -/** - * Role names we are willing to interpolate as a quoted identifier — the value - * comes from `SELECT current_user`, not user input, but is validated anyway. - * PlanetScale roles are dotted (`pscale_api_.`); `.` is safe because we - * always double-quote, where Postgres treats it as a literal character. - */ -const ROLE_PATTERN = /^[A-Za-z_][A-Za-z0-9_$.-]*$/ - -const quoteIdent = (role: string): string => { - if (!ROLE_PATTERN.test(role)) { - fail(`Refusing to use unsafe role name ${JSON.stringify(role)} (allowed: ${ROLE_PATTERN})`) - } - return `"${role}"` -} - -/** - * Statements are ordered: schema usage, then the default privileges that make - * FUTURE objects correct, then the backfill sweep for existing ones. - * - * Defaults are keyed to EVERY role migrations might create objects as, not - * just one: this script's `SET ROLE postgres` lasts only for its own session, - * and `drizzle-kit migrate` runs later as a separate process whose connections - * authenticate as the login role. Unless that login carries a persisted - * `role=postgres` (the brokered prod path's `ALTER ROLE … SET role`, see - * planetscale-connection.ts — a standalone run has no such guarantee), - * its objects are created by the login role and postgres-keyed defaults never - * fire — recreating exactly the owner-only-table outage this script prevents. - * - * PUBLIC is a keyword, not an identifier — it must never be quoted. - */ -export const defaultPrivilegeStatements = (owner: string): readonly string[] => { - const ident = quoteIdent(owner) - return [ - `ALTER DEFAULT PRIVILEGES FOR ROLE ${ident} IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO PUBLIC`, - `ALTER DEFAULT PRIVILEGES FOR ROLE ${ident} IN SCHEMA public GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO PUBLIC`, - ] -} - -export const sweepStatements: readonly string[] = [ - "GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO PUBLIC", - "GRANT USAGE, SELECT, UPDATE ON ALL SEQUENCES IN SCHEMA public TO PUBLIC", -] - -/** Ask the server — a PlanetScale URL username carries a routing suffix that is - * stripped before the server sees it, so it is not a usable role name. */ -const currentUser = async (sql: postgres.Sql): Promise => { - const [row] = await sql`SELECT current_user` - const role: unknown = row?.current_user - if (!Predicate.isString(role) || role.length === 0) { - return fail("`SELECT current_user` returned no role") - } - return role -} - -/** - * Apply the privilege scaffolding on `connectionUrl`. Idempotent — safe to run - * before every migrate. - */ -export const ensureRuntimePrivileges = async (connectionUrl: string): Promise => { - const sql = postgres(connectionUrl, { max: 1, fetch_types: false }) - try { - const runStatements = async (batch: readonly string[]): Promise => { - for (const statement of batch) { - console.log(` → ${statement}`) - await sql.unsafe(statement) - } - } - - await runStatements(["GRANT USAGE ON SCHEMA public TO PUBLIC"]) - // Key defaults to the LOGIN role first, while the session still is it: - // `ALTER DEFAULT PRIVILEGES FOR ROLE x` needs membership in x, and - // `postgres` is not a member of its own members. - const loginRole = await currentUser(sql) - console.log(`→ Ensuring PUBLIC privileges for objects created by "${loginRole}"`) - await runStatements(defaultPrivilegeStatements(loginRole)) - // Then as `postgres` where membership allows it, so the defaults are also - // keyed to the role that creates prod's tables (the brokered path pins - // migrations to run as postgres — see planetscale-connection.ts). - try { - await sql.unsafe("SET ROLE postgres") - } catch { - console.log("→ SET ROLE postgres not permitted — defaults keyed to the session role only") - } - const owner = await currentUser(sql) - if (owner !== loginRole) { - console.log(`→ Ensuring PUBLIC privileges for objects created by "${owner}"`) - await runStatements(defaultPrivilegeStatements(owner)) - } - await runStatements(sweepStatements) - console.log(`\n✓ Privileges ensured — future tables are granted to PUBLIC at creation`) - } finally { - await sql.end() - } -} - -// CLI entry (skipped when imported). -if (import.meta.main) { - const url = process.env.DATABASE_URL?.trim() - if (!url) { - fail("DATABASE_URL is not set — usage: DATABASE_URL=… bun scripts/ensure-privileges.ts") - } - await ensureRuntimePrivileges(url as string) -} diff --git a/packages/db/scripts/migrations-preflight.ts b/packages/db/scripts/migrations-preflight.ts deleted file mode 100644 index 24d893ba77..0000000000 --- a/packages/db/scripts/migrations-preflight.ts +++ /dev/null @@ -1,234 +0,0 @@ -/** - * Preflight for `drizzle-kit migrate` on a database whose - * `drizzle.__drizzle_migrations` table predates drizzle 1.0. - * - * The v1 migrator upgrades that table once, matching every existing row to a - * local migration folder by `created_at` truncated to the second, then by hash, - * and refuses to run if any row matches nothing. Rows like that exist wherever a - * migration was applied and later renumbered or re-timestamped, or came from a - * branch that never merged. This applies the same rules ahead of time and says - * which row is which, so the fix is a deliberate UPDATE or DELETE rather than a - * failed deploy. - * - * DATABASE_URL=postgres://… bun scripts/migrations-preflight.ts - * - * It also lists every local migration no row matches, because the v1 migrator - * applies all of them (the 0.x migrator only applied those newer than the - * newest recorded timestamp). One whose DDL is already in the schema must be - * recorded, not replayed. - * - * Read-only. Exits 1 when the migrator would refuse. - */ -import { createHash } from "node:crypto" -import { readFileSync } from "node:fs" -import { spawnSync } from "node:child_process" -import { resolve } from "node:path" -import postgres from "postgres" -import { listBundledMigrations } from "../src/migrate" - -const migrationsFolder = resolve(import.meta.dir, "../drizzle") - -const url = process.env.DATABASE_URL ?? "postgres://maple:maple@localhost:5499/maple" - -interface LocalMigration { - readonly name: string - readonly suffix: string - readonly millis: number - readonly hash: string -} - -const folderMillis = (name: string): number => { - const stamp = name.slice(0, 14) - const iso = `${stamp.slice(0, 4)}-${stamp.slice(4, 6)}-${stamp.slice(6, 8)}T${stamp.slice(8, 10)}:${stamp.slice(10, 12)}:${stamp.slice(12, 14)}.000Z` - return Date.parse(iso) -} - -const locals: ReadonlyArray = listBundledMigrations().map(({ name, sqlPath }) => ({ - name, - suffix: name.slice(15), - millis: folderMillis(name), - hash: createHash("sha256").update(readFileSync(sqlPath)).digest("hex"), -})) -const byMillis = new Map>() -const byHash = new Map() -for (const local of locals) { - byMillis.set(local.millis, [...(byMillis.get(local.millis) ?? []), local]) - byHash.set(local.hash, local) -} - -/** The migration name a historical path carried: `0003_premium_korg.sql` or `_premium_korg/migration.sql`. */ -const nameFromPath = (path: string): string | undefined => { - const folder = /\/(\d{14})_([^/]+)\/migration\.sql$/.exec(path) - if (folder) return folder[2] - const flat = /\/\d{4}_([^/]+)\.sql$/.exec(path) - return flat ? flat[1] : undefined -} - -interface HistoricalBlob { - readonly oid: string - /** Every migration name this exact SQL has been filed under. More than one is ambiguous. */ - readonly names: ReadonlySet -} - -/** - * Every migration SQL blob this checkout's git history has ever held, keyed by - * its sha256. `rev-list --objects` reports a blob once per path it was reached - * through, so a renumbered file shows up under each of its names; all of them - * are kept, and a digest that maps to more than one name is left to a human. - */ -const historicalBlobs = (): Map => { - const blobs = new Map }>() - // `cwd` pins the pathspec to this folder whichever directory the script runs from. - const listing = spawnSync("git", ["rev-list", "--all", "--objects", "--", "."], { - cwd: migrationsFolder, - encoding: "utf8", - }) - if (listing.status !== 0) return blobs - const digests = new Map() - for (const line of listing.stdout.split("\n")) { - const [oid, path] = line.split(" ", 2) - if (!oid || !path?.endsWith(".sql")) continue - let digest = digests.get(oid) - if (digest === undefined) { - const blob = spawnSync("git", ["cat-file", "blob", oid], { cwd: migrationsFolder }) - if (blob.status !== 0) continue - digest = createHash("sha256").update(blob.stdout).digest("hex") - digests.set(oid, digest) - } - const entry = blobs.get(digest) ?? { oid, names: new Set() } - const name = nameFromPath(path) - if (name !== undefined) entry.names.add(name) - blobs.set(digest, entry) - } - return blobs -} - -/** - * The v1 migrator applies every unrecorded folder, where the 0.x migrator only - * applied those newer than the newest recorded timestamp. A migration whose - * DDL reached the schema without a row used to be skipped and now fails on the - * objects that already exist, so say exactly what will run. - * - * `hasNameColumn` picks the INSERT: before the upgrade the table has no `name`, - * and a row on the folder's second is matched to it by the upgrade itself. - */ -const reportPending = (recordedNames: ReadonlySet, hasNameColumn: boolean): void => { - const pending = locals.filter((local) => !recordedNames.has(local.name)) - console.log(`\n${pending.length} local migration(s) have no row and WILL be applied by the v1 migrator:`) - for (const local of pending) { - const first = readFileSync(resolve(migrationsFolder, local.name, "migration.sql"), "utf8") - .split("\n") - .find((line) => /^(create|alter|drop)\b/i.test(line)) - console.log(` ${local.name}\n ${first?.slice(0, 110) ?? "(no DDL statement)"}`) - } - if (pending.length > 0) { - const insert = hasNameColumn - ? "INSERT INTO drizzle.__drizzle_migrations (hash, created_at, name) VALUES ('', , '');" - : "INSERT INTO drizzle.__drizzle_migrations (hash, created_at) VALUES ('', );" - console.log( - ` If one of these already reached the schema without a row, record it instead of replaying it:\n ${insert}`, - ) - for (const local of pending) - console.log(` ${local.name}: hash ${local.hash} created_at ${local.millis}`) - } -} - -const sql = postgres(url, { max: 1, fetch_types: false }) - -/** The exit code; returned rather than `process.exit`ed so the connection is closed first. */ -const preflight = async (): Promise => { - const columns = await sql<{ column_name: string }[]>` - select column_name from information_schema.columns - where table_schema = 'drizzle' and table_name = '__drizzle_migrations' order by ordinal_position` - if (columns.length === 0) { - console.log("No drizzle.__drizzle_migrations table: a fresh database, nothing to upgrade.") - return 0 - } - if (columns.some((c) => c.column_name === "name")) { - console.log("Migrations table is already on the v1 layout (has `name`); the upgrade will not run.") - const named = await sql<{ name: string | null }[]>`select name from drizzle.__drizzle_migrations` - reportPending(new Set(named.flatMap((row) => (row.name === null ? [] : [row.name]))), true) - return 0 - } - const rows = await sql<{ id: number; created_at: string; hash: string }[]>` - select id, created_at, hash from drizzle.__drizzle_migrations order by id asc` - - const orphans: Array<{ id: number; createdAt: number; hash: string }> = [] - const matchedNames = new Set() - for (const row of rows) { - const createdAt = Number(row.created_at) - const millis = Math.floor(createdAt / 1000) * 1000 - const candidates = byMillis.get(millis) - const found = - candidates && candidates.length === 1 - ? candidates[0] - : candidates && candidates.length > 1 - ? candidates.find((c) => c.hash === row.hash) - : byHash.get(row.hash) - if (found) matchedNames.add(found.name) - else orphans.push({ id: row.id, createdAt, hash: row.hash }) - } - console.log( - `${rows.length} rows, ${matchedNames.size} match a local migration, ${orphans.length} would make the migrator refuse.`, - ) - - reportPending(matchedNames, false) - - if (orphans.length === 0) return 0 - - const history = historicalBlobs() - const recorded = new Set(rows.map((r) => Math.floor(Number(r.created_at) / 1000) * 1000)) - for (const orphan of orphans) { - console.log( - `\nrow ${orphan.id}: created_at ${new Date(orphan.createdAt).toISOString()} hash ${orphan.hash.slice(0, 12)}…`, - ) - const historical = history.get(orphan.hash) - if (!historical) { - console.log(" not in this checkout's history: applied from another branch, decide by hand") - continue - } - const names = [...historical.names] - if (names.length !== 1) { - console.log( - ` this SQL was filed under ${names.length} names (${names.join(", ") || "none recognisable"}): decide by hand`, - ) - continue - } - const name = names[0]! - const currents = locals.filter((l) => l.suffix === name) - console.log(` is the historical ${name}`) - if (currents.length !== 1) { - console.log(` ${currents.length} current migrations carry that name: decide by hand`) - continue - } - const current = currents[0]! - if (recorded.has(current.millis)) { - console.log( - ` superseded: the current ${current.name} is recorded on its own row, so this one is a leftover`, - ) - console.log(` DELETE FROM drizzle.__drizzle_migrations WHERE id = ${orphan.id};`) - } else if (current.hash === orphan.hash) { - // Same SQL under a new timestamp: the row only needs to point at the folder. - console.log(` renumbered as ${current.name} with identical SQL; point the row at it`) - console.log( - ` UPDATE drizzle.__drizzle_migrations SET created_at = ${current.millis} WHERE id = ${orphan.id};`, - ) - } else { - // The SQL changed after this version ran, so the current migration has - // statements this database never saw. No generated UPDATE: relabelling - // the row would record them as applied. - console.log( - ` renumbered as ${current.name} but the SQL differs; this database ran the OLD version`, - ) - console.log(` git diff ${historical.oid} HEAD:packages/db/drizzle/${current.name}/migration.sql`) - console.log(" apply whatever the current version adds by hand, then") - console.log( - ` UPDATE drizzle.__drizzle_migrations SET created_at = ${current.millis}, hash = '${current.hash}' WHERE id = ${orphan.id};`, - ) - } - } - return 1 -} - -const code = await preflight().finally(() => sql.end()) -process.exit(code) diff --git a/packages/db/scripts/planetscale-connection.ts b/packages/db/scripts/planetscale-connection.ts index f70eaf6048..5cf3b486d0 100644 --- a/packages/db/scripts/planetscale-connection.ts +++ b/packages/db/scripts/planetscale-connection.ts @@ -1,7 +1,7 @@ #!/usr/bin/env bun /** - * Shared PlanetScale-CLI credential broker for the schema-apply and data-migrate - * scripts. PlanetScale Postgres has no local proxy (unlike the MySQL + * Shared PlanetScale-CLI credential broker for the data scripts (restore drill, + * raw-SQL audit, backfills). PlanetScale Postgres has no local proxy (unlike the MySQL * `pscale connect` flow) — you connect directly with a minted credential over * TLS. So `withBranchConnection` mints an EPHEMERAL password for the target * branch via `pscale`, hands a direct (port 5432, sslmode=require) connection @@ -15,7 +15,6 @@ */ import { spawnSync } from "node:child_process" import * as Predicate from "effect/Predicate" -import postgres from "postgres" const FAILURE = 1 @@ -142,60 +141,6 @@ const deleteCredential = (database: string, branch: string, id: string): void => } } -/** - * Role names we are willing to interpolate into DDL as a quoted identifier. - * The value comes from the server (`SELECT current_user`), not from user input, - * but it is still validated rather than trusted. - */ -const ROLE_PATTERN = /^[A-Za-z_][A-Za-z0-9_$.-]*$/ - -/** - * Make the ephemeral migration role *run as* `postgres` for every session it - * opens, so DDL creates objects owned by `postgres`. - * - * This is what makes plain `drizzle-kit migrate` produce correctly-privileged - * tables with no follow-up grant pass. The branch carries default privileges - * keyed to `postgres` (`ALTER DEFAULT PRIVILEGES FOR ROLE postgres` — visible in - * `pg_default_acl` as `{=arwd/postgres}` on tables, `{=rwU/postgres}` on - * sequences), and those only fire for objects whose CREATING role is `postgres`. - * Without this, DDL runs as `migrate--`, the defaults never apply, - * and the new table lands with owner-only privileges. - * - * That gap is what caused the 2026-07-29 ingest outage. Ownership alone was - * never the issue — the role is revoked with `--successor postgres`, so tables - * end up postgres-owned anyway, and reassignment does not retroactively add - * ACLs. Three of prod's four `pscale_api_*` login roles are members of - * `postgres` with rolinherit, so they read a fresh table through inheritance; - * `pscale_api_rg068pnctlxw` — the ingest gateway, which connects via PSBouncer — - * is NOT a member and reads only through the PUBLIC grants the default - * privileges provide. Hence a new table broke the gateway alone while the API - * and alerting workers kept serving. - * - * `ALTER ROLE … SET role` (rather than a `SET ROLE` on one session) is what - * makes this hold for `drizzle-kit`, which opens its own connections: the - * setting is applied at login for every subsequent session of that role. It is - * permitted for a non-superuser altering itself because `role` is USERSET and - * the ephemeral role is created `--inherited-roles postgres`. - */ -const pinSessionRoleToPostgres = async (connectionUrl: string): Promise => { - const sql = postgres(connectionUrl, { max: 1, fetch_types: false }) - try { - // Ask the server: PlanetScale connection strings carry a routing suffix - // (`.`) that is stripped before the server sees it, so the - // URL username is not a usable role name. - const [row] = await sql`SELECT current_user` - const role: unknown = row?.current_user - if (!Predicate.isString(role) || !ROLE_PATTERN.test(role)) { - fail(`Could not determine the migration role to pin (got ${JSON.stringify(role)})`) - return - } - await sql.unsafe(`ALTER ROLE "${role}" SET role = 'postgres'`) - console.log(`✓ Migration role "${role}" pinned to run as postgres\n`) - } finally { - await sql.end() - } -} - /** * Run `fn` with a direct (port 5432, the cluster `postgres` database) * connection URL to the given branch, then revoke the ephemeral credential. @@ -218,7 +163,6 @@ export const withBranchConnection = async ( console.log(`::add-mask::${credential.password}`) console.log(`✓ Minted ephemeral credential for ${database}/${branch} (host ${host})\n`) try { - await pinSessionRoleToPostgres(credential.url) await fn(credential.url) } finally { console.log() diff --git a/packages/db/scripts/planetscale-migrations-preflight.ts b/packages/db/scripts/planetscale-migrations-preflight.ts deleted file mode 100644 index 516965c8cd..0000000000 --- a/packages/db/scripts/planetscale-migrations-preflight.ts +++ /dev/null @@ -1,21 +0,0 @@ -/** - * `migrations-preflight.ts` against a PlanetScale branch, over an ephemeral - * credential. Read-only. - * - * bun run --cwd packages/db ps:migrations-preflight main - */ -import { spawnSync } from "node:child_process" -import { resolve } from "node:path" -import { fail, withBranchConnection } from "./planetscale-connection" - -const branch = process.argv[2]?.trim() -if (!branch) fail("Usage: bun packages/db/scripts/planetscale-migrations-preflight.ts ") - -await withBranchConnection(branch as string, async (connectionUrl) => { - const proc = spawnSync("bun", ["run", "db:migrate:preflight"], { - cwd: resolve(import.meta.dir, ".."), - env: { ...process.env, DATABASE_URL: connectionUrl }, - stdio: "inherit", - }) - if (proc.status !== 0) fail("migrations preflight found rows the v1 migrator would refuse") -}) diff --git a/packages/db/scripts/reset-preview-branch.ts b/packages/db/scripts/reset-preview-branch.ts index dc47955f7a..7169b9d17b 100644 --- a/packages/db/scripts/reset-preview-branch.ts +++ b/packages/db/scripts/reset-preview-branch.ts @@ -60,7 +60,7 @@ const fail = (message: string): never => { /** * Role/publication names are interpolated as quoted identifiers (they cannot be * bind parameters), so whitelist a conservative charset — same rationale as - * ensure-privileges.ts. PlanetScale roles are dotted; `.` is literal inside + * PlanetScale roles are dotted; `.` is literal inside * double quotes. */ const IDENT_PATTERN = /^[A-Za-z_][A-Za-z0-9_$.-]*$/ From fe272e00a6714cf39f8015f5a643a5432bb801e8 Mon Sep 17 00:00:00 2001 From: Makisuo Date: Mon, 21 Sep 2026 21:55:55 +0200 Subject: [PATCH 2/4] docs(db): check the runtime roles for usage of postgres, not membership MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit pg_has_role(…, 'member') only says the role may SET ROLE; 'usage' is whether postgres's privileges apply without it, which is what a runtime role needs. --- CLAUDE.md | 4 ++-- docs/persistence.md | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 18fc7b7fc0..f15f03f272 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -183,8 +183,8 @@ database), reached from Workers via the Hyperdrive binding `MAPLE_DB`. - Migrations: `bun run --cwd packages/db db:generate`. **The prd deploy applies them**: the PlanetScale `main` branch is an alchemy `Planetscale.PostgresBranch` in `alchemy.run.ts` with `migrations` pointed at `packages/db/drizzle`; never run `drizzle-kit migrate` against prd. It - migrates as a temporary role dropped with `postgres` as successor, so every runtime role must be a - member of `postgres` (`pg_has_role(rolname, 'postgres', 'member')`) to read what it creates. PGlite + migrates as a temporary role dropped with `postgres` as successor, so every runtime role must + inherit `postgres` (`pg_has_role(rolname, 'postgres', 'usage')`) to read what it creates. PGlite applies them at layer build. - **PR preview deploys are label-gated** (2026-08, cost — re-enabled by `fd00bcd412`). A PR gets a preview only while it carries the `preview` label; `deploy-pr-preview.yml` triggers on diff --git a/docs/persistence.md b/docs/persistence.md index 5ab2f826d7..d2ed4d2f78 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -94,11 +94,11 @@ alerting Workers carry its name in their env so they upload after it. Bookkeepin `__alchemy_migrations`; `drizzle.__drizzle_migrations` was copied in once and is frozen, so never run `drizzle-kit migrate` against prd. The deploy migrates as a temporary role that is dropped with `postgres` as its successor, so the tables it creates end up owned by `postgres` with no other grants. -Every runtime role must therefore be a member of `postgres`: mint credentials with -`--inherited-roles postgres`, and this must return no rows: +Every runtime role must therefore inherit `postgres` (`USAGE`, not mere membership, which only +grants `SET ROLE`): mint credentials with `--inherited-roles postgres`, and this must return no rows: ```sql -SELECT rolname FROM pg_roles WHERE rolcanlogin AND NOT pg_has_role(rolname, 'postgres', 'member') +SELECT rolname FROM pg_roles WHERE rolcanlogin AND NOT pg_has_role(rolname, 'postgres', 'usage') ``` The deploy reads `PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN` / From 0d92a18a905c25dc267dc6e256301ee0bcab48ca Mon Sep 17 00:00:00 2001 From: Makisuo Date: Mon, 21 Sep 2026 22:25:46 +0200 Subject: [PATCH 3/4] docs(db): the role check lists pscale_api roles only, and a role without inheritance is replaced PlanetScale's own service roles never inherit postgres and never read Maple tables, and GRANT postgres is refused, so the fix for a runtime role is a rotation rather than a grant. --- docs/persistence.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/persistence.md b/docs/persistence.md index d2ed4d2f78..d6800e810d 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -95,10 +95,13 @@ alerting Workers carry its name in their env so they upload after it. Bookkeepin `drizzle-kit migrate` against prd. The deploy migrates as a temporary role that is dropped with `postgres` as its successor, so the tables it creates end up owned by `postgres` with no other grants. Every runtime role must therefore inherit `postgres` (`USAGE`, not mere membership, which only -grants `SET ROLE`): mint credentials with `--inherited-roles postgres`, and this must return no rows: +grants `SET ROLE`). Inheritance is fixed when PlanetScale creates the role and `GRANT postgres` is +refused, so a role without it is replaced: mint the new one with `--inherited-roles postgres`, rotate +the consumer's URL, then delete the old role. This must list no runtime credential (a personal dev +credential may appear): ```sql -SELECT rolname FROM pg_roles WHERE rolcanlogin AND NOT pg_has_role(rolname, 'postgres', 'usage') +SELECT rolname FROM pg_roles WHERE rolname LIKE 'pscale\_api\_%' AND NOT pg_has_role(rolname, 'postgres', 'usage') ``` The deploy reads `PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN` / From b90a9413ef87569a83961d292ae33a1280d9c0fc Mon Sep 17 00:00:00 2001 From: Makisuo Date: Mon, 21 Sep 2026 22:28:33 +0200 Subject: [PATCH 4/4] test(ui): pin the clock in the partial-tail chart test The rows are anchored to Date.now() and the chart reads it again at render. When the hour rolls over between the two, the last bucket has closed and no path is dashed, which is how CI failed this at 20:00:00 UTC. --- .../charts/line/__tests__/partial-tail.test.tsx | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/ui/src/components/charts/line/__tests__/partial-tail.test.tsx b/packages/ui/src/components/charts/line/__tests__/partial-tail.test.tsx index c388c5a250..2a52923c29 100644 --- a/packages/ui/src/components/charts/line/__tests__/partial-tail.test.tsx +++ b/packages/ui/src/components/charts/line/__tests__/partial-tail.test.tsx @@ -1,5 +1,5 @@ import { cleanup, render } from "@testing-library/react" -import { afterEach, beforeAll, describe, expect, it, vi } from "vitest" +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest" import { QueryBuilderLineChart } from "../query-builder-line-chart" @@ -7,7 +7,13 @@ import { QueryBuilderLineChart } from "../query-builder-line-chart" // observer to exist and a non-zero box to draw into; PlotFrame degrades to the // SVG renderer here (no Canvas 2D context), which is what makes the marks // inspectable as real paths. +// +// The clock is pinned mid-hour: the rows below are anchored to `Date.now()` and +// the chart reads it again at render, so an hour rolling over between the two +// closes the last bucket and nothing is dashed (CI hit this at 20:00:00). beforeAll(() => { + vi.useFakeTimers({ toFake: ["Date"] }) + vi.setSystemTime(new Date("2026-09-21T12:30:00Z")) vi.stubGlobal( "ResizeObserver", class { @@ -30,6 +36,7 @@ beforeAll(() => { }) afterEach(cleanup) +afterAll(() => vi.useRealTimers()) /** * The dashboard shape: hourly buckets anchored to wall-clock now, with NO