feat: adapt migration schema calls to query-lib value objects by abnegate · Pull Request #222 · utopia-php/migration

abnegate · 2026-08-21T04:07:12Z

Adapts the migration destination's schema calls to the query-lib value objects.

Why this approach

Database::createCollection() takes a Collection only, so destination Appwrite now wraps the collection id, attributes and indexes — along with the named permissions and documentSecurity — in new Collection(...) instead of passing positional arrays.

utopia-php/database is pinned to dev-feat-query-lib as 2.0.0, re-pinned to that branch's head whenever it moves.

Lock repairs that came with the re-pin

Two problems surfaced only when the lock was resolved again rather than reused:

utopia-php/query was locked to dev-feat-schema-order, a branch that no longer exists on the remote. The resolution survived only as long as nobody resolved it. database's branch requires query 0.6.*, which is released, so the lock now takes the release.
utopia-php/storage moved 4.0.3 → 4.0.4, because 4.0.3 capped utopia-php/validators at ^0.4 while database requires ^0.5. 4.0.4 dropped the validators dependency outright. database has required ^0.5 on main as well as on the branch, so this was already true before the query-lib work and only surfaced now.

A live bug the re-pin exposed

The Appwrite destination passed the 'ASC' / 'DESC' strings a source hands back straight into Utopia\Database\Index, which takes Order cases and rejects anything else. The resulting InvalidArgumentException is not a Migration Exception, so instead of recording a failed index the whole transfer aborted.

This was already true before this PR and simply could not be seen: the lock held query at the deleted branch, whose Index took plain strings, so CI had never built an index against the contract the released library actually has. AppwriteIndexLengthsTest covers it — two of its cases pass ['ASC', 'ASC'], and both went red on the first run against the re-pinned lock and green with the fix.

It is the same defect as the one in appwrite/appwrite#11649's Databases worker, from the same cause.

Also on this branch: provisioning ownership and recovery

The branch has grown well past its title. Alongside the value-object adaptation it carries a provisioning-owner / lifecycle-fencing subsystem added 2026-08-29..31, which does not depend on query-lib at all:

Destinations/Appwrite/ProvisioningOwner.php and owner persistence in Destinations/Appwrite.php
fencing for concurrent provisioning attempts, and recovery of terminal migration failures
explicit CLI flags for provisioning recovery in bin/MigrationCLI.php
the tests that come with them: AppwriteDatabaseConcurrencyTest (+400), AppwriteDatabaseStatusTest (+1026), MigrationCLITest (+342), ProvisioningOwnerTest (+36)

Total diff against main is +3450/-714 over 23 files. Recommendation: split those commits into a PR against main so they can ship now, and leave only the Collection/Attribute/Index/Order adaptation and the lock re-pin here, where it is blocked on database#823 being tagged.

Chain

Landing order, bottom up:

utopia-php/database#823 — the query-lib migration itself
utopia-php/abuse#124, utopia-php/migration#222, utopia-php/monorepo#206 — the schema call sites in the libraries (audit moved to the monorepo, so feat: adapt audit schema to query-lib Attribute and Index VOs audit#133 is closed in favour of monorepo#206)
appwrite/appwrite#11649
appwrite-labs/cloud#5410

The three framework PRs once stacked on #823 — #947 (ORM), #948 (repositories and seeding), #949 (migration runner and schema differ) — are closed and are not part of this train.

Verified

CI green on this head
Greptile 5/5, no unresolved threads
Pint on the Collection wrap

Not verified

Open PR Support duplicate handling for Appwrite imports #205 (duplicate handling for Appwrite imports) edits the same Destinations/Appwrite.php regions; whichever lands second will need a merge.
The utopia-php/database dependency is still a branch pin. It becomes a released tag only once #823 merges, and this PR should not land before that.

Published 2.0 still used Database::VAR_* and positional createAttribute/createIndex, which feat-query-lib removed. Rebase onto main and pass Attribute/Index/Relationship VOs plus ColumnType/IndexType so Appwrite can pin this branch as 2.0.0.

Appwrite stores ColumnType::BigInteger as biginteger. CSV export resolved that as an unsupported column type and wrote no rows.

createDocument can return an empty Mongo sequence while a subsequent getDocument has the ObjectId. Creating database_{seq} from the create return left table import looking up a collection that did not exist.

Appwrite main added huggingface as a project OAuth2 provider. Without an allow-list entry, Appwrite-to-Appwrite migrations fail on that provider even when the rest of the transfer succeeded.

Keep query-lib APIs and the huggingface PROVIDERS allow-list already on main.

Appwrite #11649 locks database at 5719edd. Staying on e593b78 would only prove the schema VO calls against an older query-lib surface.

greptile-apps · 2026-08-21T04:09:24Z

Confidence Score: 5/5

The reviewed changes appear safe to merge, with no outstanding prior findings or newly established defects.

Summary

This update re-pins the query-lib database branch and adapts provisioning ownership writes to the dependency’s timestamp-based concurrency API.

Replaces removed version-guarded metadata updates with withRequestTimestamp()-scoped writes.
Updates test doubles and assertions for the new updateDocument() signature.
Adds normalization for query-lib’s biginteger column spelling.
Retains the broader Appwrite provisioning ownership and recovery lifecycle introduced by the PR.

_{Reviews (41) · Last reviewed commit: "chore: re-pin utopia-php/database to the..."}

utopia-php/database feat-query-lib keys silenced events with Coroutine::getCid(). The CI image is vanilla PHP, so Memory-adapter tests fatalled before they could run.

createDocument can persist a row whose subsequent getDocument is empty. That throw sat outside the failed-status handler, so a later skip could flip the unusable database to ready without a backing collection.

abnegate · 2026-08-21T04:16:19Z

Addressed the reload-failure finding.

createDocument can persist a _databases row whose immediately following getDocument is empty (the Mongo sequence miss this branch already reloads for). That throw sat outside the markDatabaseFailed catch, so the document stayed provisioning and a later spec-matching skip could flip it to ready with no backing collection.

Reload + createCollection now share that catch. testReloadFailureMarksTheDatabaseFailed fails without the wrap (provisioning) and passes with it (failed).

Also stubbed Swoole\Coroutine::getCid() in the PHPUnit bootstrap so the Memory-adapter suite can run on the CI image, which has no Swoole extension. utopia-php/database feat-query-lib keys silenced events with it.

@greptile-apps review

Asterisk wildcards on utopia-php packages are replaced with equivalent caret constraints so Composer ranges stay consistent.

Keep composer.json and composer.lock in sync so `composer validate` passes, and pin utopia-php/database to the current query-lib HEAD.

abnegate · 2026-08-21T05:24:49Z

@greptileai review

abnegate · 2026-08-21T06:00:14Z

@greptile-apps review

Force re-review of HEAD 7a3a60e. Description updated for factories and caret lock refresh.

Database::createCollection no longer accepts a string id.

Database::checkAttribute now requires Attribute. Build schema models from the resource key so metadata document IDs are not used as attribute keys.

Appwrite E2E migrations failed because checkAttribute now requires Attribute, and the destination still handed it a metadata Document.

A reload failure leaves a metadata document in `failed` with no backing collection. Recovery only ran when onDuplicate was not Fail, so the default policy retried createDocument against the existing ID and stranded the database.

abnegate · 2026-08-21T10:36:44Z

@greptileai review

Index types already used IndexType; column direction was still a raw ASC string. Collection constructors with multiple named params were also jammed on one line.

…atabase The lock held utopia-php/query at dev-feat-schema-order, a branch that no longer exists on the remote, so the resolution only survived as long as nobody resolved it again. database's branch requires query 0.6.*, which is released, so this takes the release. storage 4.0.4 comes along because 4.0.3 capped utopia-php/validators at ^0.4 while database requires ^0.5; 4.0.4 dropped the validators dependency outright. database has required ^0.5 on main as well as on the branch, so this was already true before the query-lib work and only surfaced now that the lock moved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both store a plain client id and secret, so only the client id is readable and migratable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit ea342db)

The lock still pinned 6cc5f8be, two reconciles behind: the branch's green CI was proving a database revision that will not ship. a137475 carries the merge with main and the batched upsert read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:06:46Z

@greptileai review

The guard read Appwrite.php and pinned source fragments, so a reformat broke it while a real regression could still pass. Transfer a column through the public entry point instead and record what the project database is handed: the metadata row beside it is keyed by the composite attribute id, so a column named after that id is measured against a table that never holds it. Verified red on both regression shapes. Passing the metadata document fails the type the collaborator requires; rebuilding the attribute from that document's array copy reaches checkAttribute with key 1_1_title where the transfer's column is title. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:27:15Z

@greptileai review

The guard reached resolveDestinationDsn by reflection, so it never ran the line deciding whether the creation path consults the resolver at all. Writing the source DSN straight into the row, which is the comuneo incident, left all three cases green. Transfer a database through the public entry point instead and read back _databases.database. Verified red on that shape: every case reports the source host database_db_fra1_self_hosted_11_0 where the destination must carry a blank or the resolver's own value. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:36:56Z

@greptileai review

Five of the ten assertions pinned English sentences from the help text, which a reword breaks without changing behaviour. The other five are interface tokens a user types, and one of them carries a property nothing else covers: the neighbouring test proves --recover-provisioning is refused, not that it stays unadvertised. Re-adding it to getHelp() fails this test and no other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:45:24Z

@greptileai review

database's feat-query-lib merged main (1ea2f35), which raised utopia-php/validators to ^1.0. The lock was still on a137475 with validators 0.6.0. storage 4.0.5 pins validators ^0.6, so it moves to 4.1.0 (the first release accepting ^1.0), bringing the minors it requires. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Picks up the read-replica write-hook sync and the schema-bound aggregate validators, so CI here proves this branch against the base the rest of the train now uses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Picks up the main merge on the database branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Picks up the required-change fix for attribute updates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The database adapter now calls alterColumnNullable(), which 0.6.0 does not have, so an attribute update that changes required fataled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The database library no longer carries a per-document version, so the compare-and-set on `expectedVersion` has nothing to compare. The ownership writes now run under withRequestTimestamp with the timestamp the row was read with: every update moves `$updatedAt` strictly forward, so a row another writer touched between the locked read and the write is refused as a conflict exactly as before, and the rows are still locked with forUpdate inside the claim transaction. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…head Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

abnegate added 6 commits August 14, 2026 16:34

fix: accept query-lib biginteger as API bigint

d32d196

Appwrite stores ColumnType::BigInteger as biginteger. CSV export resolved that as an unsupported column type and wrote no rows.

fix: create collection metadata from the persisted database sequence

47e2597

createDocument can return an empty Mongo sequence while a subsequent getDocument has the ObjectId. Creating database_{seq} from the create return left table import looking up a collection that did not exist.

(fix): allow migrating HuggingFace OAuth2 providers

cce7c66

Appwrite main added huggingface as a project OAuth2 provider. Without an allow-list entry, Appwrite-to-Appwrite migrations fail on that provider even when the rest of the transfer succeeded.

chore: merge origin/main into feat-query-lib

340e6c6

Keep query-lib APIs and the huggingface PROVIDERS allow-list already on main.

chore: pin utopia-php/database to Appwrite's feat-query-lib SHA

0144e33

Appwrite #11649 locks database at 5719edd. Staying on e593b78 would only prove the schema VO calls against an older query-lib surface.

abnegate added 2 commits August 21, 2026 16:16

test: stub Swoole Coroutine when the extension is missing

a11db20

utopia-php/database feat-query-lib keys silenced events with Coroutine::getCid(). The CI image is vanilla PHP, so Memory-adapter tests fatalled before they could run.

fix: mark databases failed when metadata reload misses

55b92c7

createDocument can persist a row whose subsequent getDocument is empty. That throw sat outside the failed-status handler, so a later skip could flip the unusable database to ready without a backing collection.

abnegate added 2 commits August 21, 2026 17:11

(chore): use caret ranges for Utopia dependencies

b67ace1

Asterisk wildcards on utopia-php packages are replaced with equivalent caret constraints so Composer ranges stay consistent.

(chore): refresh lockfile after caret Utopia constraints

7a3a60e

Keep composer.json and composer.lock in sync so `composer validate` passes, and pin utopia-php/database to the current query-lib HEAD.

This was referenced Aug 21, 2026

Use query lib utopia-php/database#823

Open

feat: migrate to utopia-php/database query-lib appwrite/appwrite#11649

Open

abnegate added 3 commits August 21, 2026 18:52

(feat): pass Collection to createCollection

f2de906

Database::createCollection no longer accepts a string id.

(fix): pass Attribute models to checkAttribute

3fccb0c

Database::checkAttribute now requires Attribute. Build schema models from the resource key so metadata document IDs are not used as attribute keys.

(fix): pass Attribute models into checkAttribute

3a5fb77

Appwrite E2E migrations failed because checkAttribute now requires Attribute, and the destination still handed it a metadata Document.

greptile-apps Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php

abnegate and others added 2 commits August 21, 2026 22:53

(refactor): use Schema\Order and wrap Collection constructors

04ab893

Index types already used IndexType; column direction was still a raw ASC string. Collection constructors with multiple named params were also jammed on one line.

greptile-apps Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php

(fix): fence database migration finalization

ec7db1f

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread bin/MigrationCLI.php

abnegate added 3 commits September 1, 2026 11:06

(fix): reject empty guarded database writes

97863f3

(fix): finalize standalone migrations

e56b964

(chore): repin guarded database writes

60b7a52

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread bin/MigrationCLI.php Outdated

abnegate and others added 3 commits September 1, 2026 11:39

(fix): finalize partial standalone migrations

1724bf3

(chore): verify migration against the final query database

8ab3017

Add TikTok and Kakao OAuth2 providers to the migration allow-list

e74e266

Both store a plain client id and secret, so only the client id is readable and migratable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit ea342db)

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread tests/Migration/Unit/Destinations/AppwriteCheckAttributeTest.php Outdated

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread tests/Migration/Unit/MigrationCLITest.php Outdated

abnegate and others added 7 commits September 11, 2026 04:02

chore: re-pin utopia-php/database to the feat-query-lib head

1b00603

Picks up the read-replica write-hook sync and the schema-bound aggregate validators, so CI here proves this branch against the base the rest of the train now uses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into qlt-train

2d37a36

chore: re-pin utopia-php/database to the feat-query-lib head

05480de

Picks up the main merge on the database branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

chore: re-pin utopia-php/database to the feat-query-lib head

bcd0a0d

Picks up the required-change fix for attribute updates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

chore: bump utopia-php/query to 0.6.1

9a4b263

The database adapter now calls alterColumnNullable(), which 0.6.0 does not have, so an attribute update that changes required fataled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

chore: re-pin utopia-php/database to the feat-query-lib head

766b422

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

greptile-apps Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread tests/Migration/Unit/Destinations/AppwriteCheckAttributeTest.php

abnegate and others added 3 commits September 21, 2026 23:22

chore: re-pin utopia-php/database to the feat-query-lib head

8aff737

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

chore: re-pin utopia-php/database to the version-less feat-query-lib …

00eb902

…head Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Navigation Menu

feat: adapt migration schema calls to query-lib value objects - #222

feat: adapt migration schema calls to query-lib value objects#222
abnegate wants to merge 50 commits into
mainfrom
feat-query-lib

abnegate commented Aug 21, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Aug 21, 2026 •

edited

Loading

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

abnegate commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why this approach

Lock repairs that came with the re-pin

A live bug the re-pin exposed

Also on this branch: provisioning ownership and recovery

Chain

Verified

Not verified

Uh oh!

greptile-apps Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

abnegate commented Aug 21, 2026 •

edited

Loading

greptile-apps Bot commented Aug 21, 2026 •

edited

Loading