[PM-42318] Add the collection reads behind PAM's access rules - #8243
[PM-42318] Add the collection reads behind PAM's access rules#8243Hinton wants to merge 6 commits into
Conversation
Bitwarden can already answer "which collections may this user manage": callers pull GetManyByUserIdAsync and filter on Manage, then fold in the members who manage everything by role rather than by assignment. BulkCollectionAuthorizationHandler and CollectionAuthorizationHandler both work that way. Nothing answers the inverse -- given a collection, who manages it. Anything that has to reach the people responsible for a collection, rather than check one person's access to one collection, has no read to call. Composing the existing one means fetching every member of the organization and asking the per-user question once each, which is the wrong shape for the question and scales with the size of the organization instead of the size of the collection's access list. Add GetManagingUserIdsAsync, which returns the distinct confirmed members who can Manage a collection: direct Manage assignments, Manage through a group, organization Owners and Admins when the organization allows admin access to all collection items, and Custom members holding EditAnyCollection. MSSQL resolves it in Collection_ReadManagingUserIds as a union of those sources; Entity Framework reads the Permissions column in memory, because the JSON accessors the stored procedure uses are not portable across the providers. The stored procedure is deliberately shaped like its neighbour CollectionCipher_ReadUserIdsByCollectionIds, which unions the same three sources to resolve access rather than management. This does leave the manage predicate stated in a fourth place. Centralizing it is worth doing, but not here: the three existing statements are all user-first and cannot answer this question without the per-member loop above. Covered by CollectionRepositoryGetManagingUserIdsTests, which runs each route to Manage -- and, for the roles, the case where the organization withholds it -- against both implementations.
The collection browser renders a "Controlled access" column, but a collection row had nothing to put in it: no collection response model carried the PAM association, so a client could only learn that a collection is governed by reading the organization's access rules itself. Providers cannot do that -- the access-rules endpoint requires organization membership, deliberately, because rules gate who may lease credentials out of an organization and that is not a provider's to read. Their column stays empty however the client is written. Surface the fact on the collection instead, as HasEnabledAccessRule on the responses that feed sync and the organization listing. The flag is derived rather than the stored association. Collection.AccessRuleId records which rule governs a collection, but a rule that is switched off gates nothing, so reporting the association alone would start badging collections whose rule is disabled. Each read path therefore joins AccessRule and reports whether it is enabled: MSSQL in Collection_ReadByUserId, Collection_ReadByIdWithPermissions and Collection_ReadSharedCollectionsByOrganizationIdWithPermissions, Entity Framework via an EXISTS subquery in UserCollectionDetailsQuery and CollectionAdminDetailsQuery (Collection has no AccessRule navigation property to traverse) carried through the GROUP BY key of all three repository reads. Collection_ReadByUserId needed its columns qualified: joining AccessRule makes Id, OrganizationId and Name ambiguous against the UserCollectionDetails table function. Nothing writes the column and no request model gains a field -- SetAccessRuleAssociationsAsync remains the single writer of the association. Covered by CollectionRepositoryHasEnabledAccessRuleTests, which runs the same truth table -- ungoverned, governed by an enabled rule, governed by a disabled rule -- against every read path on both implementations. (cherry picked from commit e5cd839)
63f8802 to
567860e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8243 +/- ##
==========================================
+ Coverage 63.29% 68.91% +5.61%
==========================================
Files 2401 2410 +9
Lines 104043 104516 +473
Branches 9426 9457 +31
==========================================
+ Hits 65857 72027 +6170
+ Misses 35930 30112 -5818
- Partials 2256 2377 +121 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collection_ReadManagingUserIds and the three HasEnabledAccessRule read paths go out together, so they are one deployment step rather than two dated scripts. Re-dated to the day the pair lands. Also drops the note above Collection_ReadByUserId's SELECT explaining why its columns carry the UCD prefix.
🤖 Bitwarden Claude Code ReviewOverall Assessment: APPROVE Reviewed two additive collection reads: a new Code Review DetailsNo findings. Notes from validation that did not rise to findings:
|
r-tome
left a comment
There was a problem hiding this comment.
The EF query needs optimization. If an organization has for example 10k confirmed users then it has to load all of them in memory to filter them and would even have data it doesn't need -- like the Permissions column that is a NVARCHAR(MAX).
| && ou.Status == OrganizationUserStatusType.Confirmed | ||
| && ou.UserId != null) | ||
| .Select(ou => new { ou.Id, ou.UserId, ou.Type, ou.Permissions }) | ||
| .ToListAsync(); |
There was a problem hiding this comment.
This is loading all confirmed members from the database and then filters everything in memory. This could be more efficient we filtered it all in the database. We can use IQueryable to build the queries. For example:
var confirmed = dbContext.OrganizationUsers
.Where(ou => ou.OrganizationId == organizationId
&& ou.Status == OrganizationUserStatusType.Confirmed
&& ou.UserId != null);
var userIdsWithDirectManage = confirmed
.Where(ou => dbContext.CollectionUsers.Any(cu =>
cu.CollectionId == collectionId && cu.Manage && cu.OrganizationUserId == ou.Id))
.Select(ou => ou.UserId!.Value);
var userIdsWithGroupManage = confirmed
.Where(ou => dbContext.GroupUsers.Any(gu => gu.OrganizationUserId == ou.Id
&& dbContext.CollectionGroups.Any(cg =>
cg.CollectionId == collectionId && cg.Manage && cg.GroupId == gu.GroupId)))
.Select(ou => ou.UserId!.Value);
var managerUserIds = await userIdsWithDirectManage.Union(userIdsWithGroupManage).ToListAsync();
| .Where(ou => ou.OrganizationId == organizationId | ||
| && ou.Status == OrganizationUserStatusType.Confirmed | ||
| && ou.UserId != null) | ||
| .Select(ou => new { ou.Id, ou.UserId, ou.Type, ou.Permissions }) |
There was a problem hiding this comment.
Permissions are being selected for all users but only Custom users need them.
Using the IQueryable example:
var customUsers = await confirmed
.Where(ou => ou.Type == OrganizationUserType.Custom)
.Select(ou => new { ou.UserId, ou.Permissions })
.ToListAsync();
There was a problem hiding this comment.
Seems EF 10 gives us json querying capability directly in the db layer. But I guess until then we're stuck with looping over all custom.
GetManagingUserIdsAsync loaded every confirmed member of the organization -- including the Permissions column, an NVARCHAR(MAX) -- and then resolved the four Manage routes in memory against a dictionary. Cost scaled with the size of the organization rather than with the collection's access list, which is the shape the read was added to avoid, and the blob came back for members that could never need it. Each route is a query now, unioned in the database, in the shape CollectionCipherRepository.GetUserIdsByCollectionIdsAsync already uses to resolve access from the same three assignment sources. AllowAdminAccessToAllCollectionItems becomes a join predicate rather than a separate read and a branch in C#. Five round trips become two, and neither scales with member count. Permissions is still parsed in memory, but only for Custom members, who are the only ones EditAnyCollection can apply to. It stays in memory because EF maps the column as a plain string: nothing in the model says it holds JSON, so there is nothing to translate. Querying it would mean mapping the column as an owned or complex type, which hands EF's serializer the format contract that Collection_ReadManagingUserIds reads with JSON_VALUE(..., '$.editAnyCollection') and CoreHelpers.ClassToJsonData writes -- a serializer change on either side would silently break the procedure. There is no portable alternative: the SQL Server provider exposes no JSON function at all, Npgsql and Pomelo expose their own, and the issue tracking a provider-neutral one (dotnet/efcore#29306) is still open. The one place the EF layer does filter a JSON blob in the database, OrganizationReportRepository.ReadLatestByOrganizationIdAsync, does it with a substring match under a comment warning that it fails silently. Cover the two mechanisms this changes: the confirmed-status filter, which moved from a dictionary lookup into the joins, and deduplication, which moved from a HashSet to Union and Distinct. Both cases run against the procedure and the EF path, so they also hold the two implementations to the same answer.
The [HasEnabledAccessRule] alias already says what the column holds. The rest of the note argued that MAX() is safe over a group keyed by [AccessRuleId], which is a case made to a reader of the diff rather than to a reader of the procedure. Same species as the note above Collection_ReadByUserId's SELECT dropped in "Ship the collection reads in one dated migration", and removed from the same three procedures plus their copies in the dated script.
The script was dated 2026-08-21 while the newest script on main is 2026-08-14_04. Anything that merges before this PR does will claim a later date, leaving this one to sort ahead of scripts that were already applied, and 2026-08-2x_00 is a slot another PR could just as easily take. Renamed to 2026-08-25_00, unused on main and on pam/uat. Contents are unchanged and nothing refers to the filename -- Migrator embeds DbScripts\**\*.sql by wildcard.
r-tome
left a comment
There was a problem hiding this comment.
Looks good! Nice work. Thanks for making the changes I requested.
🎟️ Tracking
https://bitwarden.atlassian.net/browse/PM-42318
📔 Objective
Two independent reads, one per commit.
1. Who manages a collection
Bitwarden can already answer which collections may this user manage: callers pull
GetManyByUserIdAsyncand filter onManage, then fold in the members who manage everything by role rather than by assignment.BulkCollectionAuthorizationHandler.CanManageCollectionsAsyncandCollectionAuthorizationHandlerboth work that way.Nothing answers the inverse — given a collection, who manages it. Anything that has to reach the people responsible for a collection, rather than check one person's access to one collection, has no read to call. Composing the existing one means loading every member of the organization and asking the per-user question once each: the wrong shape for the question, and it scales with the organization instead of with the collection's access list.
ICollectionRepository.GetManagingUserIdsAsyncreturns the distinct confirmed members who can Manage, by any of the four routes:CollectionUser.ManageCollectionGroup.Manage+GroupUserAllowAdminAccessToAllCollectionItemsEditAnyCollectionMSSQL resolves it in
Collection_ReadManagingUserIdsas a union of those sources; EF reads thePermissionscolumn in memory, because the JSON accessors the procedure uses are not portable across providers. The procedure is deliberately shaped like its neighbourCollectionCipher_ReadUserIdsByCollectionIds, which unions the same three assignment sources to resolve access rather than management.2. Which collections an access rule governs
The collection browser renders a Controlled access column, but a collection row has nothing to put in it. No collection response model carried the association, so a client could only learn that a collection is governed by reading the organization's access rules itself.
A member can do that. A provider cannot:
GET organizations/{orgId}/access-rulesrequiresMemberRequirement, deliberately — providers manage an organization's billing and configuration, but access rules gate who can lease credentials out of it, which is not theirs to read. Providers do browse the client organization's collection list, so their column renders and every cell stays empty however the client is written.HasEnabledAccessRulesurfaces the fact on the collection instead, on the responses that feed sync and the organization listing.It is derived, not the stored association.
Collection.AccessRuleIdrecords which rule governs a collection, but a rule that is switched off gates nothing — returning the association alone would start badging collections whose rule is disabled. Each read path joinsAccessRuleand reports whether it is enabled: MSSQL inCollection_ReadByUserId,Collection_ReadByIdWithPermissionsandCollection_ReadSharedCollectionsByOrganizationIdWithPermissions; EF through anEXISTSsubquery inUserCollectionDetailsQueryandCollectionAdminDetailsQuery, sinceCollectionhas noAccessRulenavigation to traverse.A boolean rather than the rule id, deliberately: the id is useless to a client, and it avoids handing providers the rule identity the access-rules endpoint withholds from them. Nothing writes the column and no request model gains a field —
SetAccessRuleAssociationsAsyncremains the single writer of the association.Testing
CollectionRepositoryGetManagingUserIdsTestsandCollectionRepositoryHasEnabledAccessRuleTests, plus every other~CollectionRepositoryintegration test: 153 passing on SQL Server, SQLite and Postgres (204 with the MySQL/MariaDB rows skipped — no local instance).Api.Test187 andCore.Test219 on~Collection.All three providers earn their place here:
EXISTSsubquery in translated SQL — SQLite takes the client-side.GroupBybranch instead.CreateTestUserAsyncnames a user{identifier}-{guid}, so an identifier over 13 characters overflowsUser.Name. SQL Server passes the name to a procedure parameter declaredNVARCHAR(50)and truncates silently on assignment rather than raising, and SQLite does not enforce declared lengths at all, so the test passed on both while inserting a name it had quietly cut short.Reviewer notes
mainyet. The first is called by a notifier onpam/uatthat pushes to a collection's approvers; the second is paired with a clients change. Both are the PAM-free halves, split out so the collection layer reviews separately from PAM. The same situation as theAccessRuletables and procedures already onmain.BulkCollectionAuthorizationHandler.CanManageCollectionsAsync,CollectionAuthorizationHandler, and (onpam/uat)ApproverCollectionAccessQuery. Centralizing it is worth doing, but not here: all three existing statements are user-first and cannot answer the collection-first question without the per-member loop above.CollectionCollectionAccessDetailsResponseModelconstructors — create/update responses and the provider fallbacks — cannot computeHasEnabledAccessRuleand leave it false. Documented on the property; no vault list reads those responses.Collection_ReadByUserIdneeded its columns qualified: joiningAccessRulemakesId,OrganizationIdandNameambiguous against theUserCollectionDetailstable function.Collection_ReadManagingUserIds's migration adds aCREATE OR ALTER PROCEDUREand no table or column DDL, so it carries no existence guard. Neither migration changes the EF model, so there is no snapshot diff.