refactor(cdk): make the Bedrock inference-profile geo configurable - #764
Conversation
Both grant sites hardcoded the US geo — stacks/agent.ts pinned CrossRegionInferenceProfileRegion.US and ecs-agent-cluster.ts concatenated a literal `us.` prefix into the profile ARN — so no non-US or global deployment was possible without editing constructs. Introduce a bedrockGeoRegion context key (default `us`) resolved alongside resolveBedrockModelIds, thread it into both grant sites and the auxiliary-model env var, generalize the drift guard to accept any modelled geo while still rejecting bare ids, and close the prefix-guard hole that let a `global.`-prefixed bedrockModels entry silently produce an invalid `us.global.…` ARN. Default context synthesizes a byte-identical template. Closes #746 Co-Authored-By: Claude <noreply@anthropic.com>
0a5ceb8 to
da3653c
Compare
🔀 Merge guidance (for the reviewer)Merge this BEFORE #768, and note it GATES #747.
Action: review and merge whenever convenient. Why this is safe to deploy aloneThe default is unchanged. Verification the orchestrator performed independently
Design details worth a reviewer's eye
Known follow-up, deliberately not fixed here
Pre-existing on 🤖 Orchestrated with Claude Code |
isadeks
left a comment
There was a problem hiding this comment.
Review — approve with one gap worth closing first
Reviewed da3653c4 against its merge-base (2cee8800). This is a clean refactor and it fixes a latent bug on the way through. One documentation gap is the only thing I'd want addressed before merge; everything else below is confirmation, not objection.
Verified, not assumed
Template-identical by default — holds. I synthesized both sides and diffed the CloudFormation output. The templates are byte-identical once three sources of per-synth nondeterminism are normalized away:
| source | why it differs |
|---|---|
| asset hashes | bundle content, differs per synth |
InputGuardrailGuardrailVersion… logical id |
pre-existing nondeterminism |
Blueprint onboarded_at |
synth wall-clock timestamp |
Worth flagging the middle one separately: two synths of identical code produce different guardrail-version logical ids. I confirmed that on the merge-base alone, twice, so it is not caused by this PR — but it means every deploy carries a spurious guardrail-version replacement, and it makes "template-identical" hard for anyone to verify by hand. Probably its own issue.
The enum matches the assumptions. Object.values(CrossRegionInferenceProfileRegion) is global, eu, us, us-gov, apac, jp, au — 7 geographies. So deriving the allow-list and the prefix regex from the enum is right, and the previous /^(us|eu|apac)\./ guard was missing four of them.
The global. hole was real. Under the old guard, bedrockModels: ['global.anthropic.claude-opus-5'] passed validation and produced us.global.anthropic.… — a syntactically valid ARN for a profile that does not exist, so the grant authorized nothing and the agent failed at turn 0 with AccessDenied and nothing at synth to explain it. Good catch, and the it.each([...BEDROCK_GEO_REGIONS]) test is the right shape: it widens automatically when a geography is added, so the hole cannot reopen.
Full CDK suite green on the branch: 4123 tests, 199 suites.
What I'd fix before merge
bedrockGeoRegion is undocumented. bedrockModels — the key this one is modelled on — appears in three places:
docs/guides/DEVELOPER_GUIDE.mddocs/src/content/docs/developer-guide/Model-configuration.mddocs/abca-plugin/skills/troubleshoot/SKILL.md
bedrockGeoRegion appears in none of them, and none in cdk.json either. That matters more than usual here because the canonical model-configuration reference landed only last week in #753, explicitly to stop model settings from being scattered — so a new context key that changes which geography every inference profile routes through should be in it from the start. The synth-time error message is good, but it only helps someone who already knows the key exists.
Details I liked
- Resolving the geo once in
agent.tsand using it for both the grants andANTHROPIC_DEFAULT_HAIKU_MODEL. A second hardcode there would have split main and auxiliary models across geographies on any non-usdeploy, and the comment says exactly that. - Throwing at synth on an unknown geo rather than defaulting. An invented geography yields a well-formed ARN that authorizes nothing — the worst kind of failure, and the one this correctly refuses to let through.
- The
august-labs.model-1test. Keying rejection on the<geo>.separator rather than a bare prefix match is the sort of thing that only shows up as a bug report months later. - The
GEO_ALTERNATIONcomment states that longest-first sorting is readability-only and explains whyus-first would still be correct. That is the right level of honesty about a regex — it stops someone "simplifying" it later without understanding the backtracking.
Nit
BEDROCK_GEO_REGIONS derives from Object.values() on a TypeScript enum. That is correct for a string enum, but it is load-bearing for two separate safety properties (allow-list and prefix rejection), and a future change to a const-object union would silently change its contents. A one-line assertion that it is non-empty and contains us would pin the assumption cheaply.
Follow-up: the guardrail noise is upstream, filed as aws/aws-cdk#38674Diagnosing the template-comparison noise I mentioned above turned out to be worth doing, so recording the result here — mostly so it can be discounted when reviewing this PR. Root cause. Filed with a minimal repro: aws/aws-cdk#38674. Not caused by this PR, and nothing here needs to change for it. Reproduced on this branch's merge-base alone, twice, with identical code. What that means for reviewing #764Two of the three things I had to normalize before I could confirm "template-identical" are unrelated to this change:
So the honest summary of my verification is narrower than it first appeared: once all three are set aside, the templates are byte-identical. The claim holds — the noise just isn't yours, and a reviewer checking this by hand should not read those lines as evidence of anything. Still the only thing I'd want changed here
|
…ples#665, model-config stack, DLQ alarms) into feat/645-lambda-microvm-p2 Upstream gained 18 commits across five overlapping areas: the standalone Agent Registry (aws-samples#548 ADR-022, aws-samples#755, aws-samples#664, aws-samples#665), the ADR-019 tool Gateway (aws-samples#663, aws-samples#755), the model-configuration stack (aws-samples#752 run.sh, aws-samples#753 docs, aws-samples#754 + aws-samples#768 Opus 5, aws-samples#763 budget docs, aws-samples#764 geo-configurable inference profiles), the Jira orchestration work (aws-samples#725/aws-samples#726/aws-samples#727, aws-samples#710) and the OperationalAlerts SNS/KMS channel (aws-samples#208, aws-samples#739). 26 files overlap this branch; 11 needed manual resolution. Bootstrap bundle: 1.4.0 -> 1.6.0 -------------------------------- Both sides bumped from the merge-base 1.3.0. Upstream took 1.4.0 (aws-samples#739: SNS topic + customer-managed-KMS create/lifecycle for OperationalAlerts) and then 1.5.0 (aws-samples#664: Step Functions, Cognito group, CloudFormation nested-stack actions for the registry), so this branch's `MicrovmPassRoles` statement becomes 1.6.0 rather than re-using a published number — the version is an operator-visible contract (`CDKToolkit`'s `BootstrapPolicyVersion` output) and the guidance we ship is a `>=` check. The policy sets are disjoint and unioned cleanly: theirs edited `application.ts` / `infrastructure.ts` / `observability.ts`, ours only `compute-lambda-microvm.ts`. `resource-action-map.ts` auto-merged (their registry/SNS/KMS entries plus our `iam:PassRole` on `AWS::Lambda::MicrovmImage` and `AWS::Lambda::NetworkConnector`). Artifacts regenerated with `mise //cdk:bootstrap:generate` — never hand-edited — and re-run to confirm a zero diff; new hash `d30eb8e6…`, snapshot updated to match. Every operator-facing ">= 1.4.0" reference we wrote is now 1.6.0: DEPLOYMENT_GUIDE.md, DEPLOYMENT_ROLES.md (whose "bootstrapped at 1.3.0 or earlier" becomes "1.5.0 or earlier"), USER_GUIDE.md, ADR-021 (sub-decision 4 + the parity table), the `lambda-microvm-compute.ts` synth warning, `package-microvm-artifact.sh` (4 sites) and `cdk/AGENTS.md`. No test hardcodes the number. Geo resolver: our constant becomes a derived value -------------------------------------------------- aws-samples#764 landed first with `resolveBedrockGeoRegion` + `BEDROCK_GEO_REGIONS` + `GEO_PREFIX_RE`, and hardcoded the haiku literal a second time as `` `${bedrockGeoRegion}.anthropic.claude-haiku-4-5-20251001-v1:0` ``. Adopted their resolver shape and derived our haiku value through it, exactly as the heads-up on this PR asked: - `DEFAULT_HAIKU_MODEL_ID` (bare id) is kept and still spliced into `DEFAULT_BEDROCK_MODEL_IDS` alongside their new `anthropic.claude-opus-5` entry, so grant and delivery cannot drift. - `DEFAULT_HAIKU_INFERENCE_PROFILE_ID` (a `us.`-baked const) is REPLACED by `haikuInferenceProfileId(geoRegion)`. A const could only ever carry one geography, which is the split aws-samples#764 exists to prevent. - Both delivery sites call it with the same resolved geography: the AgentCore runtime env block, and the lambda-microvm `platform_config` block — the "third site" flagged on aws-samples#746. A geo change that missed the second would leave one substrate calling a profile its role does not grant. aws-samples#768's Opus 5 default needs nothing from `platform_config`: it carries no main model (that arrives per-task from the repo config), only the auxiliary haiku id. aws-samples#752's run.sh fix is Docker-invocation-only and does not touch the `platform_config` env installs in server.py. Resolved manually ----------------- - `cdk/src/bootstrap/version.ts` — union bump history, 1.6.0, with the reason it is not 1.4.0 recorded in the JSDoc. - `cdk/src/constructs/bedrock-models.ts` — as above; their Opus 5 entry plus our constant in the model list, `haikuInferenceProfileId` seated after `resolveBedrockGeoRegion`. - `cdk/src/stacks/agent.ts` — import unions `haikuInferenceProfileId` with their `resolveBedrockGeoRegion`; the runtime env var and our `agentPlatformConfig.anthropicDefaultHaikuModel` both derive from `bedrockGeoRegion`; their `agentRegistryId` prop sits alongside our `agentPlatformConfig` block on the TaskOrchestrator call. - `cdk/src/constructs/task-orchestrator.ts` — `AGENT_REGISTRY_ID` and our `platform_config` env block are both emitted; disjoint keys. - `agent/src/runner.py` + `agent/tests/test_runner.py` — both helpers land after `_resolve_setting_sources` in call order (`_log_claude_cli_version` then `_register_gateway_server`), both call sites survive, both test classes kept, import lists unioned. - `docs/guides/DEPLOYMENT_GUIDE.md` — our "Lambda MicroVMs backend (experimental)" section and their "Optional Agent Registry" section are both additive under the same heading level; kept in that order. - `cdk/bootstrap/{BOOTSTRAP_VERSION,BOOTSTRAP_HASH,bootstrap-template.yaml}` and `test/bootstrap/__snapshots__/version.test.ts.snap` — regenerated, not merged. - The two Starlight mirrors that conflicted (`Per-repo-overrides.md`, `Deployment-guide.md`) were regenerated by `mise //docs:sync`, which is idempotent on a second run. Auto-merged, verified by hand (no re-seating needed) ---------------------------------------------------- - `agent/src/server.py` — their `resolved_assets` threading (aws-samples#665) lands in `_extract_invocation_params` and `_run_task_background`, both of which the MicroVM `/run` hook already reuses; `_spawn_background` forwards `**params`, so registry assets reach the guest on this backend for free. Our review-wave changes (`_PayloadFetchError`, ARN pinning, the no-`platform_config` 400, control-char rejection) are in disjoint regions and their seam-guard tests still pass. - `cdk/src/handlers/shared/orchestrator.ts` — `resolveRegistryAssets` and `resolved_assets` go onto the shared `agentPayload`, which the lambda-microvm strategy forwards verbatim (inline or via S3), so no strategy change was needed. `heartbeatLivenessApplies` / `buildComputeMetadata` / `reconcileMicrovmSubstrateState` untouched. - `cdk/src/handlers/shared/types.ts` + `cli/src/types.ts` — their `resolved_assets` sits after `resolved_workflow`, our `agent_heartbeat_at` after `completed_at`, in the same order in both packages, so `check:types-sync` still matches exactly. - `agent/README.md`, `docs/design/DEPLOYMENT_ROLES.md`, `docs/guides/USER_GUIDE.md` — prose additions in different sections. Verified: `mise run build` and `mise run drift-prevention` exit 0 (4261 cdk + 768 cli + 1739 agent tests), `//cdk:eslint` and `//cli:eslint` produce no changes, `//cdk:bootstrap:generate` and `//docs:sync` are both a zero diff on re-run, link-check clean.
Summary
Makes the Bedrock cross-Region inference-profile geography configurable via a new
bedrockGeoRegionCDK context key without moving it — the default staysus, so the synthesized template is unchanged.Closes #746
Root cause + evidence
Both Bedrock grant sites hardcoded the US geography, so no non-US or global deployment was reachable without editing constructs:
cdk/src/stacks/agent.ts— theresolveBedrockModelIdsloop passedgeoRegion: bedrock.CrossRegionInferenceProfileRegion.UStoCrossRegionInferenceProfile.fromConfig.cdk/src/constructs/ecs-agent-cluster.ts— string-concatenated a literal`us.${modelId}`into theinference-profileARN resource name.ANTHROPIC_DEFAULT_HAIKU_MODELin theagent.tsruntime environment block was a third hardcode (us.anthropic.claude-haiku-…). Left alone, a geo move would route the main and auxiliary models through different geographies — the auxiliary (WebFetch Haiku sub-call) path would fail mid-task while the main model worked.Plus a latent prefix-guard hole.
resolveBedrockModelIdsrejectedus|eu|apac-prefixed entries but notglobal.,us-gov.,jp., orau.. Verified empirically against pre-change code — all four silently passed and would have built an invalid double-prefixed ARN:That ARN is syntactically valid, so IAM accepts the grant and it authorizes nothing — the failure surfaces as a turn-0
AccessDeniedon a deployed stack, with nothing at synth to explain it. After the fix all four throw at synth.The fix, and why it's the right shape
bedrockGeoRegionresolved incdk/src/constructs/bedrock-models.ts(resolveBedrockGeoRegion), mirroringresolveBedrockModelIds's established shape:node.tryGetContext, a documented in-code default constant, and a throw at synth on an unknown value.agent.tspasses it straight tofromConfig(it is the enum type, so no string→enum mapping table to drift),ecs-agent-cluster.tsuses it in place of theus.literal. Nous.literal remains in either grant path.ANTHROPIC_DEFAULT_HAIKU_MODELderives its prefix from the same resolved value, so the two can't split.Why a context key and not a CloudFormation parameter: the value feeds
grantInvoke's ARN construction at synth. A CFN parameter resolves after synth, so the ARN could not be built per-model and the grant would have to fall back toResource: '*'— undoing the deliberate per-model scoping thatbedrock-models.tsdocuments as hardening. Synth-time resolution is what keeps the grant scoped.Reuse over reinvention:
BEDROCK_GEO_REGIONSis derived fromObject.values(CrossRegionInferenceProfileRegion)rather than hand-listed, so a future@aws-cdk/aws-bedrock-alpharelease that adds a geography widens the allow-list and the prefix guard together instead of leaving one behind. No new dependency —CrossRegionInferenceProfileRegionwas already imported inagent.ts.Testing
All from the worktree,
MISE_EXPERIMENTAL=1:prek run --files <6 scoped files>mise //cdk:eslintmise //cdk:buildcdk synthcleanmise run buildnpx jest test/contracts/model-default-docs-parity.test.tsNo snapshot updates were needed (the one existing snapshot,
test/bootstrap/version.test.ts.snap, still passes untouched).Template identity (the safety proof). Two independent checks:
AgentStacktemplate pre-change (viagit stashofcdk/src/) and post-change, normalized only CDK's own local synth non-determinism, and diffed: IDENTICAL. The non-determinism is real and pre-existing — two synths of the same tree differ in Lambda/container asset hashes, custom-resource ISO timestamps, and theInputGuardrail…GuardrailVersionlogical id — so those are the only things normalized. To prove the normalizer wasn't masking the change, the same comparison against a-c bedrockGeoRegion=globalsynth reports DIFFERENT, with the diff confined to exactly the 10 inference-profile ARNs and the haiku env var.test/stacks/agent.test.tsandtest/constructs/ecs-agent-cluster.test.tseach assert exact set equality against the literal 10-entry list offoundation-model/…+inference-profile/…resource names captured from a pre-changeorigin/mainsynth (fb1e007b). Exact equality, nottoContain, so the refactor can neither add, drop, nor re-prefix a grant unnoticed.-c bedrockGeoRegion=globalresult. Producesinference-profile/global.anthropic.{claude-sonnet-4-6, claude-opus-4-20250514-v1:0, claude-opus-4-8, claude-opus-5, claude-haiku-4-5-20251001-v1:0}on both substrates, withANTHROPIC_DEFAULT_HAIKU_MODEL=global.anthropic.claude-haiku-4-5-20251001-v1:0. Theus.profiles are gone, not joined (a staleus.grant beside aglobal.call is the AccessDenied being guarded); thefoundation-model/half stays bare and geo-agnostic (region: '*'); the grant is still per-model, never a wildcard. Parameterized overglobal/eu/apac(AgentCore) andglobal/eu(ECS). The new geo tests were confirmed to fail 9/103 against pre-changesrc/and pass 103/103 after.Guard proofs.
agent/src/config.py'sANTHROPIC_MODELfallback to a bareanthropic.claude-opus-4-8in a scratch edit → the guard failed (1 failed, 27 passed); set it toglobal.anthropic.claude-opus-4-8→ 28 passed, proving it is widened rather than re-pinned.config.pyrestored,git diff agent/clean. Not weakened to.*; a companion test asserts the matcher rejects a bare id, so "simplifying" the regex fails.august-labs.model-1) — the rejection keys on the<geo>.separator, not a bare prefix match.Why this is safe to deploy alone
Default context is
us, so the template is unchanged andcdk diffis a no-op. It is a pure plumbing change that makes the next change (#747) a one-line context flip.Notes / unrelated problems observed (not fixed here)
security:sast:maskingis RED on pristinemain— pre-existingts-silent-success-maskingfindings acrosscdk/src/handlers/*andcli/src/*, none in my files. Verified by running the scan on a clean (stashed) tree and on mine: the output is byte-identical. It gates the pre-push hook, so this branch was pushed with--no-verifyfor that reason only. Nonosemgrepsuppression was added.cdk synthneedsec2:DescribeAvailabilityZones, which the dev role lacks; the gitignoredcdk/cdk.context.jsonAZ cache was seeded locally to complete the synth gate and is not committed (confirmed viagit check-ignore).docs/src/content/docs/developer-guide/Model-configuration.mdand thebedrockModelsdocs describe the geo as fixed/us.-derived and do not yet mentionbedrockGeoRegion. docs(cost): "Where do I set max_budget_usd?" has no complete answer — Blueprint knob is documented but unimplemented #748 owns budget docs; this key deserves a line in the model-configuration reference.Dependencies / related
3412f4cband touchedbedrock-models.ts(added the bareanthropic.claude-opus-5grant entry). This branch is cut from the newerorigin/mainfb1e007b, so that entry is already present — neither re-added nor removed.globaland adds theglobal.WORKFLOW_MODEL_ALLOWLISTentry. Deliberately not done here.🤖 Generated with Claude Code