From 6acc48c2323114725448988cae67f6f4c12f5e53 Mon Sep 17 00:00:00 2001 From: PaulieB14 <94752445+PaulieB14@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:34:29 -0400 Subject: [PATCH 1/4] fix: track indexing issuance through the IssuanceAllocator (GIP-0089) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `GraphNetwork.networkGRTIssuancePerBlock` has read 120.73 GRT/block on Arbitrum since GIP-0089 activated on 2026-09-01, when the rate the RewardsManager actually issues at became 96.584. Every consumer computing indexing-reward APR from this field — the schema documents it for exactly that — currently overstates by 120.73/96.584 = 1.25x. Two independent causes, either sufficient on its own: 1. WRONG GETTER. rewardsManager.ts read `issuancePerBlock()`. Once the IssuanceAllocator was wired up (GIP-0076/GIP-0088) the RewardsManager mints only its own allocation, and that legacy storage slot stopped being the rate it issues at. On Arbitrum One today the slot still returns 120.73e18 while `getAllocatedIssuancePerBlock()` returns 96.584e18. The getter was also absent from abis/RewardsManagerStitched.json, so the mapping could not have called it. 2. NO TRIGGER. The value is only refreshed by the RewardsManager's `ParameterUpdated('issuancePerBlock')`. GIP-0089 changed the split on the ALLOCATOR, so no such event was emitted and the field was never re-read. Reproducible — the same block-scoped query either side of the change: { before: graphNetwork(id:"1", block:{number:500600000}) { networkGRTIssuancePerBlock } after: graphNetwork(id:"1", block:{number:501148942}) { networkGRTIssuancePerBlock } } Both return 120730000000000000000. GIP-0089 took effect at Arbitrum block 500,701,945, between them. On chain at the same moment: RewardsManager 0x971B9d3d0Ae3ECa029CAB5eA1fB0F72c85e6a525 getAllocatedIssuancePerBlock() 0xe208d721 -> 96.584e18 issuancePerBlock() 0x6c080f18 -> 120.730e18 IssuanceAllocator 0xb64f29b2d81140ffc3a135e319561a1bd03b1a7e getTargetAllocation(RewardsManager).selfMintingRate -> 96.584e18 CHANGES - New IssuanceAllocator data source handling `TargetAllocationUpdated(address,uint256,uint256)`. When the target is the RewardsManager, `newSelfMintingRate` is the rate it will issue at, so the event carries the value and no contract call is needed. This is the only place a future re-split is observable: GIP-0088 Phase 3 moves a further 6 GRT/block to the Recurring Agreement Manager, taking the RewardsManager to 90.584, and would otherwise go unnoticed in the same way. - rewardsManager.ts prefers `getAllocatedIssuancePerBlock()`, via try_ so it still works on deployments predating the upgrade where the getter is absent. - abis/RewardsManagerStitched.json gains the two getters it was missing. - The allocator address is resolved from @graphprotocol/address-book, which required bumping 1.1.0 -> ^1.3.0: 1.1.0 neither ships issuance/addresses.json nor exports the path, while 1.3.0 exports "./*/addresses.json". It is deployed on Arbitrum One and Arbitrum Sepolia only, and resolves empty elsewhere, matching how subgraphService is handled. VERIFIED `yarn prepare:arbitrum` and `yarn prepare:arbitrum-sepolia` both resolve the allocator (0xb64f29b2… and 0x76a0d756… respectively) and `yarn build` compiles. The CI test-prep step (testAddressesL1/L2 + mustache) still produces a valid config/addresses.ts on both layers. Note for reviewers: picking up the 2026-09-01 allocation requires reindexing from a start block at or before it, since the handler is event-driven. Whether that warrants a resync of the deployed subgraphs is a maintainer call. --- abis/IssuanceAllocator.json | 46 ++++++++++++++++++++++++++ abis/RewardsManagerStitched.json | 28 +++++++++++++++- config/addresses.template.ts | 2 ++ config/arbitrumSepoliaAddressScript.ts | 9 ++++- config/localNetworkAddressScript.ts | 1 + config/mainnetArbitrumAddressScript.ts | 9 ++++- config/testAddressesL1.ts | 1 + config/testAddressesL2.ts | 1 + package.json | 2 +- src/mappings/issuanceAllocator.ts | 34 +++++++++++++++++++ src/mappings/rewardsManager.ts | 11 +++++- subgraph.template.yaml | 25 ++++++++++++++ yarn.lock | 8 ++--- 13 files changed, 168 insertions(+), 9 deletions(-) create mode 100644 abis/IssuanceAllocator.json create mode 100644 src/mappings/issuanceAllocator.ts diff --git a/abis/IssuanceAllocator.json b/abis/IssuanceAllocator.json new file mode 100644 index 00000000..40b5b5fa --- /dev/null +++ b/abis/IssuanceAllocator.json @@ -0,0 +1,46 @@ +[ + { + "anonymous": false, + "inputs": [ + { "indexed": true, "internalType": "address", "name": "target", "type": "address" }, + { "indexed": false, "internalType": "uint256", "name": "newAllocatorMintingRate", "type": "uint256" }, + { "indexed": false, "internalType": "uint256", "name": "newSelfMintingRate", "type": "uint256" } + ], + "name": "TargetAllocationUpdated", + "type": "event" + }, + { + "anonymous": false, + "inputs": [ + { "indexed": false, "internalType": "uint256", "name": "oldIssuancePerBlock", "type": "uint256" }, + { "indexed": false, "internalType": "uint256", "name": "newIssuancePerBlock", "type": "uint256" } + ], + "name": "IssuancePerBlockUpdated", + "type": "event" + }, + { + "inputs": [], + "name": "getIssuancePerBlock", + "outputs": [{ "internalType": "uint256", "name": "", "type": "uint256" }], + "stateMutability": "view", + "type": "function" + }, + { + "inputs": [{ "internalType": "address", "name": "target", "type": "address" }], + "name": "getTargetAllocation", + "outputs": [ + { + "components": [ + { "internalType": "uint256", "name": "totalAllocationRate", "type": "uint256" }, + { "internalType": "uint256", "name": "allocatorMintingRate", "type": "uint256" }, + { "internalType": "uint256", "name": "selfMintingRate", "type": "uint256" } + ], + "internalType": "struct IIssuanceAllocationStatus.Allocation", + "name": "", + "type": "tuple" + } + ], + "stateMutability": "view", + "type": "function" + } +] diff --git a/abis/RewardsManagerStitched.json b/abis/RewardsManagerStitched.json index 4c58dd58..3efbe0ce 100644 --- a/abis/RewardsManagerStitched.json +++ b/abis/RewardsManagerStitched.json @@ -603,5 +603,31 @@ ], "stateMutability": "nonpayable", "type": "function" + }, + { + "inputs": [], + "name": "getAllocatedIssuancePerBlock", + "outputs": [ + { + "internalType": "uint256", + "name": "", + "type": "uint256" + } + ], + "stateMutability": "view", + "type": "function" + }, + { + "inputs": [], + "name": "getIssuanceAllocator", + "outputs": [ + { + "internalType": "address", + "name": "", + "type": "address" + } + ], + "stateMutability": "view", + "type": "function" } -] +] \ No newline at end of file diff --git a/config/addresses.template.ts b/config/addresses.template.ts index 94537328..8c0ff97c 100644 --- a/config/addresses.template.ts +++ b/config/addresses.template.ts @@ -22,6 +22,7 @@ export class Addresses { l2GraphTokenGateway: string ethereumDIDRegistry: string subgraphService: string + issuanceAllocator: string graphPayments: string paymentsEscrow: string graphTallyCollector: string @@ -52,6 +53,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '{{l2GraphTokenGateway}}', ethereumDIDRegistry: '{{ethereumDIDRegistry}}', subgraphService: '{{subgraphService}}', + issuanceAllocator: '{{issuanceAllocator}}', graphPayments: '{{graphPayments}}', paymentsEscrow: '{{paymentsEscrow}}', graphTallyCollector: '{{graphTallyCollector}}', diff --git a/config/arbitrumSepoliaAddressScript.ts b/config/arbitrumSepoliaAddressScript.ts index 378cfae0..5b2913ea 100644 --- a/config/arbitrumSepoliaAddressScript.ts +++ b/config/arbitrumSepoliaAddressScript.ts @@ -5,6 +5,7 @@ import { Addresses } from './addresses.template' const horizonAddresses = require('@graphprotocol/address-book/horizon/addresses.json') const subgraphServiceAddresses = require('@graphprotocol/address-book/subgraph-service/addresses.json') +const issuanceAddresses = require('@graphprotocol/address-book/issuance/addresses.json') // mustache doesn't like numbered object keys // eslint-disable-next-line @typescript-eslint/no-explicit-any @@ -15,10 +16,15 @@ renameHorizonAddresses['arbsep'] = horizonAddresses['421614'] let renameSubgraphServiceAddresses: any = subgraphServiceAddresses renameSubgraphServiceAddresses['arbsep'] = subgraphServiceAddresses['421614'] +// eslint-disable-next-line @typescript-eslint/no-explicit-any +let renameIssuanceAddresses: any = issuanceAddresses +renameIssuanceAddresses['arbsep'] = issuanceAddresses['421614'] || {} + const combinedAddresses = { arbsep: { ...renameHorizonAddresses['arbsep'], - ...renameSubgraphServiceAddresses['arbsep'] + ...renameSubgraphServiceAddresses['arbsep'], + ...renameIssuanceAddresses['arbsep'] } } @@ -45,6 +51,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '{{arbsep.L2GraphTokenGateway.address}}', ethereumDIDRegistry: '{{arbsep.EthereumDIDRegistry.address}}', subgraphService: '{{arbsep.SubgraphService.address}}', + issuanceAllocator: '{{arbsep.IssuanceAllocator.address}}', graphPayments: '{{arbsep.GraphPayments.address}}', paymentsEscrow: '{{arbsep.PaymentsEscrow.address}}', graphTallyCollector: '{{arbsep.GraphTallyCollector.address}}', diff --git a/config/localNetworkAddressScript.ts b/config/localNetworkAddressScript.ts index 0efd624b..7932d785 100644 --- a/config/localNetworkAddressScript.ts +++ b/config/localNetworkAddressScript.ts @@ -10,6 +10,7 @@ import { Addresses } from './addresses.template' let renameAddresses: any = { horizon: horizonAddresses['1337'], subgraphService: subgraphServiceAddresses['1337'], + issuanceAllocator: '', // IssuanceAllocator is only deployed on Arbitrum } export let addresses: Addresses = { diff --git a/config/mainnetArbitrumAddressScript.ts b/config/mainnetArbitrumAddressScript.ts index a61504bc..cc848da9 100644 --- a/config/mainnetArbitrumAddressScript.ts +++ b/config/mainnetArbitrumAddressScript.ts @@ -4,6 +4,7 @@ import { Addresses } from './addresses.template' const horizonAddresses = require('@graphprotocol/address-book/horizon/addresses.json') const subgraphServiceAddresses = require('@graphprotocol/address-book/subgraph-service/addresses.json') +const issuanceAddresses = require('@graphprotocol/address-book/issuance/addresses.json') // mustache doesn't like numbered object keys // eslint-disable-next-line @typescript-eslint/no-explicit-any @@ -14,10 +15,15 @@ renameHorizonAddresses['arbitrum'] = horizonAddresses['42161'] || {} let renameSubgraphServiceAddresses: any = subgraphServiceAddresses renameSubgraphServiceAddresses['arbitrum'] = subgraphServiceAddresses['42161'] || {} +// eslint-disable-next-line @typescript-eslint/no-explicit-any +let renameIssuanceAddresses: any = issuanceAddresses +renameIssuanceAddresses['arbitrum'] = issuanceAddresses['42161'] || {} + const combinedAddresses = { arbitrum: { ...renameHorizonAddresses['arbitrum'], - ...renameSubgraphServiceAddresses['arbitrum'] + ...renameSubgraphServiceAddresses['arbitrum'], + ...renameIssuanceAddresses['arbitrum'] } } @@ -44,6 +50,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '{{arbitrum.L2GraphTokenGateway.address}}', ethereumDIDRegistry: '{{arbitrum.IEthereumDIDRegistry.address}}', subgraphService: '{{arbitrum.SubgraphService.address}}', + issuanceAllocator: '{{arbitrum.IssuanceAllocator.address}}', graphPayments: '{{arbitrum.GraphPayments.address}}', paymentsEscrow: '{{arbitrum.PaymentsEscrow.address}}', graphTallyCollector: '{{arbitrum.GraphTallyCollector.address}}', diff --git a/config/testAddressesL1.ts b/config/testAddressesL1.ts index 8de91df5..a5c59b8f 100644 --- a/config/testAddressesL1.ts +++ b/config/testAddressesL1.ts @@ -44,6 +44,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '', ethereumDIDRegistry: '0x0000000000000000000000000000000000000000', subgraphService: '0x0000000000000000000000000000000000000000', + issuanceAllocator: '', graphPayments: '0x0000000000000000000000000000000000000000', paymentsEscrow: '0x0000000000000000000000000000000000000000', graphTallyCollector: '0x0000000000000000000000000000000000000000', diff --git a/config/testAddressesL2.ts b/config/testAddressesL2.ts index ce8d1e34..07df5144 100644 --- a/config/testAddressesL2.ts +++ b/config/testAddressesL2.ts @@ -44,6 +44,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '0x0000000000000000000000000000000000000000', ethereumDIDRegistry: '0x0000000000000000000000000000000000000000', subgraphService: '0x0000000000000000000000000000000000000000', + issuanceAllocator: '', graphPayments: '0x0000000000000000000000000000000000000000', paymentsEscrow: '0x0000000000000000000000000000000000000000', graphTallyCollector: '0x0000000000000000000000000000000000000000', diff --git a/package.json b/package.json index dc5e698f..42d116fa 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "prepare:test-l2": "yarn && yarn prep:addresses:test:l2 && mustache ./config/generatedAddresses.json subgraph.template.yaml > subgraph.yaml && graph codegen --output-dir src/types/" }, "devDependencies": { - "@graphprotocol/address-book": "1.1.0", + "@graphprotocol/address-book": "^1.3.0", "@graphprotocol/contracts": "6.2.0", "@graphprotocol/graph-cli": "0.97.0", "@graphprotocol/graph-ts": "0.36.0", diff --git a/src/mappings/issuanceAllocator.ts b/src/mappings/issuanceAllocator.ts new file mode 100644 index 00000000..9eb48910 --- /dev/null +++ b/src/mappings/issuanceAllocator.ts @@ -0,0 +1,34 @@ +import { TargetAllocationUpdated } from '../types/IssuanceAllocator/IssuanceAllocator' +import { createOrLoadGraphNetwork } from './helpers/helpers' +import { addresses } from '../../config/addresses' +import { Address } from '@graphprotocol/graph-ts' + +/** + * @dev handleTargetAllocationUpdated + * - Keeps GraphNetwork.networkGRTIssuancePerBlock in step with the share of + * issuance the RewardsManager actually receives. + * + * Since GIP-0076/GIP-0088 the RewardsManager no longer owns the issuance rate: + * the IssuanceAllocator splits total issuance across targets and the + * RewardsManager mints only its own allocation. GIP-0089 exercised that for the + * first time on 2026-09-01, moving 24.146 GRT/block (20%) to the Foundation's + * Innovation Allocation and leaving the RewardsManager 96.584 of the unchanged + * 120.73 total. + * + * That change is invisible to the RewardsManager's own ParameterUpdated event, + * so without this handler networkGRTIssuancePerBlock stays at whatever the + * legacy `issuancePerBlock` storage slot last held — 120.73 — and every + * downstream APR calculation built on it overstates indexing rewards by 25%. + * + * The RewardsManager is an allocator target that mints its own issuance, so + * `newSelfMintingRate` is the rate it will issue at. The event carries it, so + * no contract call is needed here. + */ +export function handleTargetAllocationUpdated(event: TargetAllocationUpdated): void { + if (event.params.target != Address.fromString(addresses.rewardsManager)) { + return + } + let graphNetwork = createOrLoadGraphNetwork(event.block.number, event.address) + graphNetwork.networkGRTIssuancePerBlock = event.params.newSelfMintingRate + graphNetwork.save() +} diff --git a/src/mappings/rewardsManager.ts b/src/mappings/rewardsManager.ts index d66f33ed..52c29327 100644 --- a/src/mappings/rewardsManager.ts +++ b/src/mappings/rewardsManager.ts @@ -69,7 +69,16 @@ export function handleParameterUpdated(event: ParameterUpdated): void { if (parameter == 'issuanceRate') { graphNetwork.networkGRTIssuance = rewardsManager.issuanceRate() } else if (parameter == 'issuancePerBlock') { - graphNetwork.networkGRTIssuancePerBlock = rewardsManager.issuancePerBlock() + // Read the allocator-aware rate, not the legacy `issuancePerBlock` slot. + // Since the IssuanceAllocator was wired up (GIP-0088) the RewardsManager + // issues only its own allocation, and the legacy slot is no longer that + // number — on Arbitrum it still reads 120.73 while the allocated rate is + // 96.584. try_ so the call is safe on deployments predating the upgrade, + // where the getter does not exist. + let allocated = rewardsManager.try_getAllocatedIssuancePerBlock() + graphNetwork.networkGRTIssuancePerBlock = allocated.reverted + ? rewardsManager.issuancePerBlock() + : allocated.value } else if (parameter == 'subgraphAvailabilityOracle') { graphNetwork.subgraphAvailabilityOracle = rewardsManager.subgraphAvailabilityOracle() } diff --git a/subgraph.template.yaml b/subgraph.template.yaml index 09ffddb4..601cff61 100644 --- a/subgraph.template.yaml +++ b/subgraph.template.yaml @@ -529,6 +529,31 @@ dataSources: EpochManager.blockNum: EpochManager[{{epochManager}}].blockNum() - event: AllowedLockedVerifierSet(indexed address,bool) handler: handleAllowedLockedVerifierSet + # The IssuanceAllocator owns the split of protocol issuance across targets + # (GIP-0076, deployed by GIP-0088). The RewardsManager is one target and mints + # only its own allocation, so this is the only place a change to the indexing + # reward rate is observable — the RewardsManager itself emits nothing when the + # split moves. Arbitrum only; the address resolves empty elsewhere. + - kind: ethereum/contract + name: IssuanceAllocator + network: {{network}} + source: + address: "{{issuanceAllocator}}" + abi: IssuanceAllocator + startBlock: {{blockNumber}} + mapping: + kind: ethereum/events + apiVersion: 0.0.9 + language: wasm/assemblyscript + file: ./src/mappings/issuanceAllocator.ts + entities: + - GraphNetwork + abis: + - name: IssuanceAllocator + file: ./abis/IssuanceAllocator.json + eventHandlers: + - event: TargetAllocationUpdated(indexed address,uint256,uint256) + handler: handleTargetAllocationUpdated - kind: ethereum/contract name: SubgraphService network: {{network}} diff --git a/yarn.lock b/yarn.lock index 0f6cd4e1..034e1077 100644 --- a/yarn.lock +++ b/yarn.lock @@ -818,10 +818,10 @@ graphql-import-node "^0.0.5" js-yaml "^4.1.0" -"@graphprotocol/address-book@1.1.0": - version "1.1.0" - resolved "https://registry.npmjs.org/@graphprotocol/address-book/-/address-book-1.1.0.tgz" - integrity sha512-38NiutGOWdDIYgB/3kGp/7LxPRCNomipjo1hGlaj6RvyydySooVSmb3A+j84faUso+uhYxt8+9y1EFN7XVnT7Q== +"@graphprotocol/address-book@^1.3.0": + version "1.3.0" + resolved "https://registry.yarnpkg.com/@graphprotocol/address-book/-/address-book-1.3.0.tgz#234de18cd3931c309cf0385d46f88cc6d3a14355" + integrity sha512-rkrgBOOxM5WLNsoVrr6Gvg6DYvDes+P13vL5k1/QYWc5/z/G6pt4DBgzR/7wJV5xhIMevCumpg6L7ZuVQwS0ug== "@graphprotocol/common-ts@^2.0.7": version "2.0.11" From e931392f6352b41737b48379253efd25a14f0b20 Mon Sep 17 00:00:00 2001 From: PaulieB14 <94752445+PaulieB14@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:39:18 -0400 Subject: [PATCH 2/4] fix: use the zero address for IssuanceAllocator where it is not deployed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI caught this: `Path: dataSources > 9 > source > address / Contract address is invalid` on both L1 and L2 test prep. I had used an empty string as the placeholder on networks without an allocator, but graph-cli requires 40 hex characters and every other absent contract in these fixtures — subgraphService, graphPayments, paymentsEscrow, graphTallyCollector — uses the zero address. Verified by running the exact CI steps rather than assuming: testAddressesL1 and testAddressesL2 both now complete mustache + codegen, and the matchstick suite passes 88/88. --- config/localNetworkAddressScript.ts | 2 +- config/testAddressesL1.ts | 2 +- config/testAddressesL2.ts | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/config/localNetworkAddressScript.ts b/config/localNetworkAddressScript.ts index 7932d785..ee55619c 100644 --- a/config/localNetworkAddressScript.ts +++ b/config/localNetworkAddressScript.ts @@ -10,7 +10,7 @@ import { Addresses } from './addresses.template' let renameAddresses: any = { horizon: horizonAddresses['1337'], subgraphService: subgraphServiceAddresses['1337'], - issuanceAllocator: '', // IssuanceAllocator is only deployed on Arbitrum + issuanceAllocator: '0x0000000000000000000000000000000000000000', // IssuanceAllocator is only deployed on Arbitrum } export let addresses: Addresses = { diff --git a/config/testAddressesL1.ts b/config/testAddressesL1.ts index a5c59b8f..23741222 100644 --- a/config/testAddressesL1.ts +++ b/config/testAddressesL1.ts @@ -44,7 +44,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '', ethereumDIDRegistry: '0x0000000000000000000000000000000000000000', subgraphService: '0x0000000000000000000000000000000000000000', - issuanceAllocator: '', + issuanceAllocator: '0x0000000000000000000000000000000000000000', graphPayments: '0x0000000000000000000000000000000000000000', paymentsEscrow: '0x0000000000000000000000000000000000000000', graphTallyCollector: '0x0000000000000000000000000000000000000000', diff --git a/config/testAddressesL2.ts b/config/testAddressesL2.ts index 07df5144..10108fbe 100644 --- a/config/testAddressesL2.ts +++ b/config/testAddressesL2.ts @@ -44,7 +44,7 @@ export let addresses: Addresses = { l2GraphTokenGateway: '0x0000000000000000000000000000000000000000', ethereumDIDRegistry: '0x0000000000000000000000000000000000000000', subgraphService: '0x0000000000000000000000000000000000000000', - issuanceAllocator: '', + issuanceAllocator: '0x0000000000000000000000000000000000000000', graphPayments: '0x0000000000000000000000000000000000000000', paymentsEscrow: '0x0000000000000000000000000000000000000000', graphTallyCollector: '0x0000000000000000000000000000000000000000', From 445e4ea44b86c5c8a62811c7b188672a28a99f14 Mon Sep 17 00:00:00 2001 From: Juan Manuel Rodriguez Defago Date: Mon, 7 Sep 2026 18:18:42 -0300 Subject: [PATCH 3/4] fix: missing EpochManager abi for IssuanceAllocator --- subgraph.template.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/subgraph.template.yaml b/subgraph.template.yaml index 601cff61..81a20784 100644 --- a/subgraph.template.yaml +++ b/subgraph.template.yaml @@ -551,6 +551,8 @@ dataSources: abis: - name: IssuanceAllocator file: ./abis/IssuanceAllocator.json + - name: EpochManager + file: ./node_modules/@graphprotocol/contracts/dist/abis/EpochManager.json eventHandlers: - event: TargetAllocationUpdated(indexed address,uint256,uint256) handler: handleTargetAllocationUpdated From bbc312511049bcc152e3c819a936cc53484f05bb Mon Sep 17 00:00:00 2001 From: Juan Manuel Rodriguez Defago Date: Wed, 16 Sep 2026 11:03:14 -0300 Subject: [PATCH 4/4] fix: parse deployment manifests safely with native YAML Replace line-based manifest extraction with graph-ts YAML parsing to avoid indexing failures from CRLF line endings and malformed metadata. Preserve the original manifest and extract schema links, network, source kind, and minimum start block independently, leaving invalid fields unset with warnings. Guard YAML types and missing keys, validate unsigned block numbers and IPFS references, and preserve zero when calculating the minimum start block. Upgrade graph-ts to 0.38.2 and document the Graph Node 0.37.0 requirement. Validation: Arbitrum and L1 test-configuration codegen/builds, targeted lint, and diff checks passed. Matchstick was skipped as requested. Compiled-handler smoke checks also passed with an emulated YAML host, including the reported manifest, malformed inputs, and CID/path validation. --- README.md | 8 +- package.json | 2 +- src/mappings/helpers/manifest.ts | 128 ++++++++++++++++++++++++ src/mappings/ipfs.ts | 166 +++++++++++++++++++------------ yarn.lock | 43 ++++---- 5 files changed, 255 insertions(+), 92 deletions(-) create mode 100644 src/mappings/helpers/manifest.ts diff --git a/README.md b/README.md index 64b1a416..854985f5 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,12 @@ Everytime a new release is merged into `master` there will be a new github relea # Deploying the subgraph +The deployment manifest IPFS handler uses the native YAML API from `graph-ts` 0.38.2 +and requires **Graph Node 0.37.0 or newer**. Malformed manifests retain their raw +content; derived fields that cannot be safely extracted remain unset and produce +warnings. Schema links support CIDv0 and CIDv1 in base32 or base58btc, optionally +with an IPFS file path. Unsupported links are skipped without creating a file data source. + The npm scripts are set up to deploy the subgraphs in one command. Mainnet is connected to a hook where it will be deployed automatically when the `master` branch is updated. Therefore, we never have to use npm scripts to directly deploy to `graph-network-mainnet`. @@ -91,4 +97,4 @@ the schema. Copyright © 2020 The Graph Foundation. -Licensed under the [MIT license](./LICENSE). \ No newline at end of file +Licensed under the [MIT license](./LICENSE). diff --git a/package.json b/package.json index 42d116fa..769248f9 100644 --- a/package.json +++ b/package.json @@ -45,7 +45,7 @@ "@graphprotocol/address-book": "^1.3.0", "@graphprotocol/contracts": "6.2.0", "@graphprotocol/graph-cli": "0.97.0", - "@graphprotocol/graph-ts": "0.36.0", + "@graphprotocol/graph-ts": "0.38.2", "@types/node": "^14.0.13", "@typescript-eslint/eslint-plugin": "^3.3.0", "@typescript-eslint/parser": "^3.3.0", diff --git a/src/mappings/helpers/manifest.ts b/src/mappings/helpers/manifest.ts new file mode 100644 index 00000000..0b355170 --- /dev/null +++ b/src/mappings/helpers/manifest.ts @@ -0,0 +1,128 @@ +import { BigInt, YAMLValue } from '@graphprotocol/graph-ts' + +// Avoid YAMLValue's [] accessor: it asserts when a key is absent. +export function yamlField(value: YAMLValue | null, key: string): YAMLValue | null { + if (value === null || !value.isObject()) return null + return value.toObject().get(YAMLValue.newString(key)) +} + +export function yamlString(value: YAMLValue | null): string | null { + if (value === null || !value.isString()) return null + let text = value.toString().trim() + return text.length > 0 ? text : null +} + +// Block numbers must be unsigned integers. YAML NUMBER also includes floats, +// and YAMLValue.toBigInt() does not protect against invalid numeric strings. +export function manifestStartBlock(value: YAMLValue): BigInt | null { + let text: string + if (value.isNumber()) { + text = value.toNumber() + } else if (value.isString()) { + text = value.toString().trim() + } else { + return null + } + if (text.length == 0 || text.length > 20) return null + for (let i = 0; i < text.length; i++) { + let code = text.charCodeAt(i) + if (code < 48 || code > 57) return null + } + if (text.length == 20 && text > '18446744073709551615') return null + return BigInt.fromString(text) +} + +// Decode the CID encodings normally used in published manifests. Unsupported +// encodings are skipped rather than passed to a host function that can abort. +function decodeCid(text: string): Uint8Array | null { + if (text.length == 0 || text.length > 128) return null + let base58 = text.startsWith('Qm') || text.startsWith('z') + if (base58) { + let encoded = text.startsWith('z') ? text.slice(1) : text + let alphabet = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz' + let bytes = new Array() + for (let i = 0; i < encoded.length; i++) { + let carry = alphabet.indexOf(encoded.charAt(i)) + if (carry < 0) return null + for (let j = 0; j < bytes.length; j++) { + carry += i32(bytes[j]) * 58 + bytes[j] = u8(carry & 255) + carry >>= 8 + } + while (carry > 0) { + bytes.push(u8(carry & 255)) + carry >>= 8 + } + } + for (let i = 0; i < encoded.length && encoded.charAt(i) == '1'; i++) bytes.push(0) + let result = new Uint8Array(bytes.length) + for (let i = 0; i < bytes.length; i++) result[i] = bytes[bytes.length - i - 1] + return result + } + if (!text.startsWith('b') && !text.startsWith('B')) return null + let alphabet = text.startsWith('b') ? 'abcdefghijklmnopqrstuvwxyz234567' : 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567' + let result = new Uint8Array((text.length - 1) * 5 / 8) + let bits = 0 + let buffer = 0 + let offset = 0 + for (let i = 1; i < text.length; i++) { + let digit = alphabet.indexOf(text.charAt(i)) + if (digit < 0) return null + buffer = (buffer << 5) | digit + bits += 5 + if (bits >= 8) { + bits -= 8 + result[offset++] = u8(buffer >> bits) + buffer &= (1 << bits) - 1 + } + } + return bits < 5 && buffer == 0 ? result : null +} + +function validCid(text: string): bool { + let bytes = decodeCid(text) + if (bytes === null) return false + if (text.startsWith('Qm')) { + return text.length == 46 && bytes.length == 34 && bytes[0] == 0x12 && bytes[1] == 0x20 + } + // CIDv1 contains four unsigned varints: version, codec, hash code, digest + // length; followed by the digest. Graph Node supports digests up to 64 bytes. + let offset = 0 + for (let field = 0; field < 4; field++) { + let value: u64 = 0 + let terminated = false + for (let i = 0; i < 10 && offset < bytes.length; i++) { + let byte = bytes[offset++] + if (i == 9 && byte > 1) return false + value |= u64(byte & 127) << (i * 7) + if ((byte & 128) == 0) { + if (i > 0 && byte == 0) return false + terminated = true + break + } + } + if (!terminated) return false + if (field == 0 && value != 1) return false + if (field == 3) return value <= 64 && value == u64(bytes.length - offset) + } + return false +} + +export function manifestSchemaPath(value: YAMLValue | null): string | null { + // Published manifests use { '/': '/ipfs/CID' }; also accept string links. + if (value !== null && value.isObject()) value = yamlField(value, '/') + let path = yamlString(value) + if (path === null || path.length > 2048) return null + if (path.startsWith('/ipfs/')) path = path.slice(6) + else if (path.startsWith('ipfs://')) path = path.slice(7) + let segments = path.split('/') + if (!validCid(segments[0])) return null + for (let i = 1; i < segments.length; i++) { + if (segments[i].length == 0 || segments[i] == '.' || segments[i] == '..') return null + } + for (let i = 0; i < path.length; i++) { + let code = path.charCodeAt(i) + if (code <= 32 || code == 127 || path.charAt(i) == '?' || path.charAt(i) == '#' || path.charAt(i) == '%' || path.charAt(i) == '\\') return null + } + return path +} diff --git a/src/mappings/ipfs.ts b/src/mappings/ipfs.ts index b182440d..30b5db86 100644 --- a/src/mappings/ipfs.ts +++ b/src/mappings/ipfs.ts @@ -1,4 +1,4 @@ -import { json, Bytes, dataSource, JSONValueKind, log, DataSourceContext, BigInt } from '@graphprotocol/graph-ts' +import { json, Bytes, dataSource, JSONValueKind, log, DataSourceContext, BigInt, yaml, YAMLValue } from '@graphprotocol/graph-ts' import { SubgraphMeta, SubgraphVersionMeta, @@ -10,6 +10,7 @@ import { SubgraphDeploymentSchema as SubgraphDeploymentSchemaTemplate } from '../types/templates' import { jsonToString } from './utils' +import { yamlField, yamlString, manifestStartBlock, manifestSchemaPath } from './helpers/manifest' export function handleSubgraphMetadata(content: Bytes): void { let id = dataSource.context().getString("id") @@ -80,78 +81,111 @@ export function handleSubgraphDeploymentSchema(content: Bytes): void { subgraphDeploymentSchema.save() } -export function handleSubgraphDeploymentManifest(content: Bytes): void { - // Shouldn't need ID since the handler isn't gonna be called more than once, given that it's only on deployment creation. - let subgraphDeploymentManifest = new SubgraphDeploymentManifest(dataSource.stringParam()) - if (content !== null) { - subgraphDeploymentManifest.manifest = content.toString() - - let manifest = subgraphDeploymentManifest.manifest! - // we take the right side of the split, since it's the one which will have the schema ipfs hash - let schemaSplitTry = manifest.split('schema:\n', 2) - if (schemaSplitTry.length == 2) { - let schemaSplit = schemaSplitTry[1] - - let schemaFileSplitTry = schemaSplit.split('/ipfs/', 2) - if (schemaFileSplitTry.length == 2) { - let schemaFileSplit = schemaFileSplitTry[1] +function manifestWarning(id: string, field: string): void { + log.warning('[MANIFEST PARSING FAIL] deployment: {}, invalid or unsupported {}', [id, field]) +} - let schemaIpfsHashTry = schemaFileSplit.split('\n', 2) - if (schemaIpfsHashTry.length == 2) { - let schemaIpfsHash = schemaIpfsHashTry[0] - let schemaId = subgraphDeploymentManifest.id.concat('-').concat(schemaIpfsHash) - subgraphDeploymentManifest.schema = schemaId - subgraphDeploymentManifest.schemaIpfsHash = schemaIpfsHash +function readManifestSchema(manifest: SubgraphDeploymentManifest, root: YAMLValue): void { + let path = manifestSchemaPath(yamlField(yamlField(root, 'schema'), 'file')) + if (path === null) { + manifestWarning(manifest.id, 'schema.file') + return + } + let schemaId = manifest.id.concat('-').concat(path) + manifest.schema = schemaId + manifest.schemaIpfsHash = path + let context = new DataSourceContext() + context.setString('id', schemaId) + SubgraphDeploymentSchemaTemplate.createWithContext(path, context) +} - let context = new DataSourceContext() - context.setString('id', schemaId) - SubgraphDeploymentSchemaTemplate.createWithContext(schemaIpfsHash, context) - } else { - log.warning("[MANIFEST PARSING FAIL] subgraphDeploymentManifest: {}, schema file hash can't be retrieved. Error: schemaIpfsHashTry.length isn't 2, actual length: {}", [dataSource.stringParam(), schemaIpfsHashTry.length.toString()]) - } - } else { - log.warning("[MANIFEST PARSING FAIL] subgraphDeploymentManifest: {}, schema file hash can't be retrieved. Error: schemaFileSplitTry.length isn't 2, actual length: {}", [dataSource.stringParam(), schemaFileSplitTry.length.toString()]) +function readManifestNetwork(manifest: SubgraphDeploymentManifest, root: YAMLValue): void { + // Keep the first usable network, falling back to templates when necessary. + let sections = ['dataSources', 'templates'] + for (let section = 0; section < sections.length; section++) { + let entries = yamlField(root, sections[section]) + if (entries === null || !entries.isArray()) continue + let sources = entries.toArray() + for (let i = 0; i < sources.length; i++) { + let network = yamlString(yamlField(sources[i], 'network')) + if (network !== null && validManifestNetwork(network)) { + manifest.network = network + return } - } else { - log.warning("[MANIFEST PARSING FAIL] subgraphDeploymentManifest: {}, schema file hash can't be retrieved. Error: schemaSplitTry.length isn't 2, actual length: {}", [dataSource.stringParam(), schemaSplitTry.length.toString()]) } + } + manifestWarning(manifest.id, 'network') +} - // We get the first occurrence of `network` since subgraphs can only have data sources for the same network - let networkSplitTry = manifest.split('network: ', 2) - if (networkSplitTry.length == 2) { - let networkSplit = networkSplitTry[1] - let networkTry = networkSplit.split('\n', 2) - if (networkTry.length == 2) { - let network = networkTry[0] +function validManifestNetwork(network: string): bool { + if (network.length > 256) return false + for (let i = 0; i < network.length; i++) { + let code = network.charCodeAt(i) + if (code <= 32 || code == 127) return false + } + return true +} - subgraphDeploymentManifest.network = network - } else { - log.warning("[MANIFEST PARSING FAIL] subgraphDeploymentManifest: {}, network can't be parsed. Error: networkTry.length isn't 2, actual length: {}", [dataSource.stringParam(), networkTry.length.toString()]) - } - } else { - log.warning("[MANIFEST PARSING FAIL] subgraphDeploymentManifest: {}, network can't be parsed. Error: networkSplitTry.length isn't 2, actual length: {}", [dataSource.stringParam(), networkSplitTry.length.toString()]) - } - let substreamsSplitTry = manifest.split('- kind: substreams', 2) - subgraphDeploymentManifest.poweredBySubstreams = substreamsSplitTry.length > 1 +function readManifestDataSources(manifest: SubgraphDeploymentManifest, root: YAMLValue): void { + let sources = yamlField(root, 'dataSources') + if (sources === null || !sources.isArray() || sources.toArray().length == 0) { + manifestWarning(manifest.id, 'dataSources') + return + } - // startBlock calculation - let templatesSplit = manifest.split("templates:") - let nonTemplateManifestSplit = templatesSplit[0] // we take the left as we want to remove the templates for the source checks. - let sourcesSplit = nonTemplateManifestSplit.split("source:") // We want to know how many source definitions we have - let startBlockSplit = nonTemplateManifestSplit.split("startBlock: ") // And how many startBlock definitions we have to know if we should set startBlock to 0 - - if (sourcesSplit.length > startBlockSplit.length) { - subgraphDeploymentManifest.startBlock = BigInt.fromI32(0) - } else { - // need to figure the minimum startBlock defined, we skip i = 0 as we know it's not gonna contain a start block num, since it's before the first appearance of "startBlock:" - let min = BigInt.fromI32(0) - for(let i = 1; i < startBlockSplit.length; i++) { - let numString = startBlockSplit[i].split("\n", 1)[0].toString() - let num = BigInt.fromString(numString) - min = min == BigInt.fromI32(0) ? num : min <= num ? min : num - } - subgraphDeploymentManifest.startBlock = min + let dataSources = sources.toArray() + let minimum: BigInt | null = null + let validStartBlocks = true + let validKinds = true + let poweredBySubstreams = false + // Only inspect actual dataSources. Templates, comments and context values + // must not affect the minimum start block or the deployment's source kind. + for (let i = 0; i < dataSources.length; i++) { + let dataSource = dataSources[i] + let kind = yamlString(yamlField(dataSource, 'kind')) + if (kind === null) validKinds = false + else if (kind == 'substreams') poweredBySubstreams = true + + let source = yamlField(dataSource, 'source') + if (source === null || !source.isObject()) { + validStartBlocks = false + continue + } + let startBlockValue = yamlField(source, 'startBlock') + // A missing startBlock defaults to zero. An explicit null or malformed + // value is unknown, so we cannot reliably report a minimum. + let startBlock = startBlockValue === null ? BigInt.fromI32(0) : manifestStartBlock(startBlockValue) + if (startBlock === null) { + validStartBlocks = false + } else if (minimum === null || startBlock < minimum) { + minimum = startBlock } } - subgraphDeploymentManifest.save() + + if (poweredBySubstreams || validKinds) manifest.poweredBySubstreams = poweredBySubstreams + else manifestWarning(manifest.id, 'dataSources.kind') + if (validStartBlocks && minimum !== null) manifest.startBlock = minimum + else manifestWarning(manifest.id, 'dataSources.source.startBlock') +} + +export function handleSubgraphDeploymentManifest(content: Bytes): void { + let manifest = new SubgraphDeploymentManifest(dataSource.stringParam()) + manifest.manifest = content.toString() + // Match the native parser's input limit; retain the raw manifest on failure. + if (content.length > 10000000) { + manifestWarning(manifest.id, 'manifest size') + manifest.save() + return + } + let parsed = yaml.try_fromBytes(content) + if (!parsed.isOk) { + manifestWarning(manifest.id, 'YAML') + } else if (!parsed.value.isObject()) { + manifestWarning(manifest.id, 'manifest root') + } else { + readManifestSchema(manifest, parsed.value) + readManifestNetwork(manifest, parsed.value) + readManifestDataSources(manifest, parsed.value) + } + manifest.save() } diff --git a/yarn.lock b/yarn.lock index 034e1077..4562c113 100644 --- a/yarn.lock +++ b/yarn.lock @@ -898,12 +898,12 @@ web3-eth-abi "4.4.1" yaml "2.7.0" -"@graphprotocol/graph-ts@0.36.0": - version "0.36.0" - resolved "https://registry.npmjs.org/@graphprotocol/graph-ts/-/graph-ts-0.36.0.tgz" - integrity sha512-yJNQK5YZWEThuawSboQQ+U4Fb2C78KBjmaoeOK7Nn0CFoChmHc+woRvW3yj+IKVSPc7JNHt4JSUHxVDJfUZbTA== +"@graphprotocol/graph-ts@0.38.2": + version "0.38.2" + resolved "https://registry.yarnpkg.com/@graphprotocol/graph-ts/-/graph-ts-0.38.2.tgz#e99bdb6e0e50485084c442d7a02409082a836ff8" + integrity sha512-87KIFSFs2+Te+mnmb7Y+M57oqzlLy20cIyPIRbn9qJfpZFSZHTKtBLT6KQmcsK0YkoWis9Ur3c3M2c9mmaaEHQ== dependencies: - assemblyscript "0.19.10" + assemblyscript "0.27.31" "@graphprotocol/pino-sentry-simple@0.7.1": version "0.7.1" @@ -2164,14 +2164,6 @@ array-union@^2.1.0: resolved "https://registry.npmjs.org/array-union/-/array-union-2.1.0.tgz" integrity sha512-HGyxoOTYUyCM6stUe6EJgnd4EoewAI7zMdfqO+kGjnlZmBDz/cR5pf8r/cR4Wq60sL/p0IkcjUEEPwS3GFrIyw== -assemblyscript@0.19.10: - version "0.19.10" - resolved "https://registry.npmjs.org/assemblyscript/-/assemblyscript-0.19.10.tgz" - integrity sha512-HavcUBXB3mBTRGJcpvaQjmnmaqKHBGREjSPNsIvnAk2f9dj78y4BkMaSSdvBQYWcDDzsHQjyUC8stICFkD1Odg== - dependencies: - binaryen "101.0.0-nightly.20210723" - long "^4.0.0" - assemblyscript@0.19.23: version "0.19.23" resolved "https://registry.npmjs.org/assemblyscript/-/assemblyscript-0.19.23.tgz" @@ -2181,6 +2173,14 @@ assemblyscript@0.19.23: long "^5.2.0" source-map-support "^0.5.20" +assemblyscript@0.27.31: + version "0.27.31" + resolved "https://registry.yarnpkg.com/assemblyscript/-/assemblyscript-0.27.31.tgz#07412b1bc42c67f78080dbaddca030ab74d3b9b2" + integrity sha512-Ra8kiGhgJQGZcBxjtMcyVRxOEJZX64kd+XGpjWzjcjgxWJVv+CAQO0aDBk4GQVhjYbOkATarC83mHjAVGtwPBQ== + dependencies: + binaryen "116.0.0-nightly.20240114" + long "^5.2.1" + astral-regex@^2.0.0: version "2.0.0" resolved "https://registry.npmjs.org/astral-regex/-/astral-regex-2.0.0.tgz" @@ -2261,16 +2261,16 @@ binary-extensions@^2.0.0: resolved "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz" integrity sha512-Ceh+7ox5qe7LJuLHoY0feh3pHuUDHAcRUeyL2VYghZwfpkNIy/+8Ocg0a3UuSoYzavmylwuLWQOf3hl0jjMMIw== -binaryen@101.0.0-nightly.20210723: - version "101.0.0-nightly.20210723" - resolved "https://registry.npmjs.org/binaryen/-/binaryen-101.0.0-nightly.20210723.tgz" - integrity sha512-eioJNqhHlkguVSbblHOtLqlhtC882SOEPKmNFZaDuz1hzQjolxZ+eu3/kaS10n3sGPONsIZsO7R9fR00UyhEUA== - binaryen@102.0.0-nightly.20211028: version "102.0.0-nightly.20211028" resolved "https://registry.npmjs.org/binaryen/-/binaryen-102.0.0-nightly.20211028.tgz" integrity sha512-GCJBVB5exbxzzvyt8MGDv/MeUjs6gkXDvf4xOIItRBptYl0Tz5sm1o/uG95YK0L0VeG5ajDu3hRtkBP2kzqC5w== +binaryen@116.0.0-nightly.20240114: + version "116.0.0-nightly.20240114" + resolved "https://registry.yarnpkg.com/binaryen/-/binaryen-116.0.0-nightly.20240114.tgz#ad8bfbde77d4cb4715b93997114eefc30f45155b" + integrity sha512-0GZrojJnuhoe+hiwji7QFaL3tBlJoA+KFUN7ouYSDGZLSo9CKM8swQX8n/UcbR0d1VuZKU+nhogNzv423JEu5A== + bintrees@1.0.2: version "1.0.2" resolved "https://registry.npmjs.org/bintrees/-/bintrees-1.0.2.tgz" @@ -4811,12 +4811,7 @@ log-symbols@^4.1.0: chalk "^4.1.0" is-unicode-supported "^0.1.0" -long@^4.0.0: - version "4.0.0" - resolved "https://registry.npmjs.org/long/-/long-4.0.0.tgz" - integrity sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA== - -long@^5.2.0: +long@^5.2.0, long@^5.2.1: version "5.3.2" resolved "https://registry.npmjs.org/long/-/long-5.3.2.tgz" integrity sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==