Skip to content

[SVLS-9526] Prototype: serverless-compat inventory payload + 2-platform POC deploy scripts - #150

Draft
nina9753 wants to merge 21 commits into
mainfrom
nina/svls-9604-inventory-payload
Draft

nina9753 wants to merge 21 commits into
mainfrom
nina/svls-9604-inventory-payload

Conversation

@nina9753

Copy link
Copy Markdown
Contributor

What does this PR do?

Motivation

Additional Notes

Describe how to test/QA your changes

- Set flavor to "serverless-compat" so EPRW routes to serverless_compat_agent
  table instead of the generic datadog_agent ECS Fargate path
- Build resource_id CCRID per platform:
  - Azure Functions: //microsoft.azure/functionApps/{sub}/{rg}/{name} using
    WEBSITE_OWNER_NAME for subscription, WEBSITE_RESOURCE_GROUP for rg
  - GCP Cloud Functions gen1: //cloudfunctions.googleapis.com/projects/{p}/
    locations/{r}/functions/{f} from FUNCTION_NAME/FUNCTION_REGION/GCP_PROJECT
- Fix field names to match EPRW stringFields allowlist: dd_env/dd_service/
  dd_version/dd_site (was config_env/config_service/config_version/config_site)
- Fix: region (was cloud_region), gcp_project_id (was cloud_account_id)
- Add: azure_subscription_id, azure_resource_group from Azure env vars
- Add: serverless_compat_runtime_version from FUNCTIONS_WORKER_RUNTIME (Azure)
  or runtime env detection (GCP)
- Update workload type for CloudFunction: gcp_cloud_function_gen1 (EPRW allowlist)

SVLS-9604
- demo.sh: combined script that triggers both serverless-compat (Azure
  Function + GCP Cloud Function gen1) and serverless-init (all 9 services)
  in one run; supports SKIP_* flags for incremental runs
- check-redapl.sh: updated to query serverless_compat_agent (primary target)
  in addition to datadog_agent; includes field-completeness query and
  cross-reference query against serverless_init_agent

SVLS-9604
Replace hardcoded api_key_uuid with a placeholder. API key was
inadvertently included in a comment — rotate immediately if not
already done.

SVLS-9604
Runs N concurrent trigger waves (L0=1, L1=10, L2=50, L3=100) against
the Azure Function and GCP Cloud Function gen1 test functions.

Measures per-request latency (p50/p95/p99), captures EPRW metrics
to watch, and prints DDSQL cardinality queries to verify:
- Row count stays at 1 per resource_id regardless of trigger count
- Intake lag from trigger time to first_seen_at
- No rows from unknown workload_type rejections

SVLS-9604
- Use Rust 2021 let-chains to collapse nested if-let+if patterns (8x collapsible_if)
- Replace .last() with .next_back() on DoubleEndedIterator (Iterator::last lint)
- Fix formatting: remove alignment spaces, collapse single-arg format! call signature
- Bump h2 0.4.13 → 0.4.18 to fix RUSTSEC-2026-0258 (DoS via empty DATA frames)
@nina9753
nina9753 force-pushed the nina/svls-9604-inventory-payload branch from 37e3322 to 79233ad Compare August 24, 2026 11:07
- Run rustfmt --edition 2024: wrap long warn! line, collapse two-line let bindings in tests
- Regenerate LICENSE-3rdparty.csv: h2 0.4.18 no longer pulls windows_*
  v0.53.x crates unconditionally on Linux; removes 9 extraneous records
…ME fallback

- Return empty resource_id when FUNCTION_NAME is absent (Gen2 functions
  set K_SERVICE+FUNCTION_TARGET but not FUNCTION_NAME, so absence of
  FUNCTION_NAME is the Gen2 signal — skip the write as unsupported)
- Add REGION_NAME as region fallback for Gen1 functions running on
  Cloud Run infrastructure where FUNCTION_REGION may be absent
- Add ENV_LOCK mutex to tests to prevent concurrent env-var races
- Add two new tests covering the Gen2-skip and REGION_NAME-fallback paths
…N_REGION absent

Gen1 Cloud Functions on newer Cloud Run infrastructure (nodejs18+) no
longer have FUNCTION_REGION injected at runtime by GCP. When env var
lookup fails, fetch the region from the GCP instance metadata server:
  http://metadata.google.internal/computeMetadata/v1/instance/region
Returns 'projects/<num>/regions/<region>'; extract the last component.

The metadata server call is async, timeout 2s, only attempted for
CloudFunction + when FUNCTION_NAME is present but region is missing.
…r call

Using bare reqwest::Client::builder() with default-features=false may fail
silently. Switch to the same create_reqwest_client_builder() used by the
existing inventory payload POST so the HTTP transport is correctly configured.

Also add warn! log lines so metadata server failures surface in Cloud Logging
rather than silently producing an empty resource_id.
- Extract shared `fetch_gcp_metadata_value` helper; `fetch_gcp_region_from_metadata`
  and `fetch_gcp_project_from_metadata` become 3-line wrappers (~40 lines saved)
- Fix `enrich_platform_fields` for CloudFunction to use the full env-var
  fallback chain (FUNCTION_REGION → GOOGLE_CLOUD_REGION → REGION_NAME) so
  `region` in the REDAPL payload is consistent with how `resource_id` is built
- azure-sanity matrix: add SI-06/SI-08/SI-09 sidecar profiles
- runner: fix profile name in manifest, add SC-L1/SC-L2/SC-L3 compat load stages
Aligns with EPRW allowlist and schema rename. cloud_function is the canonical
name for GCP Gen1 Cloud Functions in the compat agent table.
…ripts

- cloud_function_gen2 → cloud_run_function (SI-04)
- gcp_cloud_function_gen1 → cloud_function (SC-02)
- Remove deployment_model from SC (compat) entries: no such column
- report.py: update revision_rows filter to cloud_run_function
- report.py: replace nonexistent resource_edge_response metric with
  correct serverless_write.accepted scoped by resource_type tag
- Spawn inventory::send_inventory_payload so it never blocks cold-start
  path (previously awaited with a 5 s timeout before agent init)
- Remove K_SERVICE fallback: when FUNCTION_NAME is absent build_resource_identity
  already returns (empty, empty) as the Gen2 skip signal; the outer block
  in send_inventory_payload now guards on !resource_name.is_empty() so it
  can only reach the metadata-server fetch when FUNCTION_NAME was present
  but region/project were missing (the one legitimate Gen1-on-Cloud-Run case)
- Separate AzureSpringApp from AzureFunction in build_resource_identity so
  spring apps produce an empty resource_id and EPRW skips the write (azure_spring_app
  is not in the EPRW workload_type allowlist)
- Remove stale comment claiming uuid is a required EPRW identity field
…untimes

read_cloud_env() detects Gen1 Cloud Functions via K_SERVICE+FUNCTION_TARGET
on newer runtimes (e.g. nodejs20), but build_resource_identity() only read
FUNCTION_NAME. When FUNCTION_NAME is absent, resource_name was empty so the
GCP metadata-server fallback for region/project was also skipped, leaving
resource_id empty and the serverless_compat_agent REDAPL write silently
dropped by EPRW.

Fix: fall back to K_SERVICE when FUNCTION_NAME is absent. The metadata-server
path then fills in region and project, completing the CCRID.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant