Skip to content

feat(waterdata): request v1 of the Water Data API, pinnable through api_version - #422

Draft
thodson-usgs wants to merge 1 commit into
DOI-USGS:mainfrom
thodson-usgs:feat/waterdata-api-v1
Draft

thodson-usgs wants to merge 1 commit into
DOI-USGS:mainfrom
thodson-usgs:feat/waterdata-api-v1

Conversation

@thodson-usgs

@thodson-usgs thodson-usgs commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #421.

What changes

  • Every Water Data OGC request now goes to /ogcapi/v1 (release post; v0 stays online until June 2027, then redirects).
  • The version is its own variable, OGC_API_VERSION in waterdata/endpoints.py, combined into the path at request time. Statistics and STAC keep v0 literals: neither has a v1 (/statistics/v1 and /stac/v1 answer 404, probed 2026-09-22).
  • New setting api_version on WaterdataConfiguration, readable as configuration.api_version() the way base_url() is:
    with dataretrieval.configure(WaterdataConfiguration(api_version="v0")):
        ...
    [waterdata]
    api_version = "v0"
  • get_time_series_metadata keeps working for the four queryables v1 dropped (begin_utc, end_utc, state_name, hydrologic_unit_code, and state since it resolves into state_name). v1 answers 400 or 500 to them, so a call naming one, as a filter or in properties, is sent to v0 under a DeprecationWarning. This mirrors R dataRetrieval #926.
  • Docstrings, tests, demos, the configuration guide, ADR 0011 notes, and NEWS updated.

Parameter or config? Config.

The version describes the service deployment the adapter talks to, the same category as base_url (ADR 0010), not a property of a query. A per-call kwarg would touch every getter and collide with the OGC getters' **queryables passthrough. SDK convention agrees: botocore, Azure, and Stripe pin a default per release and expose the override on the client or config object, never per call. R dataRetrieval does the same with options("dataRetrieval.api_version").

Within config, the sources differ from base_url on purpose:

Source base_url api_version Why
configure() block yes yes
file, [waterdata] table refused accepted the block-only rule exists because a base URL can redirect requests to another host; a version cannot, and one file moves every script on a machine
environment refused refused a variable is package-wide, and a version names one service; keeps API_USGS_* the set R shares

Mechanically this separates scope (adapter-only: no package-wide value exists) from provenance (which sources may supply it). Both axes were already in the vocabulary, so no third concept is added: BLOCK_ONLY_SETTINGS = ("base_url",) keys the file's refusal, while _REFUSED_ENV_VARS still derives from ADAPTER_ONLY_SETTINGS. A top-level api_version in the file raises with the remedy (move it into [waterdata]).

Behavior changes (upstream, no code change)

  • get_time_series_metadata: begin and end are now UTC with a time zone; begin_utc, end_utc, state_name, hydrologic_unit_code columns are gone.
  • get_field_measurements: time is a date, parsed to a tz-naive midnight as get_daily already does; time of day is in time_of_day. (An earlier revision of this description also named get_latest_field_measurements; no such getter exists, and field-measurements-metadata has no time field, so only the one getter is affected.)
  • The CSV qualifier change in the release post does not affect this package (OGC responses are JSON).

Second commit: cleanup pass

A four-angle review (reuse, simplification, efficiency, altitude) ran over the first commit. The substantive result was a fix, not a tidy:

The v0 fallback was writing on the caller's precedence ladder. It routed by entering configure(WaterdataConfiguration(api_version="v0")), which put this library on rung 1 of ADR 0011's ladder — the caller's own rung. A caller who pinned api_version="v1" was silently outranked, and show_configuration() would report a version they never asked for. The destination is already a per-call argument everywhere below that line (ogc/policy.py: "which API an OGC call targets is the adapter's policy, supplied per call as base_url"), so ogc_api_url() and waterdata.utils.get_ogc_data() now take an optional api_version and the getter pins the one request. Both configuration imports leave metadata.py. New test test_v0_routing_does_not_write_on_the_callers_configuration pins it.

Also in that commit: CODE_ONLY_SETTINGS → BLOCK_ONLY_SETTINGS ("code" was a second identifier spelling of the _BLOCK source, which ADR 0013 forbids for a core term); the split's rationale cut from eight restatements to one, in ADR 0011 (ADR 0000); the warning moved back to its call site instead of passing _WARN_STACKLEVEL + 1; one refusal message instead of two near-identical ones; the horizon read from REMOVALS once instead of also being spelled in the advisory's prose; and a comment saying why the two renames do not use _accept_legacy_kwargs.

Test data that the v0→v1 move had left stale: tests/data/waterdata_queryables.json was still v0 while QUERYABLES_RE had moved to v1, so the scheduled live monitor would have gone red. Regenerated by the documented procedure; it confirms the four dropped queryables and surfaces one addition, method_category on continuous. The time-series-metadata OGC fixture was regenerated from v1 for the same reason.

Judgment calls to confirm

  1. Horizon 2027-06-01 for the v0 shim is the service's date, not the repo's announcement + 1 year: the shim cannot outlive the endpoint it sends to.
  2. File accepts api_version (see table). If you would rather keep every adapter-only setting block-only, drop BLOCK_ONLY_SETTINGS and the file test.
  3. The two renames route to v0 rather than translating forward. ADR 0012 makes _accept_legacy_kwargs the default for a renamed argument. It is not used here because begin_utc may also arrive in properties, where translating forward would silently rename the returned column. Splitting behavior by call position seemed worse. Happy to switch if you read it the other way.
  4. method_category is a new v1 queryable on continuous, reachable today through **queryables. Promoting it to a documented parameter is a feature, so it is left for a follow-up.
  5. NEWS is dated 09/22/2026; adjust to the land date.

Verification

  • ruff, mypy --strict, lint-imports (8 kept), xenon, complexipy, and the full pre-commit suite pass. Offline suite 1192 passed, coverage 98.95% (ratchet 98.9). Docs build clean.
  • Live against the service: v1 default call is silent under -W error::DeprecationWarning and returns tz-aware begin with no begin_utc; each legacy filter warns under the caller's own spelling and returns from v0; both get_combined_metadata remedies return rows on v1; a caller-pinned v1 survives the getter's internal v0 routing and still applies to every other getter and to show_configuration(); file, top-level, and environment sources behave as documented; the live queryables monitor passes against the regenerated snapshot.

🤖 Generated with Claude Code

thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Sep 22, 2026
Taken so this branch carries no edit that conflicts with DOI-USGS#422. The
`data_gap_interval` fix this branch made to the v0 column list is
subsumed by DOI-USGS#422's rewrite of that list for v1, so the resolution keeps
DOI-USGS#422's version -- including `statistics_begin` -- and this branch keeps
only the monitor.

`dataretrieval/waterdata/metadata.py` is byte-identical to DOI-USGS#422's copy
after this merge, which is the check worth making: resolving a column
list the wrong way would silently revert part of the other PR while every
offline test still passed.
thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Sep 23, 2026
Comparing each collection's /schema with its getter's signature found 20
returned columns that were reachable only through **queryables, so the
getter documented neither the column nor the filter:

- get_field_measurements: control_condition, day,
  field_measurements_series_id, measurement_rated, month, reading_type,
  time_of_day, year
- get_peaks: qualifier, time_of_day, value
- get_monitoring_locations: revision_created, revision_modified,
  revision_note
- get_combined_metadata: data_gap_interval, reading_type
- get_time_series_metadata: data_gap_interval, parameter_description
- get_field_measurements_metadata: reading_type
- get_channel: channel_location_direction

Each is now a named parameter, described in the service's own words. day,
month and year take the integer annotation get_peaks already uses. The
monitoring-location attributes every collection accepts as filters but
does not return stay in **queryables. Existing calls send the same
request as before.

Stacked: this commit also carries DOI-USGS#422 (Water Data API v1), DOI-USGS#423
(continuous method_category and the API-version monitor) and DOI-USGS#424 (the
documented-columns monitor), which merge first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Sep 23, 2026
get_daily, get_continuous and get_time_series_metadata list their
returned columns in the properties docstring. The list is hand-written
and went stale twice without anything noticing: continuous was missing
method_category (DOI-USGS#423) and time-series-metadata data_gap_interval
(DOI-USGS#422).

Add a live test that compares each list with the collection's /schema
and names the docstring to edit when they differ. id is excluded on both
sides because only time-series-metadata lists it in its schema, though
all three accept it.

Stacked: this commit also carries DOI-USGS#422 (Water Data API v1) and DOI-USGS#423
(continuous method_category and the API-version monitor), which merge
first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n setting

The Water Data OGC collections moved to v1 in September 2026; v0 stays
online until June 2027. Every OGC request now goes to /ogcapi/v1.
Statistics and STAC stay on v0, because neither has a v1.

The version is a setting, like base_url, not a getter parameter.
WaterdataConfiguration gains api_version, which can be set in a
configure() block or in the [waterdata] table of the configuration file,
but not from the environment. The file now refuses the settings in
BLOCK_ONLY_SETTINGS rather than all of ADAPTER_ONLY_SETTINGS: base_url
can send requests to another host, and a version cannot.

v1 drops four time-series-metadata queryables (begin_utc, end_utc,
state_name, hydrologic_unit_code) and answers 400 or 500 to them.
get_time_series_metadata sends a call that names any of them (state
resolves to state_name) to v0 and emits a DeprecationWarning; the removal
date is 2027-06-01, when the service retires v0. The v0 URL is set per
request, through an api_version argument to ogc_api_url() and
get_ogc_data(), rather than by entering configure(), so it never
overrides a version the caller set.

Two behavior changes, both in NEWS: time-series-metadata begin/end are
UTC with a time zone, and get_field_measurements returns time as a date.
v1 also adds statistics_begin to time-series-metadata. The queryables
snapshot and the time-series-metadata fixture are regenerated from v1.

Closes DOI-USGS#421.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Sep 24, 2026
Comparing each collection's /schema with its getter's signature found 20
returned columns that could be passed only through **queryables, so the
getter documented neither the column nor the filter:

- get_field_measurements: control_condition, day,
  field_measurements_series_id, measurement_rated, month, reading_type,
  time_of_day, year
- get_peaks: qualifier, time_of_day, value
- get_monitoring_locations: revision_created, revision_modified,
  revision_note
- get_combined_metadata: data_gap_interval, reading_type
- get_time_series_metadata: data_gap_interval, parameter_description
- get_field_measurements_metadata: reading_type
- get_channel: channel_location_direction

Each is now a named parameter, described in the service's own words. day,
month and year are typed as integers, as get_peaks already types them.
The monitoring-location attributes that every collection accepts as
filters but does not return stay in **queryables. Existing calls send
the same request as before.

Stacked on DOI-USGS#422, DOI-USGS#423 and DOI-USGS#424, which merge first; review this commit
alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v1 of the water data apis has been released

1 participant