feat(waterdata): request v1 of the Water Data API, pinnable through api_version - #422
Draft
thodson-usgs wants to merge 1 commit into
Draft
thodson-usgs wants to merge 1 commit into
thodson-usgs wants to merge 1 commit into
Conversation
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Sep 22, 2026
Taken so this branch carries no edit that conflicts with DOI-USGS#422. The `data_gap_interval` fix this branch made to the v0 column list is subsumed by DOI-USGS#422's rewrite of that list for v1, so the resolution keeps DOI-USGS#422's version -- including `statistics_begin` -- and this branch keeps only the monitor. `dataretrieval/waterdata/metadata.py` is byte-identical to DOI-USGS#422's copy after this merge, which is the check worth making: resolving a column list the wrong way would silently revert part of the other PR while every offline test still passed.
This was referenced Sep 22, 2026
thodson-usgs
force-pushed
the
feat/waterdata-api-v1
branch
from
September 23, 2026 02:36
1e2e1c8 to
574f448
Compare
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Sep 23, 2026
Comparing each collection's /schema with its getter's signature found 20 returned columns that were reachable only through **queryables, so the getter documented neither the column nor the filter: - get_field_measurements: control_condition, day, field_measurements_series_id, measurement_rated, month, reading_type, time_of_day, year - get_peaks: qualifier, time_of_day, value - get_monitoring_locations: revision_created, revision_modified, revision_note - get_combined_metadata: data_gap_interval, reading_type - get_time_series_metadata: data_gap_interval, parameter_description - get_field_measurements_metadata: reading_type - get_channel: channel_location_direction Each is now a named parameter, described in the service's own words. day, month and year take the integer annotation get_peaks already uses. The monitoring-location attributes every collection accepts as filters but does not return stay in **queryables. Existing calls send the same request as before. Stacked: this commit also carries DOI-USGS#422 (Water Data API v1), DOI-USGS#423 (continuous method_category and the API-version monitor) and DOI-USGS#424 (the documented-columns monitor), which merge first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Sep 23, 2026
get_daily, get_continuous and get_time_series_metadata list their returned columns in the properties docstring. The list is hand-written and went stale twice without anything noticing: continuous was missing method_category (DOI-USGS#423) and time-series-metadata data_gap_interval (DOI-USGS#422). Add a live test that compares each list with the collection's /schema and names the docstring to edit when they differ. id is excluded on both sides because only time-series-metadata lists it in its schema, though all three accept it. Stacked: this commit also carries DOI-USGS#422 (Water Data API v1) and DOI-USGS#423 (continuous method_category and the API-version monitor), which merge first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n setting The Water Data OGC collections moved to v1 in September 2026; v0 stays online until June 2027. Every OGC request now goes to /ogcapi/v1. Statistics and STAC stay on v0, because neither has a v1. The version is a setting, like base_url, not a getter parameter. WaterdataConfiguration gains api_version, which can be set in a configure() block or in the [waterdata] table of the configuration file, but not from the environment. The file now refuses the settings in BLOCK_ONLY_SETTINGS rather than all of ADAPTER_ONLY_SETTINGS: base_url can send requests to another host, and a version cannot. v1 drops four time-series-metadata queryables (begin_utc, end_utc, state_name, hydrologic_unit_code) and answers 400 or 500 to them. get_time_series_metadata sends a call that names any of them (state resolves to state_name) to v0 and emits a DeprecationWarning; the removal date is 2027-06-01, when the service retires v0. The v0 URL is set per request, through an api_version argument to ogc_api_url() and get_ogc_data(), rather than by entering configure(), so it never overrides a version the caller set. Two behavior changes, both in NEWS: time-series-metadata begin/end are UTC with a time zone, and get_field_measurements returns time as a date. v1 also adds statistics_begin to time-series-metadata. The queryables snapshot and the time-series-metadata fixture are regenerated from v1. Closes DOI-USGS#421. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Sep 24, 2026
Comparing each collection's /schema with its getter's signature found 20 returned columns that could be passed only through **queryables, so the getter documented neither the column nor the filter: - get_field_measurements: control_condition, day, field_measurements_series_id, measurement_rated, month, reading_type, time_of_day, year - get_peaks: qualifier, time_of_day, value - get_monitoring_locations: revision_created, revision_modified, revision_note - get_combined_metadata: data_gap_interval, reading_type - get_time_series_metadata: data_gap_interval, parameter_description - get_field_measurements_metadata: reading_type - get_channel: channel_location_direction Each is now a named parameter, described in the service's own words. day, month and year are typed as integers, as get_peaks already types them. The monitoring-location attributes that every collection accepts as filters but does not return stay in **queryables. Existing calls send the same request as before. Stacked on DOI-USGS#422, DOI-USGS#423 and DOI-USGS#424, which merge first; review this commit alone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
thodson-usgs
force-pushed
the
feat/waterdata-api-v1
branch
from
September 24, 2026 14:29
574f448 to
649a208
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #421.
What changes
/ogcapi/v1(release post; v0 stays online until June 2027, then redirects).OGC_API_VERSIONinwaterdata/endpoints.py, combined into the path at request time. Statistics and STAC keepv0literals: neither has a v1 (/statistics/v1and/stac/v1answer 404, probed 2026-09-22).api_versiononWaterdataConfiguration, readable asconfiguration.api_version()the waybase_url()is:get_time_series_metadatakeeps working for the four queryables v1 dropped (begin_utc,end_utc,state_name,hydrologic_unit_code, andstatesince it resolves intostate_name). v1 answers 400 or 500 to them, so a call naming one, as a filter or inproperties, is sent to v0 under aDeprecationWarning. This mirrors R dataRetrieval #926.Parameter or config? Config.
The version describes the service deployment the adapter talks to, the same category as
base_url(ADR 0010), not a property of a query. A per-call kwarg would touch every getter and collide with the OGC getters'**queryablespassthrough. SDK convention agrees: botocore, Azure, and Stripe pin a default per release and expose the override on the client or config object, never per call. R dataRetrieval does the same withoptions("dataRetrieval.api_version").Within config, the sources differ from
base_urlon purpose:base_urlapi_versionconfigure()block[waterdata]tableAPI_USGS_*the set R sharesMechanically this separates scope (adapter-only: no package-wide value exists) from provenance (which sources may supply it). Both axes were already in the vocabulary, so no third concept is added:
BLOCK_ONLY_SETTINGS = ("base_url",)keys the file's refusal, while_REFUSED_ENV_VARSstill derives fromADAPTER_ONLY_SETTINGS. A top-levelapi_versionin the file raises with the remedy (move it into[waterdata]).Behavior changes (upstream, no code change)
get_time_series_metadata:beginandendare now UTC with a time zone;begin_utc,end_utc,state_name,hydrologic_unit_codecolumns are gone.get_field_measurements:timeis a date, parsed to a tz-naive midnight asget_dailyalready does; time of day is intime_of_day. (An earlier revision of this description also namedget_latest_field_measurements; no such getter exists, andfield-measurements-metadatahas notimefield, so only the one getter is affected.)Second commit: cleanup pass
A four-angle review (reuse, simplification, efficiency, altitude) ran over the first commit. The substantive result was a fix, not a tidy:
The v0 fallback was writing on the caller's precedence ladder. It routed by entering
configure(WaterdataConfiguration(api_version="v0")), which put this library on rung 1 of ADR 0011's ladder — the caller's own rung. A caller who pinnedapi_version="v1"was silently outranked, andshow_configuration()would report a version they never asked for. The destination is already a per-call argument everywhere below that line (ogc/policy.py: "which API an OGC call targets is the adapter's policy, supplied per call asbase_url"), soogc_api_url()andwaterdata.utils.get_ogc_data()now take an optionalapi_versionand the getter pins the one request. Both configuration imports leavemetadata.py. New testtest_v0_routing_does_not_write_on_the_callers_configurationpins it.Also in that commit:
CODE_ONLY_SETTINGS→BLOCK_ONLY_SETTINGS("code" was a second identifier spelling of the_BLOCKsource, which ADR 0013 forbids for a core term); the split's rationale cut from eight restatements to one, in ADR 0011 (ADR 0000); the warning moved back to its call site instead of passing_WARN_STACKLEVEL + 1; one refusal message instead of two near-identical ones; the horizon read fromREMOVALSonce instead of also being spelled in the advisory's prose; and a comment saying why the two renames do not use_accept_legacy_kwargs.Test data that the v0→v1 move had left stale:
tests/data/waterdata_queryables.jsonwas still v0 whileQUERYABLES_REhad moved to v1, so the scheduled live monitor would have gone red. Regenerated by the documented procedure; it confirms the four dropped queryables and surfaces one addition,method_categoryoncontinuous. Thetime-series-metadataOGC fixture was regenerated from v1 for the same reason.Judgment calls to confirm
2027-06-01for the v0 shim is the service's date, not the repo's announcement + 1 year: the shim cannot outlive the endpoint it sends to.api_version(see table). If you would rather keep every adapter-only setting block-only, dropBLOCK_ONLY_SETTINGSand the file test._accept_legacy_kwargsthe default for a renamed argument. It is not used here becausebegin_utcmay also arrive inproperties, where translating forward would silently rename the returned column. Splitting behavior by call position seemed worse. Happy to switch if you read it the other way.method_categoryis a new v1 queryable oncontinuous, reachable today through**queryables. Promoting it to a documented parameter is a feature, so it is left for a follow-up.Verification
ruff,mypy --strict,lint-imports(8 kept),xenon,complexipy, and the full pre-commit suite pass. Offline suite 1192 passed, coverage 98.95% (ratchet 98.9). Docs build clean.-W error::DeprecationWarningand returns tz-awarebeginwith nobegin_utc; each legacy filter warns under the caller's own spelling and returns from v0; bothget_combined_metadataremedies return rows on v1; a caller-pinnedv1survives the getter's internal v0 routing and still applies to every other getter and toshow_configuration(); file, top-level, and environment sources behave as documented; the live queryables monitor passes against the regenerated snapshot.🤖 Generated with Claude Code