fix(feature_store): register HubContent Dataset from DatasetBuilder CSV path - #6212
Conversation
…SV path DatasetBuilder defined _register_as_hub_content_dataset and the register_as_dataset flag, but _to_csv_from_feature_group never invoked the helper. Wire the call into the CSV extraction path, gated on register_as_dataset, passing the Athena QueryExecutionId. Add unit tests asserting the helper is invoked when the flag is set and skipped when it is not.
|
Claude finished @Vishakha263's task in 1m 6s —— View job Claude review:
|
Description
DatasetBuilderalready defines theregister_as_datasetflag and the_register_as_hub_content_datasethelper, but_to_csv_from_feature_groupreturnedself._extract_result(result)directly and never invoked the helper. As a result, passingregister_as_dataset=Truehad no effect on the CSV extraction path.This change wires the helper into
_to_csv_from_feature_group, gated onregister_as_dataset, and passes the AthenaQueryExecutionIdthrough so it is recorded in the Dataset content metadata.Changes
dataset_builder.py: after extracting the CSV result, call_register_as_hub_content_dataset(csv_path, query_execution_id)whenregister_as_datasetis set.test_dataset_builder.py: add two unit tests asserting the helper is invoked when the flag is set and skipped when it is not.Testing
pytest tests/unit/sagemaker/mlops/feature_store/test_dataset_builder.py— 39 passed.black --line-length 100 --checkandflake8 --max-line-length 100clean on the changed lines.Merge Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.