Bump dbt-sql to DBR 16.4 LTS for classic compute - #6418
Merged
Conversation
Collaborator
Integration test reportCommit: 16953f4
Top 3 slowest tests (at least 2 minutes):
|
rugpanov
force-pushed
the
dbt-sql-versions-tmpl-followup
branch
from
August 28, 2026 09:00
c0f06e2 to
16953f4
Compare
rugpanov
changed the title
rugpanov
marked this pull request as ready for review
August 28, 2026 09:23
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Aug 31, 2026
## Changes Delete `libs/template/templates/default-sql/library/versions.tmpl`. The `default-sql` bundle template renders no cluster (it targets a SQL warehouse), so its two macros — `latest_lts_dbr_version` and `latest_lts_db_connect_version_spec` — are never referenced by any template. The file contained only these two dead macros. Split out of #6418 (dbt-sql DBR 16.4 bump) to keep that PR focused on a single change. ## Tests - No rendered template output changes (the macros were never invoked); no changelog fragment needed. - `default-sql` template acceptance tests pass unchanged after the deletion. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
Follow-up from #6378: - dbt-sql: bump latest_lts_dbr_version 15.4 -> 16.4 LTS for classic (non-serverless) compute, matching the default template's 16.4 pin and aligning the cluster runtime's Python version (3.12) with serverless environment version 5. - Remove the unused, TODO-flagged latest_lts_db_connect_version_spec macro from dbt-sql (flagged for removal in #6378; no references). - Add a dbt-sql-classic acceptance variant exercising the classic new_cluster path, locking in the bumped spark_version. Co-authored-by: Isaac <no-reply@databricks.com>
rugpanov
force-pushed
the
dbt-sql-versions-tmpl-followup
branch
from
August 31, 2026 11:26
16953f4 to
14882d1
Compare
denik
reviewed
Sep 2, 2026
It reads the sibling dbt-sql test's committed golden via $TESTDIR, so in update mode it must run after phase 0 regenerates that golden. Co-authored-by: Isaac <no-reply@databricks.com>
denik
approved these changes
Sep 2, 2026
Main added EnvMatrix.DMS to the global test matrix; refresh this test's generated out.test.toml to match so the "no files changed" CI gate passes. Co-authored-by: Isaac <no-reply@databricks.com>
rugpanov
added a commit
that referenced
this pull request
Sep 2, 2026
…omment - Drop the claims about default-sql's versions.tmpl and the SQL templates pinning an older DBR, both invalidated by #6420 and #6418 - Attribute the 16.4 DB Connect floor to the Python/serverless-env compatibility constraint instead of DBR 15's support status - Remove the comment's absolute "never bump to match the newest serverless environment", which contradicted the hard compatibility constraint - Extend the sync rule and the globs to the hardcoded environment_version and requires-python occurrences outside library/versions.tmpl - Shrink the versions.tmpl comment to the invariant plus a pointer, so the rule has a single home - Reference the rule file by repo-root path and name ./task test-update-templates Co-authored-by: Isaac <no-reply@databricks.com>
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 3, 2026
## Release v1.15.0 ### CLI * When `uv python install` fails, `databricks environments setup-local` now falls back to a compatible Python interpreter already installed on the machine. ([#6457](#6457)) * Allow `databricks environments setup-local` to update `pyproject.toml` files containing TOML multi-line strings. ([#6445](#6445)) ### Bundles * Before committing the automatic terraform→direct migration, run a deployment plan against the converted state; if the plan fails the migration is abandoned. ([#6486](#6486)) * The `dbt-sql` bundle template now uses Databricks Runtime 16.4 LTS (up from 15.4 LTS) for classic (non-serverless) compute. ([#6418](#6418)) * Fixed the direct engine silently ignoring edits to duration and timestamp fields, such as a Lakebase endpoint's `suspend_timeout_duration`. Such a change planned `0 to change` and was never applied. ([#6377](#6377)) * Fixed `$${...}` not escaping a literal `${...}` on the direct engine, which failed with an `invalid dependency` error. ([#6484](#6484), [#6489](#6489)) * Remove forward_user_access_token from update_mask for Apps because it's not supported. Fixes regression in 1.14.1. ([#6510](#6510)) * direct: Fix deploying an update to `postgres_projects.default_endpoint_settings`. ([#6440](#6440)) * direct: Fix deploying an update to `postgres_endpoints.settings.pg_settings`. ([#6441](#6441)) * direct: Fix deploying an update to `expire_time`, `ttl` or `suspend_timeout_duration` on Lakebase resources. ([#6443](#6443)) * Added PyDABs (Python) support for catalogs: `Resources.add_catalog` and the `catalog_mutator` decorator. ([#6408](#6408)) * Bundle templates now use serverless [environment version 5](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/five), which offers better performance, and `databricks-connect` 16.4. ([#6378](#6378)) * Fixed a job with a `table_update` trigger never converging on the direct engine. ([#6442](#6442)) ### Dependency Updates * Bump Go toolchain to 1.26.8. ([#6476](#6476))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Follow-up from #6378: bump the
dbt-sqlbundle template's classic (non-serverless) compute to Databricks Runtime 16.4 LTS.latest_lts_dbr_versionfrom15.4.x-scala2.12to16.4.x-scala2.12. This matches thedefaulttemplate (already pinned to 16.4) and aligns the cluster runtime's Python version (3.12) with serverless environment version 5.latest_lts_db_connect_version_specmacro (flagged for removal in Bump default serverless environment version to 5 #6378; no template references it).The unrelated dead-code removal in
default-sql/library/versions.tmplwas split into #6420 (merged); this branch is rebased on top of it.Tests
acceptance/bundle/templates/dbt-sql-classicinitializes dbt-sql withserverless: noand diffs the generated job against the serverless golden, locking in the classicnew_clusterspark_version: 16.4.x-scala2.12. Before this, no acceptance test rendered the classic (non-serverless) path.dbt-sqltest's committed golden, so any change to the basedbt-sqljob template requires regenerating both goldens.This pull request and its description were written by Isaac.