Skip to content
Open
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

## Unreleased

- Kernel backend (`useKernel: true`): preserve qualified `INTERVAL MONTH` and
Comment thread
cathleeny marked this conversation as resolved.
`INTERVAL DAY` parameter types on the SEA wire by using the kernel raw-parameter
path.
- Kernel backend source builds (`useKernel: true`, built from `KERNEL_REV`): `getTypeInfo()` now matches the Thrift backend's canonical 18-column, 20-row type-info result. Customer-facing npm installs require a follow-up bump to a published native package containing this Kernel change. ([databricks-sql-kernel#291](https://github.com/databricks/databricks-sql-kernel/pull/291), PECOBLR-4166)
- Kernel backend (`useKernel: true`): **Azure Entra (Azure AD) auth is now threaded through the kernel path.** On `authType: 'databricks-oauth'`: **U2M** (no secret) always routes to `OAuthU2m` — the kernel runs one cloud-blind in-house workspace-federated browser flow (it uses the workspace's OIDC-discovered authorize endpoint verbatim), which works against Azure workspaces, so Azure U2M forwards the in-house app (`databricks-sql-connector`) + `sql offline_access` scopes exactly like AWS/GCP, regardless of `useDatabricksOAuthInAzure` (verified E2E against a live Azure workspace). **M2M** (secret): `useDatabricksOAuthInAzure: true` (or non-Azure) → `OAuthM2m` (workspace-OIDC client-credentials); an Azure host with `useDatabricksOAuthInAzure` absent/`false` → the Entra-direct Azure service-principal M2M (`AzureSpM2m`, the Entra SP creds ride `oauthClientId`/`oauthClientSecret`, `azureTenantId` optional and auto-discovered when omitted). On a non-Azure host `useDatabricksOAuthInAzure` is inert. The `AzureSpM2m` path requires a `databricks-sql-kernel` native module that exposes the Azure SP surface — landed on `main` via [databricks-sql-kernel#282](https://github.com/databricks/databricks-sql-kernel/pull/282) (which the pinned `KERNEL_REV` `ef1a6f2` carries; the surface was originally proposed in [#280](https://github.com/databricks/databricks-sql-kernel/pull/280), which never reached `main`); U2M works on any kernel build. (PECOBLR-4141 / PECOBLR-4120)

Expand Down
2 changes: 1 addition & 1 deletion KERNEL_REV
Original file line number Diff line number Diff line change
@@ -1 +1 @@
628abd6f5045897efcadb38ec77a1e9e0c23544e
bb4a0770673926201d386d1d9295da8d4bf459aa
16 changes: 9 additions & 7 deletions lib/kernel/KernelNativeLoader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,15 @@ export type KernelArrowSchema = NativeArrowSchema;
export type KernelConnection = NativeConnection;
export type KernelStatement = NativeStatement;

// Per-statement execution options and bound-parameter inputs are kernel
// concerns: the napi binding generates the canonical shapes (`positionalParams`
// / `namedParams` as `TypedValueInput` / `NamedTypedValueInput`, plus
// `rowLimit`, `statementConf`, `queryTags`). We re-export
// rather than re-declare so the driver-side param codec can never drift from
// the kernel contract.
export type KernelNativeExecuteOptions = NativeExecuteOptions;
// rawParams is available in the source-pinned kernel before its published types.
export interface KernelNativeRawParameterInput {
name?: string;
sqlType: string;
value?: string;
}
export type KernelNativeExecuteOptions = NativeExecuteOptions & {
rawParams?: KernelNativeRawParameterInput[];
};
export type KernelNativeTypedValueInput = NativeTypedValueInput;
export type KernelNativeNamedTypedValueInput = NativeNamedTypedValueInput;

Expand Down
31 changes: 14 additions & 17 deletions lib/kernel/KernelPositionalParams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

import { DBSQLParameter, DBSQLParameterValue } from '../DBSQLParameter';
import ParameterError from '../errors/ParameterError';
import { KernelNativeTypedValueInput, KernelNativeNamedTypedValueInput } from './KernelNativeLoader';
import { KernelNativeRawParameterInput } from './KernelNativeLoader';
import assertBindableValue from './KernelInputValidation';

/**
Expand Down Expand Up @@ -58,15 +58,14 @@ function decimalPrecisionScale(v: string): string {

/**
* Reduce a `DBSQLParameter | DBSQLParameterValue` to the napi
* `TypedValueInput` (`{ sqlType, value? }`) the kernel's positional-param
* codec (`parse_typed_value`) accepts. Reuses `DBSQLParameter.toSparkParameter`
* — the same type-inference + value-stringification the Thrift backend uses —
* then adapts the type name to the codec's expectations:
* `RawParameterInput` (`{ name?, sqlType, value? }`) accepted by the kernel's
* raw-parameter path. Reuses `DBSQLParameter.toSparkParameter` — the same
* type-inference + value-stringification the Thrift backend uses — then adapts
* the type name where required:
* - DECIMAL → `DECIMAL(p,s)` (parenthesised form required)
* - INTERVAL * → `INTERVAL` (the codec's single interval type name)
* - a missing value ⇒ SQL NULL (`parse_typed_value` maps `value: None` to NULL).
* - a missing value ⇒ SQL NULL.
*/
function toTypedValueInput(value: DBSQLParameter | DBSQLParameterValue): KernelNativeTypedValueInput {
function toRawParameterInput(value: DBSQLParameter | DBSQLParameterValue): KernelNativeRawParameterInput {
const param = value instanceof DBSQLParameter ? value : new DBSQLParameter({ value });
const spark = param.toSparkParameter();
const stringValue = spark.value?.stringValue ?? undefined;
Expand All @@ -81,44 +80,42 @@ function toTypedValueInput(value: DBSQLParameter | DBSQLParameterValue): KernelN
const upper = sqlType.toUpperCase();
if (upper === 'DECIMAL') {
sqlType = `DECIMAL(${decimalPrecisionScale(stringValue)})`;
} else if (upper.startsWith('INTERVAL')) {
sqlType = 'INTERVAL';
}
return { sqlType, value: stringValue };
}

/**
* Convert the public `ordinalParameters` option into the napi
* `positionalParams` array (1-based `?` placeholders). Returns `undefined`
* `rawParams` array (1-based `?` placeholders). Returns `undefined`
* when none were supplied, so the caller can keep the minimal no-options
* call shape.
*/
export function buildKernelPositionalParams(
ordinalParameters?: Array<DBSQLParameter | DBSQLParameterValue>,
): Array<KernelNativeTypedValueInput> | undefined {
): Array<KernelNativeRawParameterInput> | undefined {
if (ordinalParameters === undefined || ordinalParameters.length === 0) {
return undefined;
}
return ordinalParameters.map((value, i) => {
assertBindableValue(value, `ordinalParameters[${i}]`);
return toTypedValueInput(value);
return toRawParameterInput(value);
});
}

/**
* Convert the public `namedParameters` option (`Record<name, value>`) into
* the napi `namedParams` array (`:name` placeholders). Each value reuses the
* same `toTypedValueInput` mapping (DECIMAL → DECIMAL(p,s), NULL → VOID, …),
* the napi `rawParams` array (`:name` placeholders). Each value reuses the
* same `toRawParameterInput` mapping (DECIMAL → DECIMAL(p,s), NULL → VOID, …),
* then carries its name. Returns `undefined` when none were supplied.
*/
export function buildKernelNamedParams(
namedParameters?: Record<string, DBSQLParameter | DBSQLParameterValue>,
): Array<KernelNativeNamedTypedValueInput> | undefined {
): Array<KernelNativeRawParameterInput> | undefined {
if (namedParameters === undefined || Object.keys(namedParameters).length === 0) {
return undefined;
}
return Object.keys(namedParameters).map((name) => {
assertBindableValue(namedParameters[name], `namedParameters[${name}]`);
return { name, ...toTypedValueInput(namedParameters[name]) };
return { name, ...toRawParameterInput(namedParameters[name]) };
});
}
9 changes: 4 additions & 5 deletions lib/kernel/KernelSessionBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -298,11 +298,10 @@ export default class KernelSessionBackend implements ISessionBackend {
}

const execOptions: KernelNativeExecuteOptions = {};
if (positionalParams !== undefined) {
execOptions.positionalParams = positionalParams;
}
if (namedParams !== undefined) {
execOptions.namedParams = namedParams;
// Raw binding preserves qualified SQL types such as INTERVAL MONTH.
const rawParams = positionalParams ?? namedParams;
if (rawParams !== undefined) {
execOptions.rawParams = rawParams;
}
// NB: `queryTimeout` is intentionally NOT forwarded — it is a no-op on kernel
// (SQL Warehouses use `STATEMENT_TIMEOUT`; mapping it to `wait_timeout` would
Expand Down
86 changes: 85 additions & 1 deletion tests/e2e/kernel/execution-e2e.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// limitations under the License.

import { expect } from 'chai';
import { DBSQLClient } from '../../../lib';
import { DBSQLClient, DBSQLParameter, DBSQLParameterType } from '../../../lib';
import { ConnectionOptions } from '../../../lib/contracts/IDBSQLClient';
import { InternalConnectionOptions } from '../../../lib/contracts/InternalConnectionOptions';

Expand Down Expand Up @@ -121,4 +121,88 @@ describe('kernel execution end-to-end', function e2eSuite() {
await session.close();
await client.close();
});

it('binds ordinary positional parameters through rawParams', async () => {
const client = new DBSQLClient();

await client.connect({
host: hostName as string,
path: httpPath as string,
token: token as string,
useKernel: true,
} as ConnectionOptions & InternalConnectionOptions);

const session = await client.openSession({ initialCatalog: 'main' });
let operation;
try {
operation = await session.executeStatement('SELECT ? AS p_int, ? AS p_string, ? AS p_bool', {
ordinalParameters: [42, 'hello', true],
});
expect(await operation.fetchAll()).to.deep.equal([{ p_int: 42, p_string: 'hello', p_bool: true }]);
} finally {
await operation?.close();
await session.close();
await client.close();
}
});

it('binds named NULL and empty string through rawParams on the async path', async () => {
const client = new DBSQLClient();

await client.connect({
host: hostName as string,
path: httpPath as string,
token: token as string,
useKernel: true,
} as ConnectionOptions & InternalConnectionOptions);

const session = await client.openSession({ initialCatalog: 'main' });
let operation;
try {
operation = await session.executeStatement('SELECT :null_value AS null_value, :empty_value AS empty_value', {
namedParameters: { null_value: null, empty_value: '' },
runAsync: true,
});
expect(await operation.fetchAll()).to.deep.equal([{ null_value: null, empty_value: '' }]);
} finally {
await operation?.close();
await session.close();
await client.close();
}
});

it('preserves INTERVAL MONTH on the SEA wire', async () => {
const client = new DBSQLClient();

await client.connect({
host: hostName as string,
path: httpPath as string,
token: token as string,
useKernel: true,
} as ConnectionOptions & InternalConnectionOptions);

const session = await client.openSession({ initialCatalog: 'main' });
let operation;
let caught: unknown;
try {
operation = await session.executeStatement('SELECT ?', {
ordinalParameters: [
new DBSQLParameter({
type: DBSQLParameterType.INTERVALMONTH,
value: '2-6',
}),
],
});
await operation.fetchAll();
} catch (error) {
caught = error;
} finally {
await operation?.close();
await session.close();
await client.close();
}

// "2-6" is valid YEAR TO MONTH syntax, but invalid for INTERVAL MONTH.
Comment thread
cathleeny marked this conversation as resolved.
Comment thread
cathleeny marked this conversation as resolved.
expect(caught).to.be.instanceOf(Error);
});
});
46 changes: 37 additions & 9 deletions tests/unit/kernel/execution.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import ParameterError from '../../../lib/errors/ParameterError';
import OperationStateError, { OperationStateErrorCode } from '../../../lib/errors/OperationStateError';
import { ConnectionOptions } from '../../../lib/contracts/IDBSQLClient';
import { OperationState } from '../../../lib/contracts/OperationStatus';
import { DBSQLParameter, DBSQLParameterType } from '../../../lib/DBSQLParameter';

// -----------------------------------------------------------------------------
// Fakes — minimal stand-ins for the napi-rs generated surface and the
Expand Down Expand Up @@ -708,26 +709,53 @@ describe('KernelSessionBackend', () => {
expect(connection.statementToReturn.cancelled, 'cancel reaches the terminal statement').to.equal(true);
});

it('executeStatement forwards ordinalParameters as napi positionalParams', async () => {
it('executeStatement forwards ordinalParameters through napi rawParams', async () => {
const connection = new FakeNativeConnection();
const session = makeSession(connection);
await session.executeStatement('SELECT ?', { ordinalParameters: [42, 'hi'] });
const options = connection.lastOptions as { positionalParams?: Array<{ sqlType: string; value?: string }> };
const options = connection.lastOptions as { rawParams?: Array<{ sqlType: string; value?: string }> };
expect(options, 'options should be passed').to.not.equal(undefined);
expect(options.positionalParams).to.have.length(2);
expect(options.positionalParams?.[0]).to.deep.equal({ sqlType: 'INTEGER', value: '42' });
expect(options.positionalParams?.[1]).to.deep.equal({ sqlType: 'STRING', value: 'hi' });
expect(options.rawParams).to.have.length(2);
expect(options.rawParams?.[0]).to.deep.equal({ sqlType: 'INTEGER', value: '42' });
expect(options.rawParams?.[1]).to.deep.equal({ sqlType: 'STRING', value: 'hi' });
});

it('executeStatement forwards namedParameters as napi namedParams (:name carried)', async () => {
it('executeStatement forwards namedParameters through napi rawParams (:name carried)', async () => {
const connection = new FakeNativeConnection();
const session = makeSession(connection);
await session.executeStatement('SELECT :x', { namedParameters: { x: 7 } });
const options = connection.lastOptions as {
namedParams?: Array<{ name: string; sqlType: string; value?: string }>;
rawParams?: Array<{ name?: string; sqlType: string; value?: string }>;
};
expect(options.namedParams).to.have.length(1);
expect(options.namedParams?.[0]).to.deep.equal({ name: 'x', sqlType: 'INTEGER', value: '7' });
expect(options.rawParams).to.have.length(1);
expect(options.rawParams?.[0]).to.deep.equal({ name: 'x', sqlType: 'INTEGER', value: '7' });
});

it('executeStatement (runAsync: true) forwards named NULL and empty string through napi rawParams', async () => {
const connection = new FakeNativeConnection();
const session = makeSession(connection);
await session.executeStatement('SELECT :null_value, :empty_value', {
namedParameters: { null_value: null, empty_value: '' },
runAsync: true,
});
const options = connection.lastOptions as {
rawParams?: Array<{ name?: string; sqlType: string; value?: string }>;
};
expect(connection.lastAsyncStatement, 'runAsync should use submitStatement').to.not.equal(undefined);
expect(options.rawParams).to.deep.equal([
{ name: 'null_value', sqlType: 'VOID' },
{ name: 'empty_value', sqlType: 'STRING', value: '' },
]);
});

it('executeStatement preserves a qualified INTERVAL type in napi rawParams', async () => {
const connection = new FakeNativeConnection();
const session = makeSession(connection);
await session.executeStatement('SELECT ?', {
ordinalParameters: [new DBSQLParameter({ type: DBSQLParameterType.INTERVALMONTH, value: '2-6' })],
});
const options = connection.lastOptions as { rawParams?: Array<{ sqlType: string; value?: string }> };
expect(options.rawParams).to.deep.equal([{ sqlType: 'INTERVAL MONTH', value: '2-6' }]);
});

it('executeStatement sends no options object on the no-params path', async () => {
Expand Down
16 changes: 12 additions & 4 deletions tests/unit/kernel/positionalParams.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,15 +63,15 @@ describe('KernelPositionalParams.buildKernelPositionalParams', () => {
expect(decimal('')).to.throw(ParameterError, /not a plain decimal numeral/);
});

it('collapses every INTERVAL subtype to the kernel codec\'s single "INTERVAL" type name', () => {
it('preserves qualified INTERVAL types for the kernel raw binder', () => {
expect(
buildKernelPositionalParams([
new DBSQLParameter({ type: DBSQLParameterType.INTERVALMONTH, value: '13' }),
new DBSQLParameter({ type: DBSQLParameterType.INTERVALMONTH, value: '2-6' }),
new DBSQLParameter({ type: DBSQLParameterType.INTERVALDAY, value: '1 02:03:04' }),
]),
).to.deep.equal([
{ sqlType: 'INTERVAL', value: '13' },
{ sqlType: 'INTERVAL', value: '1 02:03:04' },
{ sqlType: 'INTERVAL MONTH', value: '2-6' },
{ sqlType: 'INTERVAL DAY', value: '1 02:03:04' },
]);
});

Expand Down Expand Up @@ -127,4 +127,12 @@ describe('KernelPositionalParams.buildKernelNamedParams', () => {
it('maps a named NULL to a value-less VOID input (with the name)', () => {
expect(buildKernelNamedParams({ x: null })).to.deep.equal([{ name: 'x', sqlType: 'VOID' }]);
});

it('preserves a named qualified INTERVAL type', () => {
expect(
buildKernelNamedParams({
duration: new DBSQLParameter({ type: DBSQLParameterType.INTERVALMONTH, value: '2-6' }),
}),
).to.deep.equal([{ name: 'duration', sqlType: 'INTERVAL MONTH', value: '2-6' }]);
});
});
Loading