Skip to content

bundle: warn when a task parameter is an empty string under the terraform engine#5951

Draft
ankit481 wants to merge 2 commits into
databricks:mainfrom
ankit481:warn-empty-task-parameters
Draft

bundle: warn when a task parameter is an empty string under the terraform engine#5951
ankit481 wants to merge 2 commits into
databricks:mainfrom
ankit481:warn-empty-task-parameters

Conversation

@ankit481

@ankit481 ankit481 commented Jul 17, 2026

Copy link
Copy Markdown

Changes

Adds a validate-time warning when a task's parameters list contains an empty string element and the bundle deploys with the terraform engine. Such deploys currently pass bundle validate and then fail while creating the job with a provider panic:

Error: cannot create job: panic: task: spark_python_task: parameters: task.0.spark_python_task.0.parameters[<nil>] is not a string

The warning names the config location of the empty element and suggests removing it or deploying with the direct engine, which handles the same configuration correctly. It is emitted only when the effective engine is not direct, following the same engine resolution as validate.TFOnlyReferences.

This is a draft on purpose. The underlying defect looks like it sits in the terraform provider's schema read layer (the generated bundle.tf.json is well formed and renders the element as ""), so the real fix may belong in terraform-provider-databricks. This PR covers the CLI side: catching it at validate time instead of letting terraform apply crash. Direction welcome on warning versus error, the exact wording, and whether you would rather see only the regression tests merged here.

Related: #5950. The warning is a guard on the CLI side; the provider panic itself remains open.

Why

An empty parameter usually comes from a bundle variable that is deliberately blank in some targets, for example a per-developer prefix that is empty in shared environments. That configuration is legal, validate passes, and the crash appears only at deploy time in whichever target leaves the variable empty. We hit this four times over two months before spotting the pattern. A warning at validate time with the file and line turns an opaque provider panic into something the user can act on before deploying.

Tests

  • Unit tests for the new validator: warnings for spark_python_task, python_wheel_task, and a for_each_task nested spark_jar_task under the terraform engine, and silence under the direct engine or when no parameter is empty.
  • Acceptance test bundle/variables/empty-in-task-params runs validate and deploy under both engines. The direct variant deploys successfully and jobs/create receives the empty string. The terraform variant records the new warning and the current provider panic, and also records the parameters as rendered into bundle.tf.json to show the generated configuration is well formed.
  • Acceptance test bundle/variables/empty-in-task-params-variants pins that every empty-variable shape (short, long, and null target overrides, BUNDLE_VAR_*, --var, a variable referencing another variable, and a complex variable field) resolves to an empty string rather than null.

ankit481 added 2 commits July 16, 2026 22:55
…form engine

An empty string element in a task parameters list passes bundle validate but
fails bundle deploy under the terraform engine: the provider reads the element
back as nil and job creation panics with "parameters[<nil>] is not a string".
The generated bundle.tf.json is well formed (the element renders as ""), and
the direct engine deploys the same configuration correctly, so the defect sits
in the provider layer. See databricks#5950.

This adds a validate-time warning that names the config location of the empty
parameter and suggests removing it or using the direct engine. The warning is
only emitted when the effective engine is not direct.

Acceptance tests cover both engines: the direct variant deploys successfully
and jobs/create receives the empty string; the terraform variant records the
warning and the current provider panic. A second validate-only test pins that
every empty-variable shape (target overrides, BUNDLE_VAR, --var, var-in-var,
complex field) resolves to an empty string rather than null.
@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 5951
  • Commit SHA: b7d28b8eb3e3f75440b53c737926d033c11af1ee

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant