Steps to Reproduce
- The
prod environment is initially planned with vde_mode = All
vde_mode is set to dev_only
- A plan of a dev environment will assume that prod is set as physical models, and perform table copy operations as if this is the case, despite
prod still being in vde_mode = All.
For Databricks, this can cause errors as SQLMesh attempts to clone the prod environment assuming it is a table, and fails with [DELTA_CLONE_UNSUPPORTED_SOURCE] as the prod model is still a view at this point.
Location in Code
The VDE mode is set in the self.config object of the loader, which is set as self.context.configs[self.config_path] in sqlmesh/core/loader.py::Loader::__init__.
Proposed Solution
When in a plan or run, VDE mode should be derived from both the config, and the current state of the prod environment in the target gateway. Potentially a warning should be raised when they do not agree.
Steps to Reproduce
prodenvironment is initially planned withvde_mode = Allvde_modeis set todev_onlyprodstill being invde_mode = All.For Databricks, this can cause errors as SQLMesh attempts to clone the
prodenvironment assuming it is a table, and fails with[DELTA_CLONE_UNSUPPORTED_SOURCE]as theprodmodel is still a view at this point.Location in Code
The VDE mode is set in the
self.configobject of the loader, which is set asself.context.configs[self.config_path]insqlmesh/core/loader.py::Loader::__init__.Proposed Solution
When in a plan or run, VDE mode should be derived from both the config, and the current state of the
prodenvironment in the target gateway. Potentially a warning should be raised when they do not agree.