Migrate to using decoupled new decoupled core/adapters architecture

[colin-rogers-dbt]

Describe the feature

Opening this issue to track dbt-databricks decoupling migration and provide additional guidance/context.

Specific recommendations:

• In the feature branch update https://github.com/databricks/dbt-databricks/blob/main/dev-requirements.txt#L27 to install dbt-core and dbt-spark from the main branch of their respective repos instead of installing from pypi.
• Review the dbt-spark migration PR, functional changes are minor but potentially subtle. One example is this change swapping out DbtProfileError for DbtConfigError

Describe alternatives you've considered

Not doing this will block dbt-databricks from working with post 1.8 dbt-core versions.

Additional context

N/A

Who will this benefit?

All dbt-databricks users and maintainers!

Are you interested in contributing this feature?

Happy to help out however we can.

[benc-db]

Ack, thanks for filing