[Feature] Introduce ha flag for table materialisation and iceberg tables to allow users to control final location

[nicor88]

Is this your first time submitting a feature request?

• I have searched the existing issues, and I could not find an existing issue for this feature

Describe the feature

The current behaviour for table_type=iceberg and materialization=table implements high availability by default, this means that a new table is created with a not final name in a unique location, then the final target table is renamed. As consequence of this implementation we need to force the user of using unique locations.

What if the user don't want to have high availability and specific locations to be fully in control of where the data is stored?

Adding the possibility to use the ha flag for iceberg table allow the user to be more in control.

Here the proposal - note only for iceberg tables, as they are the affected one:

• ha=true -> follow the current implementation
• ha=false -> drop the table and create the table to the final location - doing so we allow the user to use external_location, or data_naming: table, schema_table or whatever they want.

Describe alternatives you've considered

No response

Who will this benefit?

Users that want to control final iceberg table location in S3.

Are you interested in contributing this feature?

No response

Anything else?

The current behaviour for iceberg tables is set to "ha=true", maintain this behavior to avoid breaking changes for the users.