docs: documenting the different types of Mappings #3266

tombuildsstuff · 2023-11-01T14:31:42Z

This PR adds a new internals piece of documentation covering Mappings, used to map the Terraform Schema onto the Types used in the SDK and back again.

stephybun

Thanks @tombuildsstuff for the detailed explanation and examples.

I left some suggestions which would be good to have addressed before we merge this.

stephybun · 2023-12-06T13:10:06Z

docs/README.md

@@ -34,6 +34,7 @@ Specific information on each tool can be found in the README for each tool, [for
 * [How to add a new Common ID](common-ids.md).
 * [Resource Manager: How to import a new API Version or Service into Pandora](resource-manager-service-import.md).
 * [Resource Manager: Generating a new Resource](resource-manager-generate-new-resource.md).
+* [Internals: Types of (Terraform) Mappings](internals-mappings.md)


I think it makes sense to have a clear distinction between information that is user facing like guides and information non-user facing, relating to the inner workings of Pandora.

So can we have a separate grouping on this page for things like architecture/technical details and move this there?

stephybun · 2023-12-06T13:24:55Z

docs/internals-mappings.md

+
+In each case, these map the fields in the Terraform Schema onto either the SDK Models - or the Resource ID for a given Resource.
+
+Whilst Mappings define how to map a Type of Field from the Schema to another Field, mapping the value of those fields is left to the Terraform Generator, for example where one field is Required and another is Optional - the Terraform Generator should handle passing either a reference/the value as needed.


What is meant by type of Field. It looks like the types are mentioned down below e.g. DirectAssignment, ModelToModel etc. but that's not clear here.

stephybun · 2023-12-06T13:31:45Z

docs/internals-mappings.md

+
+### a. `DirectAssignment` Field Mappings
+
+A `DirectAssignment` mapping specifies that the value of a Field within the Terraform Schema should be mapped onto the value of a Field within the related (Go) SDK Model - and vica-versa.


Suggested change

A `DirectAssignment` mapping specifies that the value of a Field within the Terraform Schema should be mapped onto the value of a Field within the related (Go) SDK Model - and vica-versa.

A `DirectAssignment` mapping specifies that the value of a Field within the Terraform Schema should be mapped onto the value of a Field within the related (Go) SDK Model and vice-versa.

stephybun · 2023-12-06T13:35:12Z

docs/internals-mappings.md

+
+ModelToModel Field Mappings - not to be confused with ModelToModel Mappings (below) - allow mapping a Terraform Schema Model onto a field within an SDK Model.
+
+The best example of this is allowing the top-level Terraform Schema Model for a given Terraform Resource to map onto the `properties` object that exists in most payloads in Azure Resource Manager - allowing for fields which exist at the root of the Terraform Schema to be mapped to a nested object.


I think it would be helpful to give explicit examples here for what top-level Terraform Schema Models you mean, e.g. location? tags?

stephybun · 2023-12-06T13:37:14Z

docs/internals-mappings.md

+
+An example of a Direct Assignment Mapping can be seen below:
+
+> **Note:** A more complete example can be seen under `Examples` below.


I understand this is trying to be helpful but repeating it multiple times throughout this guide is unnecessary bloat and noise for a topic that's complex. Say it once, probably at the top of the document.

stephybun · 2023-12-06T13:48:15Z

docs/internals-mappings.md

+
+### c. `Manual` Field Mappings
+
+> **Note:** At this point in time Manual Mappings are only partially configured and **need reworking** - as such this section explains the intention this type of Mapping, so that we can fix this in the future.


Suggested change

> **Note:** At this point in time Manual Mappings are only partially configured and **need reworking** - as such this section explains the intention this type of Mapping, so that we can fix this in the future.

> **Note:** At this point in time Manual Mappings are only partially configured and **need reworking** - as such this section explains the intention of this type of Mapping, so that we can fix this in the future.

stephybun · 2023-12-06T13:49:54Z

docs/internals-mappings.md

+* `sdkFieldPath`- specifies the path to the Field within the SDK Model where the Schema Field should be mapped to/from.
+* `sdkToSchemaMethodName` - specifies the name of a function that should be assumed to exist within the Terraform Provider, which maps the value of the SDK Field onto the Terraform Schema Field.
+
+> **Note:** The current implementation that exists exposed only `methodName` - as such is only useful for one-way value transformations.


Suggested change

> **Note:** The current implementation that exists exposed only `methodName` - as such is only useful for one-way value transformations.

> **Note:** The current implementation that exists exposes only `methodName` and is only useful for one-way value transformations.

stephybun · 2023-12-06T13:50:54Z

docs/internals-mappings.md

+
+> **Note:** The current implementation that exists exposed only `methodName` - as such is only useful for one-way value transformations.
+
+## 2: Model to Model Mappings


Inconsistent with the other headers in this doc

Suggested change

## 2: Model to Model Mappings

## 2: `ModelToModel` Mappings

stephybun · 2023-12-06T13:51:16Z

docs/internals-mappings.md

+
+> **Note:** A ModelToModel Mapping and a ModelToModel Field Mapping are related but different - a ModelToModel Field Mapping _must_ have an associated ModelToModel Mapping - however a ModelToModel mapping can exist without a ModelToModel Field Mapping.
+
+Model to Model Mappings support the following properties:


Suggested change

Model to Model Mappings support the following properties:

ModelToModel Mappings support the following properties:

stephybun · 2023-12-06T13:58:01Z

docs/internals-mappings.md

+
+This would allow the Terraform Generator to output mapping functions similar to below:
+
+> Whilst mappings are bidirectional (that is Schema Model -> SDK Model and SDK Model -> Schema Model) - to reduce the amount of output only the Schema Model -> SDK Model mappings are shown below - but the inverse can be implied.


Leaving things to be implied for complex subjects will lead to misinterpretations, misunderstandings and errors as we've learnt. For completeness the SDK Model -> Schema Model mappings should be included.

tombuildsstuff · 2024-05-23T16:52:23Z

In the end we ended up deciding to document and rename these inline within the new SDK types, so I'm going to close this out:

stephybun · 2024-05-24T05:18:24Z

@tombuildsstuff I believe what's written here is valuable and helpful for understanding mappings and the detailed explanations and examples in it are not substituted by the comments left in the code that are linked in your closing comment for this PR.

I've restored this branch since I don't believe this information should be deleted entirely and would like this to be re-opened, without the expectation that you are required to finish this off.

docs: documenting the different types of Mappings

7230777

tombuildsstuff requested a review from a team November 1, 2023 14:33

tombuildsstuff mentioned this pull request Nov 1, 2023

Data API v2 bug: missing ModelToModel mapping for KubernetesClusterTrustedAccessRoleBinding #3267

Closed

docs: adding the new doc to the summary

d4e9ce5

stephybun linked an issue Dec 4, 2023 that may be closed by this pull request

Documentation: Mappings #3438

Closed

stephybun requested changes Dec 6, 2023

View reviewed changes

stephybun mentioned this pull request Dec 6, 2023

documentation: documenting the current architecture/workflow and summarising the proposed changes to the Data API going forward #3425

Closed

tombuildsstuff closed this May 23, 2024

tombuildsstuff deleted the docs/mappings branch May 23, 2024 16:52

stephybun restored the docs/mappings branch May 24, 2024 05:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: documenting the different types of Mappings #3266

docs: documenting the different types of Mappings #3266

tombuildsstuff commented Nov 1, 2023

stephybun left a comment

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

stephybun Dec 6, 2023

tombuildsstuff commented May 23, 2024

stephybun commented May 24, 2024


		In each case, these map the fields in the Terraform Schema onto either the SDK Models - or the Resource ID for a given Resource.

		Whilst Mappings define how to map a Type of Field from the Schema to another Field, mapping the value of those fields is left to the Terraform Generator, for example where one field is Required and another is Optional - the Terraform Generator should handle passing either a reference/the value as needed.


		### a. `DirectAssignment` Field Mappings

		A `DirectAssignment` mapping specifies that the value of a Field within the Terraform Schema should be mapped onto the value of a Field within the related (Go) SDK Model - and vica-versa.


		ModelToModel Field Mappings - not to be confused with ModelToModel Mappings (below) - allow mapping a Terraform Schema Model onto a field within an SDK Model.

		The best example of this is allowing the top-level Terraform Schema Model for a given Terraform Resource to map onto the `properties` object that exists in most payloads in Azure Resource Manager - allowing for fields which exist at the root of the Terraform Schema to be mapped to a nested object.


		An example of a Direct Assignment Mapping can be seen below:

		> Note: A more complete example can be seen under `Examples` below.


		### c. `Manual` Field Mappings

		> Note: At this point in time Manual Mappings are only partially configured and need reworking - as such this section explains the intention this type of Mapping, so that we can fix this in the future.

	> Note: The current implementation that exists exposed only `methodName` - as such is only useful for one-way value transformations.
	> Note: The current implementation that exists exposes only `methodName` and is only useful for one-way value transformations.


		> Note: The current implementation that exists exposed only `methodName` - as such is only useful for one-way value transformations.

		## 2: Model to Model Mappings


		> Note: A ModelToModel Mapping and a ModelToModel Field Mapping are related but different - a ModelToModel Field Mapping _must_ have an associated ModelToModel Mapping - however a ModelToModel mapping can exist without a ModelToModel Field Mapping.

		Model to Model Mappings support the following properties:

	Model to Model Mappings support the following properties:
	ModelToModel Mappings support the following properties:


		This would allow the Terraform Generator to output mapping functions similar to below:

		> Whilst mappings are bidirectional (that is Schema Model -> SDK Model and SDK Model -> Schema Model) - to reduce the amount of output only the Schema Model -> SDK Model mappings are shown below - but the inverse can be implied.

docs: documenting the different types of Mappings #3266

docs: documenting the different types of Mappings #3266

Conversation

tombuildsstuff commented Nov 1, 2023

stephybun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tombuildsstuff commented May 23, 2024

stephybun commented May 24, 2024