Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

hashicorp/go-azure-sdk refactoring for azurerm_subscriptions - pagination and modelling issue #24948

Open
1 task done
jschoombee opened this issue Feb 20, 2024 · 3 comments 路 May be fixed by hashicorp/pandora#3958
Open
1 task done
Labels
bug service/subscription upstream/go-azure-sdk This issue/PR requires a fix in `hashicorp/go-azure-sdk` upstream/pandora This issue/PR has a dependency on an issue in `github.com/hashicorp/pandora` v/3.x

Comments

@jschoombee
Copy link

jschoombee commented Feb 20, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.7.5

AzureRM Provider Version

3.92.0

Affected Resource(s)/Data Source(s)

azurerm_subscriptions

Terraform Configuration Files

data "azurerm_subscriptions" "all" {
   //searching with a few thousand subscriptions in tenant 
}

Debug Output/Panic Output

Less subscriptions in the tf state instances than expected.

Expected Behaviour

No response

Actual Behaviour

No response

Steps to Reproduce

Use a version prior to 3.90.0 and all subscriptions (>1000) are returned. This PR seemed to have created the issue

I debugged the provider locally and managed to resolve the issue by dropping the @odata prefix for nextLink here

Further count in the subscriptions response needed changing as it is not of type int. Below is what I ended up with which did the trick for me:

// OData is used to unmarshal OData metadata from an API response.
type OData struct {
	Context      *string `json:"@odata.context"`
	MetadataEtag *string `json:"@odata.metadataEtag"`
	Type         *Type   `json:"@odata.type"`
	Count        *Count  `json:"count"`
	NextLink     *Link   `json:"nextLink"`
	Delta        *Link   `json:"@odata.delta"`
	DeltaLink    *Link   `json:"@odata.deltaLink"`
	Id           *Id     `json:"@odata.id"`
	EditLink     *Link   `json:"@odata.editLink"`
	Etag         *string `json:"@odata.etag"`

	Error *Error `json:"-"`

	Value interface{} `json:"value"`
}

type Count struct {
	Type  *string `json:"type"`
	Value *int    `json:"value"`
}

Important Factoids

Have a local branch to fix this issue and happy to submit but not sure if other resources/data sources depending on this package might be affected

References

PR

@wuxu92
Copy link
Contributor

wuxu92 commented Feb 22, 2024

Hi @jschoombee ,

Thank you for reporting this issue. It appears that the OData structure lost some information during unmarshalling. Could you please provide the API response that caused the issue? (I don't have an account with multiple subscriptions to reproduce the problem.)

@jschoombee
Copy link
Author

jschoombee commented Feb 23, 2024

Hi @wuxu92,

I logged the raw responses when testing, below is a redacted snippet of the output. I haven't worked with the MS graph APIs much but it seems like OData was not implemented for subscriptions in azurerm - api ref

{
    "value": [ 
        {
            "id": "/subscriptions/11111111-9f71-4226-ae1b-7995c4af4ccc",
            "authorizationSource": "RoleBased",
            "managedByTenants": [],
            "tags": {
                "redacted": 1
            },
            "subscriptionId": "11111111-9f71-4226-ae1b-7995c4af4ccc",
            "tenantId": "11111111-a3da-442a-1111-235cac24cd5c",
            "displayName": "redacted",
            "state": "Enabled",
            "subscriptionPolicies": {
                "locationPlacementId": "Public_2014-09-01",
                "quotaId": "EnterpriseAgreement_2014-09-01",
                "spendingLimit": "Off"
            }
        }
    ],
    "nextLink": "https://management.azure.com/subscriptions?$skipToken=redacted%3d&api-version=2022-12-01",
    "count": {
        "type": "Total",
        "value": 5297
    }
}

@BigFrog-coding
Copy link

Is there any progress on this? This is a blocking issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug service/subscription upstream/go-azure-sdk This issue/PR requires a fix in `hashicorp/go-azure-sdk` upstream/pandora This issue/PR has a dependency on an issue in `github.com/hashicorp/pandora` v/3.x
Projects
None yet
5 participants