-
Notifications
You must be signed in to change notification settings - Fork 21.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry" #20918
Comments
Thanks for the question! We are investigating and will update you shortly. |
@starforce I tried out in my environment. Got the same error. |
Thanks for confirming you get the same error @jakaruna-MSFT @hrushib could you take a look at the doc and help us figure out why it doesnt work? |
@starforce, can you attach your cluster manifest and the PS command/script used. |
@starforce @hrushib @MicahMcKittrick-MSFT Mutual Authentication. " https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-security @MicahMcKittrick-MSFT Please close this. |
@hrushib @MicahMcKittrick-MSFT @aljo-microsoft Seems if I didn't have permission to execute the Powershell script that I should not get a thumbprint error . I can connect to my cluster using Connect-ServiceFabricCluster but don't have rights to use the Invoke-WebRequest command? Doesn't seem right. I am the owner of my AZURE portal and I set myself up as the administrator of my service fabric cluster. I have all security rights and privileges to do anything in my portal. I can upload code to the service fabric cluster and have the ability to use service fabric Explorer to manage the cluster which shows that I have admin rights. When I connect to the service fabric cluster I use the email account associated with the owner of the Azure portal. So the question is how come this isn't working. We're going live in three weeks yet, I still don't have a backup solution for my .NET 2.0 core Stateful Services running on the cluster. All you are doing is cutting and pasting documentation that I've already read but you're not giving me an answer to my question of why this isn't working. Thank you in advance |
Do you still want my cluster manifest and script? |
@starforce @hrushib |
@starforce, yes I want to have look at cluster manifest and the command you are using. Please share your cluster manifest and the command so as to have meaningful discussion. @aljo-microsoft, at the moment, there is no explicit PS cmdlets for Backup Restore APIs, those are only available as REST APIs. So the links shared by you above won't be useful in this context. |
@hrushib Using that command and parameter, against a publicly accessible party cluster runtime API yesterday, I could succeeded. The user also stated he had no actual issue publishing his application to his cluster from VS for Dev\Test; @starforce Nothing will prevent you from declaring the same certificate as both your server cert and client identity cert; though not recommended for production. Recommendation is to use AAD to provide a client id during FabricClient and FabricGateway mutual authentication. Refs: So @starforce and @hrushib am I missing something, what other questions/issues was not addressed? |
@hrushib is there an email address that i can send the manifest and PowerShell scripts to you? I don't want to publish my production Manifest for everyone to see. |
my email is aljo@microsoft.com @jakaruna-MSFT I published an application from VS to my cluster without issue. I deleted that application from my cluster using PowerShell SF Module without issue. I invoked my cluster SF API to get cluster health without issues using the following command: Invoke-WebRequest -Uri "https://sfbpcluster.westus.cloudapp.azure.com:19080/`$/GetClusterHealth?api-version=6.0" -CertificateThumbprint "MY THUMBPRINT" |
Just sent Script and Manifest |
@starforce @hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT I reproduced the error by removing my client certificate from 'My' Store. E.G. Get-ChildItem Cert:\CurrentUser\My\${THUMBPRINT} | Remove-Item Run the following to solve your problem: Please close this. |
Just as a sanity check, since you are presumably copy pasting the thumbprint, can you just validate the command text in a hex editor to make sure there are no hidden characters in the thumbprint? We've seen other people have byte order mark hidden characters when dealing with copy pasted certs in the past. The Microsoft cert UI contains these characters so they are carried then you copy the thumbprint. |
I will try this solution and make sure to use a Hex Editor. I will work on this in the morning when I get back to work. I hope this works. I actually go into Azure and to my Cert Vault and go to the Cert and use the Copy function to get the Tumbprint info. |
String encoding is a very common issue: We recommend using Visual Studio Code when making software source edits: https://code.visualstudio.com/docs/editor/codebasics @starforce (Get-Childitem -Path Cert:\CurrentUser\My | Where-Object {$_.Subject -eq "cn=westus.cloudapp.azure.com"})[0].Thumbprint E.G. Invoke-WebRequest -Uri "https://sfbpcluster.westus.cloudapp.azure.com:19080/$/GetClusterHealth?api-version=6.4" -CertificateThumbprint (Get-Childitem -Path Cert:\CurrentUser\My | Where-Object {$_.Subject -eq "cn=<YOUR DOMAIN>.westus.cloudapp.azure.com"})[0].Thumbprint -ContentType 'application/json' -Method Get |
Hello, I really need to get backup going for my .NET CORE 2.0 Stateful apps and so far the suggestions i received here are not working Begin Issue below: Hello I just tried registering my Service Fabric Cluster domain name through DigiCert and could not, I just tried to secure a CERT for my Service Fabric Cluster (http://zixtysservices.eastus2.cloudapp.azure.com/) ,but According to DigiCert who is an Integrated partner with Microsoft when acquiring SSL. You cannot request a cert on any Microsoft Domain name. See Below. “For legal reasons we unfortunately are not allowed to issue any certificates to any domain names that are using any of Microsoft's trademarks. This included "azure". You can review the General Trademark Guidelines issued by Microsoft themselves. So since you CANNOT get a cert on a Microsoft Domain, your solution for my WebInvoke issue will not work “ As I have stated before, a CA verifies ownership of a DOMAIN in a few ways; SO who can Help me get backup working properly because you cannot have in your CN .azure.com and get a CERT. |
@starforce and @MicahMcKittrick-MSFT You are correct that only MSFTE like myself can provision a DNS name for any MS Azure records. So you will need to create a DNS alias for your domain, and how is subject to your architecture: If your management endpoint API is exposed by your Public IP for your load balancer: If your management endpoint API is exposed by a Traffic Manager: With your DNS alias mapped to yours clusters management IP, you can declare your management endpoint cluster property to use your DNS alias FQDN. E.G. "managementEndpoint": "[concat('<YOUR DNS FQDN ALIAS>',':',variables('nt0fabricHttpGatewayPort'))]", Let me know if this resolves your issue. |
@aljo-microsoft |
@aljo-microsoft @MicahMcKittrick-MSFT I don't know what this means below. Please provide an example. Is this an ARM script, Powershell? Where would i make this change after setting up my Alias. Are you sure this is going to work? With your DNS alias mapped to yours clusters management IP, you can declare your management endpoint cluster property to use your DNS alias FQDN. E.G. "managementEndpoint": "[concat('',':',variables('nt0fabricHttpGatewayPort'))]", |
Following: https://docs.microsoft.com/azure/dns/dns-delegate-domain-azure-dns I've bought a new domain today from GoDaddy, and updated my name servers to a new DNS Zone created in Azure; since this is where I plan to host my domain. Currently GoDaddy's states: Ref: Going to the portal blade for a Keyvault certificate, I Generated a certificate Request, by configuring DigiCert as my provider, and using my domain name as the common name and dns name in my request; documented here: Note: The password field in Azure Portal Blade is for an API Key, not your DigiCert CertCentral account. The provider in this case not only validated the details of my GoDaddy registered domain (Address, name, phone number), but also required I provide additional details (Gov ID, work flow approval process, phone call with me, email link I had to click, and then stated they needed more time for a secondary compliance check validation by a second person before they will approve the CSR I have already approved). If things complete as intended, I should have a certificate for my custom domain when I wake up issued by a 3rd party; without a cluster or webserver provisioned (enabled by me buying and owning a domain record from a 3rd party Domain registrar, and KV's Integrated CA's verification process) Next I will use the previously provided links to map my custom domain to a traffic manager, which will front my Primary Node Types Loadbalancer, and resolve to my management API IP address (where SF system services from my cluster will serve the SF Client API https://docs.microsoft.com/rest/api/servicefabric/sfclient-index); will update tomorrow with results. Also yes the managementEndpoint is a property of the ARM template cluster resource; example here: |
Going to wait on your results before i start. |
@starforce Note: GoDaddy appends the trailing period, so including the last period of any of your 4 naming servers, will result in a GoDaddy error; just remove the last period when updating those names. |
With my certificate provisioned for my custom domain, I provisioned a TrafficManager using the following, and standard cluster resources: https://github.com/aljo-microsoft/demo/blob/master/deploy/Deploy-2NodeTypes-3ScaleSets.endpoint.json and executed az group deployment create -g sfbpdeployrg --name sfbptrafficmanager --template-file AzureDeploy.json --parameters @AzureDeploy.parameters.json Then following this documentation I configured my DNS Zone Alias Record for my provisioned trafficmanager: https://docs.microsoft.com/azure/dns/tutorial-alias-tm#create-an-alias-record |
@starforce @hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT I've now provided all steps on how to configure SF (managementendpoint) to support your custom domain; this property updates Portal SFX URL. I also provided the guidance on how to acquire from a 3rd party a certificate for your custom domain, by using an Integerated CA to get a certificate for a custom domain, which supports validation processes that don't require you to provision your cluster first. This will allow you to provision your cluster using common name and a 3rd party custom domain certificate. That said, please close this. Note: Using the managementendpoint property only changes portal. If you don't successfully create an alias for your LB or TM fronting your cluster, that name wont resolve to your cluster for SFX or Clients connecting; they would need to connect using the TM or LB public IP directly. The impact of which is your browser will still complain that your custom domain listed in your cert doesn't match the URL. I fixed this by configuring mine correctly, using "A" and "C" record properties in my DNS Zone resource. |
@starforce @hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT I will try this and get back to everyone. Why the rush to close this issue until i verify that it works for me ? |
Thanks @aljo-microsoft for all the hard work and explanations. @starforce at this point we have addressed as much as we are able to within the scope of this issue. We use these issues to identify doc problems and we of course try to help further than that when possible. Please let us know your results. If you need further assistance from here we would need you to open a technical support ticket to work with an engineer to debug further. And of course, if you find any doc issues on other docs feel free to let us know. |
@MicahMcKittrick-MSFT .. Thank you. Hopefully it will work. Would i need to buy a Technical Support plan to open a technical support issue? |
@starforce if it comes to that I can enable you for a free request so no worries :) Let me know the outcome and we can go from there. |
@starforce and @MicahMcKittrick-MSFT By configuring my DNS Zone "Alias record set" for both my "A" and "C" record; set the "A" record to all IP address backends being fronted by TM, and use a "C" record set it to the TM DNS name. |
@aljo-microsoft Thank you for everything!! |
So far so Good. Set up Azure DNS zone, Then Change Google Domain to use Azure Name Servers. Did Ns-lookup and all is ok. DigiCert going to allow me to request a 30 days test Cert which is good because don't want to spend 198 dollars until I am sure it will work. Will update after DigiCert sends me info |
I just got my DigiCert. Will continue on Monday.. |
I plan on creating my Test Service Fabric Cluster and Traffic Manager through portal.zure.com instead of trying to modify this very long script. https://github.com/aljo-microsoft/demo/blob/master/deploy/Deploy-2NodeTypes-3ScaleSets.endpoint.json |
@hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT Hello @aljo-microsoft Are you saying that if I run this script that you provided here for creating a service fabric cluster https://github.com/aljo-microsoft/demo/blob/master/deploy/Deploy-2NodeTypes-3ScaleSets.endpoint.json Instead of creating my cluster through the Azure Portal that I can set the cluster name to my CN in my cert I got DigiCert? Because when I do this from the portal, there is no way to get around creating a cluster without having a Microsoft domain name. See below. I just want to make sure I can before creating the cluster manually. Because according to you “This will allow you to provision your cluster using common name and a 3rd party custom domain “ See below. |
@hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT Hey thanks for all the help but none of the suggestions fixed my problems. Going back to my original issue: Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry" This really should work. It works currently for deploying code to Service Fabric inside of Visual Studio and it works with Service Fabric explorer when there is a custom Domain. Need to get some help from an Microsoft Engineer but I spent a lot of time on this and I must move on. Also, These documents need to reflect that some use powershell scripts to perform their work but others just use portal.azure.com or resources.azure.com because it is easier and safer. So many of the examples here assume that powershell scripts are use. In a big organization with a SCM department this can be true but for smaller companies, we don’t’ have the luxury of fulltime SCM. Now I understand for backup there isn’t a UI but, from my point of view, while there is no doubt that I can use and run PowerShell scripts, it is not the best way because it is easier to make errors than using a UI. |
@starforce resources.azure.com and portal are not safer or recommended for production Service Fabric environments; you simply are not getting the full benefits of ARM using either of them, and will not always be able to leverage the latest Azure resource features through either of them. So this isn't the original issue, as the only SF specific thing your initial issue identified that I've since documented and filled, is how to update the management URL for Service Fabric Clusters in portal for your custom domain (with you being advised to use an ARM template). Azure provides tutorials for how to host your custom domain in Azure: Hosting your domain in Azure is a not a feature of SF resource's, as you can run your Azure hosted service on many resources; SF, App Services, AKS, ect. I've also not only provisioned a 3rd Party Cert, but completed end to end configuration of hosting my service in Azure using that cert from Digit cert; which they issued me without requiring I provision any Azure resources first to host some string, and my cluster using that cert is still running today and resolves my custom domain to my environment. My recommendation given the struggles you are having is to buy appropriate support to provide the velocity of Hosting your domain in Azure that you desire. |
Thank you for all your help, and I know how to use DNS hosting and all that technology because I got the same technology with Google domains. Also, I know how to modify arm templates . But the issue is there is no documentation that says that you can get around using a Microsoft domain when creating a service fabric cluster by using an arm template. And the script has over 1600 lines which is error-prone for human beings . I think a better solution is for me to call my internal apis at night and just retrieve the data in a Json format and save it myself until this featured becomes more robust and user friendly. And if you cannot use the portal to create the service fabric explore clusters in production then that needs to be explicitly stated in the how to documentation. If you do a search on Google of other people's examples they always use the portal to create service fabric clusters even for custom domains. My production website has been running for the last 6 months and I didn't use one arm template to create it and I have SSL working. |
I've published public docs that say clearly you should use ARM to provision production solutions, explicitly: " https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-infrastructure-as-code I'm glad your issue is resolved, and highly recommend reviewing all of our best practice documentation to avoid future issues: https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-overview |
Yes you also said that I could get a certificate for a Microsoft domain which I told you you could not. Thank you and have a good one |
Only authorized MSFT's like myself can request certificates for Microsoft owned domains. |
Yes but you should have known this before you gave me that solution that didn't work for me. I lost two full days working on that solution that you provided me. And now you ask me to modify a 1600 Plus line script and lose another two days on something that might work. Clearly Microsoft doesn't understand all the work and effort it takes to get things running. Anytime I can use a front-end UI to simplify my life I do. @MicahMcKittrick-MSFT is this normal? |
Please Review the following Best Practice Documentation: Also given my understanding of your issues being faced, in addition to a support ticket and above documentation recommendations, I recommend reviewing Keyvault and Networking resource documentation. As always please open a support ticket to expedite resolution of your production Service Fabric solution. The scenario as I understand it is: Issue kind: Issue Description: Current State: 01/29/2019 and previous dates performed validation of: Key Vault Integrated Digit Cert CA issued certificate, for 3rd party registered (GoDaddy) domain, used in successful execution of Invoke-WebRequest to a Secure Azure Service Fabric Cluster. Validating Azure Service Fabric Clusters best practices for securing your clusters, is declaring your Certificate Authority issued certificate properties Common Name and Issuer Thumbprint in your Azure Resource Manager template. Ref: |
Hello, I have a question about using the Invoke-WebRequest power shell command in your example.
First let me give you some background information about my current setup. Right now, I have a Service Fabric cluster using an X.509 cert from an CA and is used to secure the cluster. I have Admin rights, Also, I have the certificate on my local computer in the My (Personal) store. I set up AAD when I first created the cluster which allows me to use Service Fabric Explorer and publish/update code from Visual Studio 2017. This all works fine ,but The problem I am having is when I try to use the Invoke-WebRequest in your example and specify the -CertificateThumbprint. I am using the same Thumbprint that I use to set up my Cluster that I got from the CA. However, I always get the following error
"Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry"
Is there some reason why I cannot use the same X.509 Thumbprint and Cert that I use for publishing code from Visual Studio to My service fabric cluster and for Service Fabric Explorer ? According to Microsoft documentation, "By default the cluster certificate has admin client privileges." So confused as to why this isn't working.
Thank you
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.
The text was updated successfully, but these errors were encountered: