question

dylanmangold-7890 avatar image
0 Votes"
dylanmangold-7890 asked ajkuma-MSFT commented

Is there a known issue with Identity Server 4 randomly not being able to find a certificate in Azure after working fine with no updates?

I am getting a startup error in my app service (500.30) that states that my Identity Server 4 cannot find the certificate that I have uploaded to my app service as a private key certificate after being online and running consistently for several weeks with no issues. I verified that I have the WEBSITE_LOAD_CERTIFICATES app setting on the app as well as verified that the thumbprint matches that of the certificate I uploaded. I am using the following in the appsettings.json file to load the certificate:

 "IdentityServer": {
         "Clients": {
             "AssessmentTool": {
                 "Profile": "IdentityServerSPA"
             }
         },
         "Key": {
             "Type": "Store",
             "StoreName": "My",
             "StoreLocation": "CurrentUser",
             "Name": "CN=IdentityCertificate"
         }
     }

Are there any known issues where Identity server will stop recognizing a healthy certificate in Azure that was created on my MacBook Pro and uploaded as a .pfx file? I have the same setup running in a production environment with the only difference is that the certificate being used is one that was generated in the Azure Portal.

 Unhandled exception. System.InvalidOperationException: Couldn't find a valid certificate with subject 'CN=IdentityCertificate' on the 'CurrentUser\My'
    at Microsoft.AspNetCore.ApiAuthorization.IdentityServer.SigningKeysLoader.LoadFromStoreCert(String subject, String storeName, StoreLocation storeLocation, DateTimeOffset currentTime)
    at Microsoft.AspNetCore.ApiAuthorization.IdentityServer.ConfigureSigningCredentials.LoadKey()
    at Microsoft.AspNetCore.ApiAuthorization.IdentityServer.ConfigureSigningCredentials.Configure(ApiAuthorizationOptions options)
    at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
    at Microsoft.Extensions.Options.OptionsManager`1.<>c__DisplayClass5_0.<Get>b__0()
    at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
    at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
    at System.Lazy`1.CreateValue()
    at System.Lazy`1.get_Value()
    at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
    at Microsoft.Extensions.Options.OptionsManager`1.Get(String name)
    at Microsoft.Extensions.Options.OptionsManager`1.get_Value()
    at Microsoft.Extensions.DependencyInjection.IdentityServerBuilderConfigurationExtensions.<>c.<AddClients>b__8_1(IServiceProvider sp)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
    at Microsoft.AspNetCore.Builder.IdentityServerApplicationBuilderExtensions.TestService(IServiceProvider serviceProvider, Type service, ILogger logger, String message, Boolean doThrow)
    at Microsoft.AspNetCore.Builder.IdentityServerApplicationBuilderExtensions.Validate(IApplicationBuilder app)
    at Microsoft.AspNetCore.Builder.IdentityServerApplicationBuilderExtensions.UseIdentityServer(IApplicationBuilder app, IdentityServerMiddlewareOptions options)
    at AssessmentTool.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, UserManager`1 userManager, RoleManager`1 roleManager) in /home/vsts/work/1/s/Startup.cs:line 114
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
    at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
    at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass15_0.<UseStartup>b__1(IApplicationBuilder app)
    at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
    at Microsoft.AspNetCore.Server.IIS.Core.IISServerSetupFilter.<>c__DisplayClass2_0.<Configure>b__0(IApplicationBuilder app)
    at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
    at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
    at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
    at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
    at AssessmentTool.Program.Main(String[] args) in /home/vsts/work/1/s/Program.cs:line 16
dotnet-aspnet-core-webapiazure-webapps-ssl-certificatesdotnet-aspnet-core-security
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@dylanmangold-7890, Thanks for posting this question with a detailed description.

1/2

When you say, ‘a healthy certificate in Azure that was created on my MacBook Pro’, I believe you’re using the certificate as described in this Apple’s doc.

To understand the root cause of the issue we may need to perform a deeper investigation, for which I would need some more details about your subscription. (Please do not share any private details publicly, for your privacy. Will connect with you privately).


Navigate to your App Service app in the Azure portal. In the left navigation, click on Diagnose and solve problems – Checkout the tile for “Diagnostic Tools” and “Availability and Performance”. Run the certificate diagnostics to fetch more details on the issue and for pointers.



101132-image.png

0 Votes 0 ·
image.png (51.5 KiB)

@dylanmangold-7890

2/2

Run F12 trace/Fiddler to narrow down the issue further.

Typically,
It's better to reference the certificate path with the environment variables instead of hardcoding the certificate path, to avoid the certificate path change.
The WEBSITE_LOAD_CERTIFICATES app setting makes the specified certificates accessible to your Windows hosted app in the Windows certificate store, in Current User\My.

Please see this article which contains best practices and advices on using X509Certificate2 in App Services.

Based on your requirement, you may check if free managed certificate fits well. However, this certificate comes with some limitations (see this doc)
-client certificate by thumbprint not supported.


0 Votes 0 ·

Kindly let us know how it goes, I'll follow-up with you further (privately).

On App Service, the free App Service managed certificate and the App Service certificate already satisfy the requirements of App Service. If you choose to upload or import a private certificate to App Service, your certificate must meet the following requirements:
See Private certificate requirements -https://docs.microsoft.com/azure/app-service/configure-ssl-certificate#private-certificate-requirements

0 Votes 0 ·

@ajkuma-MSFT

I ran the diagnostics, but they all passed, except for the minimum TLS version not being set to 1.2, though that should not be the issue since our production sight is setup the same way, except that the certificate was generated by Azure instead of on my Mac.

As far as how the certificate was created, I generated the certificate using OpenSSL in the terminal. The article that you referenced in the second comment is not a hyper link, so I am not sure what you were referencing. Please note that the certificate that I uploaded was working for a few weeks before no longer working.

What information do you need on the subscription to be of better help?

0 Votes 0 ·

@dylanmangold-7890, Thanks for the follow-up and sharing additional details.

The link in my 2nd comment - https://github.com/projectkudu/kudu/wiki/Best-X509Certificate2-Practices - Sorry for missing the hyperlink (updated now).

In few cases, such issues are caused if the certificate is wrongly marked unhealthy due to expired certificate. I understand the certificate expiry maybe defaulted to 365 days.
Kindly see this similar discussion thread.

Just to isolate, have you noticed a similar behavior locally (for a similar setup -website with Identity Server 4 and OpenSSL). To narrow-down, if it’s App Service vs OpenSSL Vs Identity Server 4 issue.

From the scenario flow, behavior and your description, “…the certificate being used is one that was generated in the Azure Portal”, it’s most likely not an Azure issue. Let us know how it goes, I’ll further follow-up.


0 Votes 0 ·

@DylanMangold-8664,

I have also added additional tags to receive insights/inputs from asp.net core SMEs. I’m sharing insights from Azure App Service standpoint, to find the root cause.

Please see these docs:

Troubleshoot certificate problems such as certificate not trusted

Trust the ASP.NET Core HTTPS development certificate on Windows and macOS

Let us know how it goes, I'll follow-up with you further.

0 Votes 0 ·

@DylanMangold-8664, Just following-up, kindly let us know if the issue still persists, or you need further assistance on this.

0 Votes 0 ·

0 Answers