SharePoint 2013 Search Topology Activation Error: "Unable to retrieve topology component health states. This may be because of the admin component is not up and running"
Scenario
- In a newly deployed farm, creating new search service application (SSA) via PowerShell or GUI of SharePoint 2013.
- The SSA gets created, however, topology in SSA will show
"Unable to retrieve topology component health states. This may be because of the admin component is not up and running."
- The trial to create and activate topology fails immediately and sends an event ID to application event as follows
Event ID: 6482
Description:
Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance
Reason: A call to SSPI failed, see inner exception.
Technical Support Details:
System.ServiceModel.Security.SecurityNegotiationException: A call to SSPI failed, see inner exception. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: No authority could be contacted for authentication.
Possible Reasons:
1. The timer service account is trying to communicate with HostController service and generates 'SSPI Connect' call demanding SPN to be created for target services Identity, if no SPN is found.
2. The Hostcontroller service's NodeRunner process is limited to use restricted amount of RAM
Possible Solutions:
Check which account is running the Timer Service account.
Check which account is running HostController Service (From Services.msc or Central Administration > Security > manage Service Accounts)
Use SetSPN -L <Domain\Accountname> to list active SPNs for these two accounts
We have to set a dummy SPN for Timer Service and HostController service account.
Set an SPN SP/SPTEST -- this is a dummy SPN for Timer Service account.
Set an SPN SP/SPTEST1 -- this is a dummy SPN, for service account of Host Controller Service also.
Do IIS Reset.
About the above issue:
- During activation of topology via PowerShell, you may observe error,
System.Management.Automation.MethodInvocationException:
Exception calling "Activate" with "0" argument(s):
"Could not connect to the HostController service on server <ServerName> Topology Activation could not be started"
During topology activation, If opened task manager and checked status of HostController.exe - It will be running always , during the activation of topology. However, NO NODERUNNER.EXE will be found in process list. No attempt to activate topology will be observed in ULS.
Application Event will show error below
Source: Microsoft-SharePoint Products-SharePoint Server
Event ID: 6482
Task Category: Shared Services
Level: Error
Keywords:
User: Timer Service Account
Computer: <Server name>
Description:
Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (Service instance GUID).
Reason: A call to SSPI failed, see inner exception.
Technical Support Details:
System.ServiceModel.Security.SecurityNegotiationException: A call to SSPI failed, see inner exception. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: No authority could be contacted for authentication
--- End of inner exception stack trace ---
at System.Net.Security.NegoState.StartSendAuthResetSignal(LazyAsyncResult lazyResult, Byte[] message, Exception exception)
at System.Net.Security.NegoState.StartSendBlob(Byte[] message, LazyAsyncResult lazyResult)
at System.Net.Security.NegoState.CheckCompletionBeforeNextSend(Byte[] message, LazyAsyncResult lazyResult)
at System.Net.Security.NegoState.ProcessReceivedBlob(Byte[] message, LazyAsyncResult lazyResult)
at System.Net.Security.NegoState.StartSendBlob(Byte[] message, LazyAsyncResult lazyResult)
at System.Net.Security.NegoState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential credential, String targetName, ProtectionLevel requiredProtectionLevel, TokenImpersonationLevel allowedImpersonationLevel)
at System.ServiceModel.Channels.WindowsStreamSecurityUpgradeProvider.WindowsStreamSecurityUpgradeInitiator.OnInitiateUpgrade(Stream stream, SecurityMessageProperty& remoteSecurity)
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.WindowsStreamSecurityUpgradeProvider.WindowsStreamSecurityUpgradeInitiator.OnInitiateUpgrade(Stream stream, SecurityMessageProperty& remoteSecurity)
at System.ServiceModel.Channels.StreamSecurityUpgradeInitiatorBase.InitiateUpgrade(Stream stream)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.InitiateUpgrade(StreamUpgradeInitiator upgradeInitiator, IConnection& connection, ClientFramingDecoder decoder, IDefaultCommunicationTimeouts defaultTimeouts, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)
ULS will have evidence for Timer Service account trying to contact HostController Service and fails trying to authenticate.
Once the activation of topology is failed, we can see detailed exception in ULS:
OWSTIMER.EXE SharePoint Server Search Administration aizcv Exception Failed to update hostcontroller primary repository version System.ServiceModel.Security.SecurityNegotiationException: A call to SSPI failed, see inner exception. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: No authority could be contacted for authentication --- End of inner exception stack trace ---
at System.Net.Security.NegoState.StartSendAuthResetSignal(LazyAsyncResult lazyResult, Byte[] message, Exception exception) at System.Net.Security.NegoState.StartSendBlob(Byte[] message, LazyAsyncResult lazyResult) System.Net.Security.NegoState.CheckCompletionBeforeNextSend(Byte[] message, LazyAsyncResult lazyResult) at System.Net.Security.NegoState.ProcessReceivedBlob(Byte[] message, LazyAsyncResult lazyResult) at System.Net.Security.NegoState.StartSendBlob(Byte[] message, LazyAsyncResult lazyResult) at System.Net.Security.NegoState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential credential, String targetName, ProtectionLevel requiredProtectionLevel, TokenImpersonationLevel allowedImpersonationLevel) at System.ServiceModel.Channels.WindowsStreamSecurityUpgradeProvider.WindowsStreamSecurityUpgradeInitiator.OnInitiateUpgrade(Stream stream, SecurityMessageProperty& remoteSecurity)