Service Fabric ReliableConcurrentQueue warning

Ron Lauzon 1 Reputation point
2021-11-23T18:55:56.013+00:00

Recently we've been seeing a strange error associated with a replica of one of our services. We want to track down what's causing it, but we don't understand the error.

'System.RA' reported Warning for property 'ReplicaOpenStatus'.
Replica had multiple failures during open on <one of our servers>. API call: IStatefulServiceReplica.Open(); Error = System.IO.InvalidDataException (-2146233087)
versionRead '0' != FileVersion '1'
at Microsoft.ServiceFabric.Data.Collections.ReliableConcurrentQueue.CheckpointManager1.<TryReadCheckpointFile>d__26.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceFabric.Data.Collections.ReliableConcurrentQueue.ReliableConcurrentQueue1.<Microsoft-ServiceFabric-Replicator-IStateProvider2-RecoverCheckpointAsync>d__86.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Replicator.DynamicStateManager.<RecoverStateProvidersAsync>d__187.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Replicator.DynamicStateManager.<OpenAsync>d__118.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Replicator.TransactionalReplicator.<OpenAsync>d__82.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Replicator.StatefulServiceReplica.<System-Fabric-IStatefulServiceReplica-OpenAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Data.ReliableStateManagerImpl.<Microsoft-ServiceFabric-Data-IStateProviderReplica-OpenAsync>d__46.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Services.Runtime.StatefulServiceReplicaAdapter.<System-Fabric-IStatefulServiceReplica-OpenAsync>d__17.MoveNext()
For more information see: <a href='https://aka.ms/sfhealth' target='_blank'>https://aka.ms/sfhealth</a>

Any help would be appreciated.

Azure Service Fabric
Azure Service Fabric
An Azure service that is used to develop microservices and orchestrate containers on Windows and Linux.
252 questions
{count} votes