SrCnnAnomalyEstimator Class

Definition

Namespace:

Microsoft.ML.Transforms.TimeSeries

Assembly:

Microsoft.ML.TimeSeries.dll

Package:

Microsoft.ML.TimeSeries v3.0.1

Detect anomalies in time series using Spectral Residual(SR) algorithm

public sealed class SrCnnAnomalyEstimator : Microsoft.ML.Data.TrivialEstimator<Microsoft.ML.Transforms.TimeSeries.SrCnnAnomalyDetector>

type SrCnnAnomalyEstimator = class
    inherit TrivialEstimator<SrCnnAnomalyDetector>

Public NotInheritable Class SrCnnAnomalyEstimator
Inherits TrivialEstimator(Of SrCnnAnomalyDetector)

Inheritance

Object

TrivialEstimator<SrCnnAnomalyDetector>

SrCnnAnomalyEstimator

Remarks

To create this estimator, use DetectAnomalyBySrCnn

Estimator Characteristics

Does this estimator need to look at the data to train its parameters?	No
Input column data type	Single
Output column data type	3-element vector ofDouble
Exportable to ONNX	No

Background

At Microsoft, we have developed a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). The SR model is borrowed from visual saliency detection domain to time-series anomaly detection. And here we onboarded this SR algorithm firstly.

The Spectral Residual (SR) algorithm is unsupervised, which means training step is not needed while using SR. It consists of three major steps: (1) Fourier Transform to get the log amplitude spectrum; (2) calculation of spectral residual; (3) Inverse Fourier Transform that transforms the sequence back to spatial domain. Mathematically, given a sequence $\mathbf{x}$, we have $$A(f) = Amplitude(\mathfrak{F}(\mathbf{x}))\P(f) = Phrase(\mathfrak{F}(\mathbf{x}))\L(f) = log(A(f))\AL(f) = h_n(f) \cdot L(f)\R(f) = L(f) - AL(f)\S(\mathbf{x}) = \mathfrak{F}^{-1}(exp(R(f) + P(f))^{2})$$ where $\mathfrak{F}$ and $\mathfrak{F}^{-1}$ denote Fourier Transform and Inverse Fourier Transform respectively. $\mathbf{x}$ is the input sequence with shape $n × 1$; $A(f)$ is the amplitude spectrum of sequence $\mathbf{x}$; $P(f)$ is the corresponding phase spectrum of sequence $\mathbf{x}$; $L(f)$ is the log representation of $A(f)$; and $AL(f)$ is the average spectrum of $L(f)$ which can be approximated by convoluting the input sequence by $h_n(f)$, where $h_n(f)$ is an $n × n$ matrix defined as: $$n_f(f) = \begin{bmatrix}1&1&1&\cdots&1\1&1&1&\cdots&1\\vdots&\vdots&\vdots&\ddots&\vdots\1&1&1&\cdots&1\end{bmatrix}$$ $R(f)$ is the spectral residual, i.e., the log spectrum $L(f)$ subtracting the averaged log spectrum $AL(f)$. The spectral residual serves as a compressed representation of the sequence while the innovation part of the original sequence becomes more significant. At last, we transfer the sequence back to spatial domain via Inverse Fourier Transform. The result sequence $S(\mathbf{x})$ is called the saliency map. Given the saliency map $S(\mathbf{x})$, the output sequence $O(\mathbf{x})$ is computed by: $$O(x_i) = \begin{cases}1, if \frac{S(x_i)-\overline{S(x_i)}}{S(x_i)} > \tau\0,otherwise,\end{cases}$$ where $x_i$ represents an arbitrary point in sequence $\mathbf{x}$; $S(x_i)$is the corresponding point in the saliency map; and $\overline{S(x_i)}$ is the local average of the preceding points of $S(x_i)$.

There are several parameters for SR algorithm. To obtain a model with good performance, we suggest to tune windowSize and threshold at first, these are the most important parameters to SR. Then you could search for an appropriate judgementWindowSize which is no larger than windowSize. And for the remaining parameters, you could use the default value directly.

For more details please refer to the Time-Series Anomaly Detection Service at Microsoft paper.

Methods

Fit(IDataView)	(Inherited from TrivialEstimator<TTransformer>)
GetOutputSchema(SchemaShape)

Extension Methods

AppendCacheCheckpoint<TTrans>(IEstimator<TTrans>, IHostEnvironment)

Append a 'caching checkpoint' to the estimator chain. This will ensure that the downstream estimators will be trained against cached data. It is helpful to have a caching checkpoint before trainers that take multiple data passes.

WithOnFitDelegate<TTransformer>(IEstimator<TTransformer>, Action<TTransformer>)

Given an estimator, return a wrapping object that will call a delegate once Fit(IDataView) is called. It is often important for an estimator to return information about what was fit, which is why the Fit(IDataView) method returns a specifically typed object, rather than just a general ITransformer. However, at the same time, IEstimator<TTransformer> are often formed into pipelines with many objects, so we may need to build a chain of estimators via EstimatorChain<TLastTransformer> where the estimator for which we want to get the transformer is buried somewhere in this chain. For that scenario, we can through this method attach a delegate that will be called once fit is called.

See also

• DetectAnomalyBySrCnn(TransformsCatalog, String, String, Int32, Int32, Int32, Int32, Int32, Double)