SrCnnAnomalyEstimator Class

Definition

Detect anomalies in time series using Spectral Residual(SR) algorithm

public sealed class SrCnnAnomalyEstimator : Microsoft.ML.Data.TrivialEstimator<Microsoft.ML.Transforms.TimeSeries.SrCnnAnomalyDetector>
type SrCnnAnomalyEstimator = class
    inherit TrivialEstimator<SrCnnAnomalyDetector>
Public NotInheritable Class SrCnnAnomalyEstimator
Inherits TrivialEstimator(Of SrCnnAnomalyDetector)
Inheritance
SrCnnAnomalyEstimator

Remarks

To create this estimator, use DetectAnomalyBySrCnn

Estimator Characteristics

Does this estimator need to look at the data to train its parameters? No
Input column data type Single
Output column data type 3-element vector ofDouble

Background

At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). The SR model is borrowed from visual saliency detection domain to time-series anomaly detection. And here we onboarded this SR algorithm firstly.

The Spectral Residual (SR) algorithm is unsupervised, which means training step is not needed while using SR. It consists of three major steps: (1) Fourier Transform to get the log amplitude spectrum; (2) calculation of spectral residual; (3) Inverse Fourier Transform that transforms the sequence back to spatial domain. Mathematically, given a sequence $\mathbf{x}$, we have $$A(f) = Amplitude(\mathfrak{F}(\mathbf{x}))\P(f) = Phrase(\mathfrak{F}(\mathbf{x}))\L(f) = log(A(f))\AL(f) = h_n(f) \cdot L(f)\R(f) = L(f) - AL(f)\S(\mathbf{x}) = \mathfrak{F}^{-1}(exp(R(f) + P(f))^{2})$$ where $\mathfrak{F}$ and $\mathfrak{F}^{-1}$ denote Fourier Transform and Inverse Fourier Transform respectively. $\mathbf{x}$ is the input sequence with shape $n × 1$; $A(f)$ is the amplitude spectrum of sequence $\mathbf{x}$; $P(f)$ is the corresponding phase spectrum of sequence $\mathbf{x}$; $L(f)$ is the log representation of $A(f)$; and $AL(f)$ is the average spectrum of $L(f)$ which can be approximated by convoluting the input sequence by $h_n(f)$, where $h_n(f)$ is an $n × n$ matrix defined as: $$n_f(f) = \begin{bmatrix}1&1&1&\cdots&1\1&1&1&\cdots&1\\vdots&\vdots&\vdots&\ddots&\vdots\1&1&1&\cdots&1\end{bmatrix}$$ $R(f)$ is the spectral residual, i.e., the log spectrum $L(f)$ subtracting the averaged log spectrum $AL(f)$. The spectral residual serves as a compressed representation of the sequence while the innovation part of the original sequence becomes more significant. At last, we transfer the sequence back to spatial domain via Inverse Fourier Transform. The result sequence $S(\mathbf{x})$ is called the saliency map. Given the saliency map $S(\mathbf{x})$, the output sequence $O(\mathbf{x})$ is computed by: $$O(x_i) = \begin{cases}1, if \frac{S(x_i)-\overline{S(x_i)}}{S(x_i)} > \tau\0,otherwise,\end{cases}$$ where $x_i$ represents an arbitrary point in sequence $\mathbf{x}$; $S(x_i)$is the corresponding point in the saliency map; and $\overline{S(x_i)}$ is the local average of the preceding points of $S(x_i)$.

There are several parameters for SR algorithm. To obtain a model with good performance, we suggest to tune windowSize and threshold at first, these are the most important parameters to SR. Then you could search for an appropriate judgementWindowSize which is no larger than windowSize. And for the remaining parameters, you could use the default value directly.

  • Link to the KDD 2019 paper will be updated after it goes public.

Methods

Fit(IDataView) (Inherited from TrivialEstimator<TTransformer>)
GetOutputSchema(SchemaShape)

Extension Methods

WithOnFitDelegate<TTransformer>(IEstimator<TTransformer>, Action<TTransformer>)

Given an estimator, return a wrapping object that will call a delegate once Fit(IDataView) is called. It is often important for an estimator to return information about what was fit, which is why the Fit(IDataView) method returns a specifically typed object, rather than just a general ITransformer. However, at the same time, IEstimator<TTransformer> are often formed into pipelines with many objects, so we may need to build a chain of estimators via EstimatorChain<TLastTransformer> where the estimator for which we want to get the transformer is buried somewhere in this chain. For that scenario, we can through this method attach a delegate that will be called once fit is called.

Applies to

See also