What is Anomaly Finder?

Note

Anomaly Finder is now in Private Preview. See Anomaly Finder Private Preview documentation for more information. To join the Anomaly Finder Private Preview, contact us.

Anomaly Finder enables you to monitor data over time and detect anomalies with machine learning that adapts to your unique data by automatically applying the right statistical model regardless of industry, scenario, or data volume. Using a time series as input, the Anomaly Finder API returns whether or not a data point is an anomaly, determines the expected value, and upper and lower bounds for visualization. As a prebuilt AI service, Anomaly Finder doesn’t require any machine learning expertise beyond understanding how to use a RESTful API. This makes development simple and versatile since it works with any time series data and can also be built into streaming data systems. Anomaly Finder encompasses a broad span of use cases – for instance, financial tools for managing fraud, theft, changing markets, and potential business incidents, or monitoring IoT device traffic while preserving anonymity. This solution can also be monetized as part of a service for end-customers to understand changes in data, spending, return on investment, or user activity. Try out the Anomaly Finder API and gain deeper understanding of your data.

See what you can build with this API:

  • Learn to predict the expected values based on historical data in the time series
  • Tell whether a data point is an anomaly out of historical pattern
  • Generate a band to visualize the range of "normal" value

Anomaly_Finder

Fig. 1: Detect anomalies in sales revenues

Anomaly_Finder

Fig. 2: Detect pattern changes in service requests

Requirements

  • Minimum data for input time series: Minimum of 13 data points for time series without clear periodicity, minimum of 4 cycles of data points for the time series with known periodicity.
  • Data integrity: time series data points are separated in the same interval and no missing points.

Identify anomalies

Anomaly detection API returns result that whether any given data points are anomalies or not, and provides additional information as follows

  • Period - The periodicity that the API used to detect the anomaly points.
  • WarningText - The possible warning information.
  • ExpectedValue - The predicted value by the learning based model
  • IsAnomaly - The result on whether the data points are anomalies or not
  • IsAnomaly_Neg - The result on whether the data points are anomalies in negative direction (dips)
  • IsAnomaly_Pos - The result on whether the data points are anomalies in positive direction (spikes)
  • UpperMargin - The sum of ExpectedValue and UpperMargin determines the upper bound that data point is still thought as normal
  • LowerMargin - (ExpectedValue - LowerMargin) determines the lower bound that data point is still thought as normal

Note

UpperMargin and LowerMargin can be used to generate a band around actual time series to visualize the range of normal values.

Adjusting lower and upper bounds in post processing on the response

Anomaly detection API returns default result on whether a data point is anomaly or not, and the upper and lower bound can be calculated from ExpectedValue and UpperMargin/LowerMargin. Those default values should work just fine for most cases. However, some scenarios require different bounds than the default ones. The recommend practice is applying a coefficiency on the UpperMargin or LowerMargin to adjust the dynamic bounds.

Examples with 1/1.5/2 as coefficiency

Default Sensitivity

1.5 Sensitivity

2 Sensitivity

Request with sample data

{
  "Period": 7,
  "Points": [
    {
      "Timestamp": "2018-03-01T00:00:00Z",
      "Value": 32858923
    },
    {
      "Timestamp": "2018-03-02T00:00:00Z",
      "Value": 29615278
    },
    {
      "Timestamp": "2018-03-03T00:00:00Z",
      "Value": 22839355
    },
    {
      "Timestamp": "2018-03-04T00:00:00Z",
      "Value": 25948736
    },
    {
      "Timestamp": "2018-03-05T00:00:00Z",
      "Value": 34139159
    },
    {
      "Timestamp": "2018-03-06T00:00:00Z",
      "Value": 33843985
    },
    {
      "Timestamp": "2018-03-07T00:00:00Z",
      "Value": 33637661
    },
    {
      "Timestamp": "2018-03-08T00:00:00Z",
      "Value": 32627350
    },
    {
      "Timestamp": "2018-03-09T00:00:00Z",
      "Value": 29881076
    },
    {
      "Timestamp": "2018-03-10T00:00:00Z",
      "Value": 22681575
    },
    {
      "Timestamp": "2018-03-11T00:00:00Z",
      "Value": 24629393
    },
    {
      "Timestamp": "2018-03-12T00:00:00Z",
      "Value": 34010679
    },
    {
      "Timestamp": "2018-03-13T00:00:00Z",
      "Value": 33893888
    },
    {
      "Timestamp": "2018-03-14T00:00:00Z",
      "Value": 33760076
    },
    {
      "Timestamp": "2018-03-15T00:00:00Z",
      "Value": 33093515
    },
    {
      "Timestamp": "2018-03-16T00:00:00Z",
      "Value": 29945555
    },
    {
      "Timestamp": "2018-03-17T00:00:00Z",
      "Value": 22676212
    },
    {
      "Timestamp": "2018-03-18T00:00:00Z",
      "Value": 25262514
    },
    {
      "Timestamp": "2018-03-19T00:00:00Z",
      "Value": 33631649
    },
    {
      "Timestamp": "2018-03-20T00:00:00Z",
      "Value": 34468310
    },
    {
      "Timestamp": "2018-03-21T00:00:00Z",
      "Value": 34212281
    },
    {
      "Timestamp": "2018-03-22T00:00:00Z",
      "Value": 38144434
    },
    {
      "Timestamp": "2018-03-23T00:00:00Z",
      "Value": 34662949
    },
    {
      "Timestamp": "2018-03-24T00:00:00Z",
      "Value": 24623684
    },
    {
      "Timestamp": "2018-03-25T00:00:00Z",
      "Value": 26530491
    },
    {
      "Timestamp": "2018-03-26T00:00:00Z",
      "Value": 35445003
    },
    {
      "Timestamp": "2018-03-27T00:00:00Z",
      "Value": 34250789
    },
    {
      "Timestamp": "2018-03-28T00:00:00Z",
      "Value": 33423012
    },
    {
      "Timestamp": "2018-03-29T00:00:00Z",
      "Value": 30744783
    },
    {
      "Timestamp": "2018-03-30T00:00:00Z",
      "Value": 25825128
    },
    {
      "Timestamp": "2018-03-31T00:00:00Z",
      "Value": 21244209
    },
    {
      "Timestamp": "2018-04-01T00:00:00Z",
      "Value": 22576956
    },
    {
      "Timestamp": "2018-04-02T00:00:00Z",
      "Value": 31957221
    },
    {
      "Timestamp": "2018-04-03T00:00:00Z",
      "Value": 33841228
    },
    {
      "Timestamp": "2018-04-04T00:00:00Z",
      "Value": 33554483
    },
    {
      "Timestamp": "2018-04-05T00:00:00Z",
      "Value": 32383350
    },
    {
      "Timestamp": "2018-04-06T00:00:00Z",
      "Value": 29494850
    },
    {
      "Timestamp": "2018-04-07T00:00:00Z",
      "Value": 22815534
    },
    {
      "Timestamp": "2018-04-08T00:00:00Z",
      "Value": 25557267
    },
    {
      "Timestamp": "2018-04-09T00:00:00Z",
      "Value": 34858252
    },
    {
      "Timestamp": "2018-04-10T00:00:00Z",
      "Value": 34750597
    },
    {
      "Timestamp": "2018-04-11T00:00:00Z",
      "Value": 34717956
    },
    {
      "Timestamp": "2018-04-12T00:00:00Z",
      "Value": 34132534
    },
    {
      "Timestamp": "2018-04-13T00:00:00Z",
      "Value": 30762236
    },
    {
      "Timestamp": "2018-04-14T00:00:00Z",
      "Value": 22504059
    },
    {
      "Timestamp": "2018-04-15T00:00:00Z",
      "Value": 26149060
    },
    {
      "Timestamp": "2018-04-16T00:00:00Z",
      "Value": 35250105
    }
  ]
}

Sample JSON response

{  
    "Period":7,
    "ExpectedValue":[  
        32894418.9615615,
        29707932.24471988,
        22651867.032410353,
        24943247.98913801,
        34098022.11082705,
        33893733.15343374,
        33668289.17375017,
        32807561.144138098,
        29618567.705954053,
        22584659.813523095,
        24896916.283606086,
        34092130.34860708,
        33930369.33544473,
        33771835.54784974,
        32979718.237272907,
        29873907.686442,
        22898518.640143186,
        25266675.592631694,
        34498225.710081585,
        34392110.769727185,
        34264840.807082534,
        33364403.57465193,
        30135700.933649357,
        23027724.20007864,
        25265911.829021864,
        34436837.95171797,
        34250788.99625075,
        33423011.962283727,
        31961651.40533486,
        29172725.64651445,
        22514221.39222613,
        24890134.85738833,
        34129441.48751191,
        33864058.08414464,
        33577519.658487104,
        32851940.511712912,
        29798095.95636674,
        22918033.675673913,
        25384135.75749503,
        34746812.760001436,
        34752514.68434451,
        34634850.418293975,
        33883602.629261605,
        30764952.72634012,
        23776724.327950727,
        26128787.792423487,
        35344244.421857625
    ],
    "IsAnomaly":[  
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        true,
        true,
        true,
        true,
        false,
        false,
        false,
        false,
        true,
        true,
        true,
        true,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false
    ],
    "IsAnomaly_Neg":[  
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        true,
        true,
        true,
        true,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false
    ],
    "IsAnomaly_Pos":[  
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        true,
        true,
        true,
        true,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false,
        false
    ],
    "UpperMargin":[  
        1644720.9480780752,
        1485396.612235994,
        1132593.3516205177,
        1247162.3994569005,
        1704901.1055413526,
        1694686.6576716872,
        1683414.4586875085,
        1640378.057206905,
        1480928.3852977026,
        1129232.9906761548,
        1244845.8141803043,
        1704606.517430354,
        1696518.4667722369,
        1688591.777392487,
        1648985.9118636455,
        1493695.3843221,
        1144925.9320071593,
        1263333.7796315849,
        1724911.2855040794,
        1719605.5384863594,
        1713242.0403541268,
        1668220.1787325966,
        1506785.046682468,
        1151386.210003932,
        1263295.5914510933,
        1721841.8975858986,
        1712539.4498125375,
        1671150.5981141864,
        1598082.5702667432,
        1458636.2823257225,
        1125711.0696113065,
        1244506.7428694165,
        1706472.0743755957,
        1693202.904207232,
        1678875.9829243552,
        1642597.0255856458,
        1489904.797818337,
        1145901.6837836958,
        1269206.7878747517,
        1737340.638000072,
        1737625.7342172256,
        1731742.520914699,
        1694180.1314630804,
        1538247.636317006,
        1188836.2163975365,
        1306439.3896211744,
        1767212.2210928814
    ],
    "LowerMargin":[  
        1644720.9480780752,
        1485396.612235994,
        1132593.3516205177,
        1247162.3994569005,
        1704901.1055413526,
        1694686.6576716872,
        1683414.4586875085,
        1640378.057206905,
        1480928.3852977026,
        1129232.9906761548,
        1244845.8141803043,
        1704606.517430354,
        1696518.4667722369,
        1688591.777392487,
        1648985.9118636455,
        1493695.3843221,
        1144925.9320071593,
        1263333.7796315849,
        1724911.2855040794,
        1719605.5384863594,
        1713242.0403541268,
        1668220.1787325966,
        1506785.046682468,
        1151386.210003932,
        1263295.5914510933,
        1721841.8975858986,
        1712539.4498125375,
        1671150.5981141864,
        1598082.5702667432,
        1458636.2823257225,
        1125711.0696113065,
        1244506.7428694165,
        1706472.0743755957,
        1693202.904207232,
        1678875.9829243552,
        1642597.0255856458,
        1489904.797818337,
        1145901.6837836958,
        1269206.7878747517,
        1737340.638000072,
        1737625.7342172256,
        1731742.520914699,
        1694180.1314630804,
        1538247.636317006,
        3523071.2279507257,
        1306439.3896211744,
        1767212.2210928814
    ],
   "WarningText":""
}