Anomaly detection

[Steven Kassakian]

I am trying to find out if there is any functions or add-ons to STATA which support anomaly detection on STATA on time-series data.

The data of have is daily frequency counts and there is a both a collective and contextual nature to their anomaly pattern.

Essentially, simple distance measures won't suffice since sometimes a zero is anomalous and sometimes it is not depending on the overall pattern of the data. Likewise sometimes a value of 50 is anomalous when it isn't precedded by a 20 and followed by a 70 but otherwise is. From my reading I am looking to use collective and contextual anomaly methods.

Anyone have any experience with this in either stata or elsewhere?

[Nick Cox]

As you're aware, identifying an anomaly requires some kind of specification of what is expected. Surprising is assessed relative to expectable.

Absent a model for the data-generating process, I'd try some kind of time series smoothing based on the series to date and calculate residuals relative to that. For an exploratory analysis of counts including zeros I would work on square roots of counts.

tssmooth is the most obvious command. in Stata.

[Steve Samuels]

It's well known that least squares fits are poor baselines for outlier detection.. For that you need a robust fit. Verardi ,and Croux's mmregress Stata package is one of the best; it will isolate outliers and high leverage points both. I don't know of a comparable robust time series command in Stata. There are several in R. See: https://cran.r-project.org/web/views/Robust.html

[Emilio Matuk]

I'm using mcd command for local outlier factor, but it's not parallelized. Is any other equivalent option? Thanks