pyhctsa.operations.distribution.outlier_include

pyhctsa.operations.distribution.outlier_include(y, threshold_how='abs', inc=0.01, fixed_thresh=None)

How the timing and spacing of extreme values change as the threshold rises.

Raises a threshold th from 0 to the maximum value of the series, in increments of inc, and at each threshold takes the “events”: the points at or beyond it (for ‘abs’, values with abs(y) >= th; for ‘pos’, y >= th; for ‘neg’, y <= -th). The threshold is applied to y itself, so the series should be z-scored. At each threshold it records:

  1. the mean gap (in samples) between successive events, and its standard error (std of the gaps / sqrt of their number),

  2. the percentage of points that are events (the number of gaps over the number of candidate points, times 100),

  3. the median and mean time of the events, rescaled so that the start of the series is -1, the middle is 0 and the end is 1, and std(times)/sqrt(their number) (in samples).

The sweep stops when events are 2% or fewer of the points. The outputs measure how these curves change with th, using exponential [f(x) = a*exp(b*x) + c] and linear [f(x) = a*x + b] fits, and simple statistics across thresholds. The exponential fits are global least-squares fits (pyhctsa.robust.bf_exp_fit()), which need no starting point; only the rate b and the fit quality are returned. If a fit is degenerate (a constant curve, too few points), its outputs are NaN.

If fixed_thresh is given, the sweep and fits are skipped, and the statistics in (1)-(3) are returned for that one threshold.

Parameters:
y : array-like

The input time series (ideally z-scored).

threshold_how : {'abs', 'pos', 'neg'}, optional

The method for determining outliers:

  • ’abs’: values furthest from zero in either direction (default).

  • ’pos’: the greatest positive values.

  • ’neg’: the greatest negative values.

inc : float, optional

The increment to move through (in units of the standard deviation if the time series is z-scored). Default is 0.01. Unused when fixed_thresh is given.

fixed_thresh : float, optional

A single threshold (in the units of y, e.g., 2 for two standard deviations of a z-scored series). If given, the sweep is skipped.

Returns:

From the sweep (fixed_thresh not given):

  • mfexpb, mfexpr2, mfexprmse: the rate b, R^2 and root-mean-square error of the exponential fit to the mean gap vs. th;

  • nfexpb, nfexpr2, nfexprmse: the same for an exponential fit to the percentage of points that are events vs. th;

  • nfla, nflb, nflr2, nflrmse: slope a, intercept b, R^2 and RMSE of a linear fit to the percentage of points that are events vs. th;

  • mdtm, mdtmd, mdtstd: mean, median and standard deviation of the mean gap across thresholds;

  • mdrm, mdrmd, mdrstd: mean, median and standard deviation, across thresholds, of the median time of the events (-1 to 1);

  • mrm, mrmd, mrstd: the same for the mean time of the events;

  • xcmerr1, xcmerrn1: cross-correlation between the mean gap and its standard error across thresholds, at lags +1 and -1 (the means are not removed);

  • stdrfexpb, stdrfexpr2, stdrfexprmse: the rate and fit quality of an exponential fit to std(times)/sqrt(their number) vs. th;

  • stdrfla, stdrflb, stdrflr2, stdrflrmse: the same for a linear fit.

From a single threshold (fixed_thresh given; all NaN except propIncluded if events are 2% or fewer of the points): meanDt, seDt (the mean gap between events and its standard error), propIncluded (the percentage of points that are events), medianRelTime, meanRelTime (the median and mean time of the events, -1 to 1) and stdRelTime (std(times)/sqrt(their number), in samples).

A constant time series returns NaN.

Return type:

dict