pyhctsa.operations.model_fit.fit_subsegments

pyhctsa.operations.model_fit.fit_subsegments(y, model='ss', order=2, subset_how='rand', sample_p=(20, 0.1), random_seed='default')

Robustness of model parameters across different segments of a time series.

The spread of parameters obtained (including in-sample goodness of fit statistics) provides some indication of stationarity. Values of goodness of fit provide some indication of model suitability. Inherits strongly from compare_test_sets() (hctsa’s MF_FitSubsegments).

For a stationary series the estimates still differ between segments by sampling error, which shrinks as the segments (a fixed fraction of the series) grow. The spread across segments is therefore also reported relative to its expectation for a stationary series, which is about the same at any length and grows with the change in the dynamics relative to the sampling error:

  • *_stdratio ('ar', 'ss'): the SD of an estimate across segments divided by the square root of its expected sample variance if the segments differed by sampling error alone; about 1 for a stationary series.

  • a_k_maxdev ('ar'): the largest deviation of a segment’s AR coefficient from the mean over segments, divided by the SD of that deviation for a stationary series (the most deviant segment). It responds more than the SD to a change confined to a few segments.

  • excess_ratio ('arcrosspred'): the error of each segment’s model in predicting the other segments, in excess of their own models’ error, relative to its expectation for a stationary series; about 1 for a stationary series.

The sampling variances come from the segments’ own fits, pooled over segments, so that differences between segments do not enter:

  • AR coefficients: the least-squares estimate from a segment with n residuals has large-sample variance sigma^2 [Gamma^-1]_kk / n, with Gamma the covariance matrix of p consecutive values. Gamma is the pooled X'X / n of the segments’ regressions and sigma^2 their pooled residual sum of squares over n - p degrees of freedom each. Conditionally heteroskedastic innovations (as in a GARCH process) make the true variance larger, which raises the AR statistics too.

  • Log FPE: the log residual mean square of a segment with nu residual degrees of freedom (residuals minus parameters) has variance psi'(nu / 2) (kappa - 1) / 2: exact for Gaussian residuals (a log chi-square variable, psi' the trigamma function) and (kappa - 1) / n for large n with residuals of kurtosis kappa, estimated from all the residuals pooled. Volatility clustering (residual variance that is autocorrelated, as in a GARCH process) also raises the log-FPE statistics.

  • Overlapping segments: each estimate is to first order an average over the samples of its segment, so two estimates whose n_i and n_j samples share o_ij are correlated by o_ij / sqrt(n_i n_j); this sets the expected sample variance and the variance of each segment’s deviation from the mean.

  • Cross-prediction: for a stationary series, an AR(p) model fitted to n_i samples predicts a new segment with mean squared error sigma^2 (1 + p / n_i) and its own segment of n_j samples with sigma^2 (1 - p / n_j), to first order. The relative excess mse_ij / mse_jj - 1 therefore has expectation p (1 / n_i + 1 / n_j).

Parameters:
y : array-like

The input time series.

model : str, optional

The model to fit in each segment of the time series:

  • ’arsbc’: fits an AR model of the best order by the Schwarz Bayesian criterion

    (ARFIT algorithm, zero mean), from orders 1 to 10 and at most a fifth of the segment length (so that every candidate has at least four equations per coefficient; with fewer, the criterion often picks the largest order). Outputs are how the optimal order and the SBC vary across segments (orders_*, sbcs_*). The order input is not used.

  • ’ar’: fits an AR model of a specified order by forward-backward least

    squares. Outputs are how Akaike’s final prediction error (fpe_*, logfpe_stdratio) and the fitted AR parameters (a_k_*, as in the polynomial 1 + a_1 z^-1 + …, the negative of the usual coefficients) vary across segments.

  • ’arcrosspred’: splits the series into sample_p non-overlapping segments

    (requires subset_how='uniform' and a scalar-like sample_p, a segment count), fits an AR model of the given order to each, and uses every segment’s model to predict, one step ahead, every segment (including itself). The result is a matrix of cross-prediction root-mean-square errors (row: predicting model, column: predicted segment), of which the spread, off-diagonal and excess statistics are returned. NaN if any segment is shorter than 5 * (order + 1) or an AR fit fails.

  • ’ss’: fits a state-space model of the given order by subspace identification

    (n4sid; the order can be 'best'). Outputs are how Akaike’s final prediction error (fpe_*, logfpe_stdratio) varies across segments.

  • ’arma’: fits an ARMA model by prediction-error minimization (armax; order

    is [p, q]). Outputs are how the FPE (fpe_*) and the fitted AR (p_k_*) and MA (q_k_*) coefficients vary across segments. (Deregistered in hctsa, as it is much like ‘ar’ and slow.)

Default is 'ss'.

order : int or two-vector, optional

The order of the model to fit (used for ‘ar’, ‘ss’, or ‘arma’ models; a two-element vector [p, q] for ‘arma’). Default is 2.

subset_how : str, optional

How to choose segments from the time series, either:

  • ’uniform’ (evenly spaced)

  • ’spread’ (start points from the golden-ratio sequence frac(j*phi): deterministic and evenly spread, without the aliasing a regular spacing has with periodic series)

  • ’rand’ (at random).

Default is 'rand'.

sample_p : list, tuple or int, optional

A two-vector specifying how many segments to take and of what length. Of the form [n_samples, length], where length can be a proportion of the time-series length. For example, [20, 0.1] takes 20 segments of 10% the time-series length. For model='arcrosspred', an integer (or length-1 list): the number of non-overlapping segments to partition the series into. Default is [20, 0.1].

random_seed : int, 'default', 'none' or None, optional

The seed of the random segment starts (subset_how='rand'), as hctsa’s BF_RandomSeed: a number, 'default' or None for seed 0, or 'none' for a seed drawn from NumPy’s global state. The draws come from the portable generator (pyhctsa.robust.bf_random()), so they are hctsa’s. Default is 'default'.

Returns:

Dictionary of statistics on the spread and mean of fitted model parameters and goodness of fit across segments. For 'ar', 'ss' and 'arma', fpe_std, fpe_mean, fpe_max, fpe_min, fpe_range (of the final prediction error); for 'ar' and 'ss' also logfpe_stdratio (of the log FPE; NaN if a segment has no residual degree of freedom or the fits are exact to rounding). For 'ar', for each lag k: a_k_std, a_k_mean, a_k_max, a_k_min, a_k_stdratio and a_k_maxdev (NaN in the same cases); for 'arma' p_k_std, p_k_mean, p_k_max, p_k_min and the same for q_k_*. For 'arsbc': orders_mode, orders_mean, orders_std, orders_max, orders_min, orders_range (of the selected order) and sbcs_mean, sbcs_std, sbcs_range, sbcs_min, sbcs_max (of the minimum SBC, the log residual variance plus the penalty k log(n) / n). For 'arcrosspred': std, range, iqr (over all entries of the cross-prediction error matrix), stdoffdiag, rangeoffdiag, iqroffdiag (over the positive off-diagonal entries), stdmean, rangemean, stdmedian, rangemedian (across predicted segments, of the mean and of the median error), rangerange, stdrange, rangestd, stdstd (across predicted segments, of the range or standard deviation of the errors), mineig (smallest real part of the eigenvalues of the matrix) and excess_ratio (NaN if the fits are exact to rounding).

Return type:

dict