Time Series, or Statistics for Stochastic Processes and Dynamical Systems

23 Jun 2009 08:38

---

Rates of convergence of estimators; confidence intervals, analogs to VC-dimension results (see Meir's paper below). Large deviation techniques; why are large deviation rate functionals, when they exist, generally relative entropies? Prediction schemes. Are there universal schemes which do not demand exponentially growing volumes of data? Can any of the "universal algorithm" schemes actually be used for anything?

If you have an ergodic process, then the sample-path mean for any nice statistic you care to measure will, almost surely, converge to the distributional mean. This is even true of trajectory probabilities (i.e., if you want to know the probability of a certain finite-length trajectory, simply count how often it happens.) So "sit and count" is a reliable and consistent statistical procedure. If the process mixes sufficiently quickly, the rate of convergence might even be respectable. But this doesn't say anything about the efficiency of such procedures, which is surely a consideration. And what do you do for non-ergodic processes? (Take multiple runs and hope they're telling you about different ergodic components?) Non-stationary, even?

I need to learn more about frequency-domain approaches; despite being raised as a physicist, I find the time domain much more natural. After all, the frequency domain is effectively just one choice of a function basis, and there are infinitely many others, which might in some sense be more appropriate to the process at hand. But that's at least in part a rationalization against having to learn more math.

LSE econometrics and its "general-to-specific" modeling procedure is very interesting, and I think possibly even related to stuff I've done, but I need to understand it much better than I do.

(This notebook probably needs subdivision.)

See also: Control Theory; Dynamical Systems; Ergodic Theory; Filtering, State Estimation and Signal Processing; Grammatical Inference; Information Theory; Machine Learning, Statistical Inference and Induction; Markov Models and Hidden Markov Models; Neural Coding; Power Law Distributions, 1/f Noise and Long-Memory Processes; Recurrence Times of Stochastic Processes (also Hitting, Waiting, and First-Passage Times) Sequential Decisions Under Uncertainty; State-Space Reconstruction; Statistical Learning Theory with Dependent Data; Statistics; Stochastic Processes; Symbolic Dynamics; Universal Prediction Algorithms

Recommended, big picture:
• M. S. Bartlett, An Introduction to Stochastic Processes, with Special Reference to Methods and Applications [Classic stuff on likelihood theory for stochastic processes]
• Ishwar V. Basawa and B. L. S. Prakasa Rao, Statistical Inference for Stochastic Processes [Assumes familiarity with normal theoretical statistics, i.e., you have to have already been taught to care about confidence intervals, hypothesis tests, estimation efficiency, etc. But very nice, given that background.]
• Peter Guttorp, Stochastic Modeling of Scientific Data [An excellent introduction to statistical inference for many different kinds of dependent data, not just time series; can be used by scientists and statisticians.]
• Holger Kantz and Thomas Schreiber, Nonlinear Time Series Analysis [An excellent presentation of the nonlinear dynamical systems approach, which comes out of physics]
• Judy Klein, Statistical Visions in Time: A History of Time-Series Analysis, 1662--1938
• Robert Shumway and David Stoffer, Time Series Analysis and Its Applications [A standard applied statistics text]
• Jorma Rissanen, Stochastic Complexity in Statistical Inquiry [Review: Less Is More, or, Ecce data!]
• David Ruelle, Chaotic Evolution and Strange Attractors: The Statistical Analysis of Deterministic Nonlinear Systems [From notes prepared by Stefano Isola]
• Norbert Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series

Recommended, closeups:
• Markus Abel, K. H. Andersen and Guglielmo Lacorata, "Hierarchical Markovian modeling of multi-time systems," nlin.CD/0201027
• Miika Ahdesmäki, Harri Lähdesmäki, Ron Pearson, Heikki Huttunen, and Olli Yli-Harja, "Robust detection of periodic time series measured from biological systems", BMC Bioinformatics 6 (2005): 117 [Open access, yay!]
• Jushan Bai, "Testing parametric conditional distributions of dynamic models", The Review of Economics and Statistics 85 (2003): 531--549 [Proposes "a nonparametric test for parametric conditional distributions of dynamic models. The test is of the Kolmogorov type.... It is asymptotically distribution-free and has nontrivial power against root-n local alternatives..."]
• Matthew J. Beal, Zoubin Ghahramani and Carl Edward Rasmussen, "The Infinite Hidden Markov Model", in NIPS 14 [Link]
• Patrick Billingsley, Statistical Inference for Markov Processes [Discrete-time and cadlag processes only]
• Bosq, Nonparametric Statistics for Stochastic Processes
• David Brillinger
  • "Remarks concerning graphical models for time series and point processes," Revista de Econometria 16 (1996): 1--23
  • "Second-order moments and mutual information in the analysis of time series and point processes," Proceedings of the Conference Statistics 2001 Canada [online]
  • "Does anyone know when the correlation coefficient is useful?: A study of the times of extreme river flows," Technometrics 43 (2001), 266-273
• S. Caires and J. A. Ferreira, "On the Non-parametric Prediction of Conditionally Stationary Sequences", Statistical Inference for Stochastic Processes 8 (2005): 151--184
• Luca Capriotti
  • "A Closed-Form Approximation of Likelihood Functions for Discretely Sampled Diffusions: the Exponent Expansion", physics/0703180
  • "The Exponent Expansion: An Effective Approximation of Transition Probabilities of Diffusion Processes and Pricing Kernels of Financial Derivatives", physics/0602107 = International Journal of Theoretical and Applied Finance 9 (2006): 1179--1199
• Tianjiao Chu and Clark Glymour, "Search for Additive Nonlinear Time Series Causal Models", Journal of Machine Learning Research 9 (2008): 967--991
• Jérôme Dedecker, Paul Doukhan, Gabriel Lang, José Rafael León R., Sana Louhichi and Clémentine Prieur, Weak Dependence: With Examples and Applications
• Piet de Jong and Jeremy Penzer, "ARMA models in state space form", Statistics and Probability Letters 70 (2004): 119--125 [preprint]
• Piet De Jong, "A Cross-Validation Filter for Time Series Models", Biometrika 75 (1988): 594--600 [JSTOR]
• Victor H. de la Pena, Rustam Ibragimov, and Shaturgun Sharakhmetov, "Characterizations of joint distributions, copulas, information, dependence and decoupling, with applications to time series", math.ST/0611166
• Andrew M. Fraser, Hidden Markov Models and Dynamical Systems [Review: Statistics of Moving Shadows]
• Neil Gershenfeld, B. Schoner and E. Metois, "Cluster-Weighted Modelling for Time-Series Analysis," Nature 397 (1999): 329--332 [Also described in Gershenfeld's incredible Nature of Mathematical Modeling]
• Gershenfeld and Weigend (eds.), Time Series Prediction: Forecasting the Future and Understanding the Past
• Christian Gouriéroux and Alain Monfort, Simulation-Based Econometric Methods [Review: By Indirection Find Direction Out]
• Kevin D. Hoover and Stephen J. Perez, "Data-Mining Reconsidered: Encompassing and the General-to-Specific Approach to Specification Search," Econometrics Journal 2 (1999): 167--191
• Marc Joannides and Francois Le Gland, "Small Noise Asymptotics of the Bayesian Estimator in Nonidentifiable Models", Statistical Inference for Stochastic Processes 5 (2002): 95--130
• M. L. Kleptsyna, A. Le Breton and M.-C. Roubaud, "Parameter Estimation and Optimal Filtering for Fractional Type Stochastic Systems", Statistical Inference for Stochastic Processes 3 (2000): 173--182
• Rudolf Kulhavy, Recursive Nonlinear Estimation: A Geometric Approach [Includes, explicitly, estimation in time-series systems]
• Kevin Judd, "Chaotic-time-series reconstruction by the Bayesian paradigm: Right results by wrong methods," Physical Review E 67 (2003): 026212 [Word.]
• Guglielmo Lacorata, Ruben A. Pasmanter and Angelo Vulpiani, "Markov-chain approach to a process with long-time memory," nlin.CD/0110010 [A special case of a more general result encompassed in my paper with Cris Moore]
• Ron Meir, "Nonparametric Time Series Prediction Through Adaptive Model Selection," Machine Learning 39 (2000): 5--34 [PDF. Extending the "structural risk minimization" framework due to Vapnik to time series. Unfortunately Meir's approach demands knowledge of the mixing rate of the process, which we don't really know how to estimate, but this is a very encouraging first step.]
• Gusztáv Morvai, Sidney J. Yakowitz and Paul Algoet, "Weakly Convergent Nonparametric Forecasting of Stationary Time Series," IEEE Trans. Info. Theory 43 (1997): 483--498
• Martin Nilsson, "Generalized Singular Spectrum Time Series Analysis," physics/0205094
• Andrey Novikov, "Optimal sequential multiple hypothesis tests",arxiv:0811.1297
• James Ramsay, Giles Hooker, David Campbell and Jiguo Cao, "Parameter Estimation for Differential Equations: A Generalized Smoothing Approach", Journal of the Royal Statistical Society forthcoming (2007) [PDF preprint]
• P. A. Robinson, "Interpretation of scaling properties of electroencephalographic fluctuations via spectral analysis and underlying physiology," Physical Review E 67 (2003): 032902 [A polite but devastating demonstration that "detrended fluctuation analysis", per Gene Stanley & co., is an obfuscated way of looking at the power spectrum.]
• George G. Roussas, "Asymptotic distribution of the log-likelihood function for stochastic processes," Zeitschrift für Wahrscheinlickkeitstheorie und verwandte Gebiete 47 (1979): 31--46 [Elegant solution of a basic problem for a pretty broad class of processes; extends work in his 1972 book, listed below because I can't lay hands on it.]
• Daniil Ryabko and Boris Ryabko, "Testing Statistical Hypotheses About Ergodic Processes", arxiv:0804.0510
• Nobusumi Sagara, "Nonparametric maximum-likelihood estimation of probability measures: existence and consistency", Journal of Statistical Planning and Inference 133 (2005): 249--271 ["This paper formulates the nonparametric maximum-likelihood estimation of probability measures and generalizes the consistency result on the maximum-likelihood estimator (MLE). We drop the independent assumption on the underlying stochastic process and replace it with the assumption that the stochastic process is stationary and ergodic. The present proof employs Birkhoff's ergodic theorem and the martingale convergence theorem. The main result is applied to the parametric and nonparametric maximum-likelihood estimation of density functions." Very cool.]
• Statistical Inference for Stochastic Processes [Journal]
• Christopher C. Strelioff and Alfred W. Hübler, "Medium-Term Prediction of Chaos", Physical Review Letters 96 (2006): 044101
• Masanobu Taniguchi and Yoshihide Kakizawa, Asymptotic Theory of Statistical Inference for Time Series [Finally, a proper statistical treatment which doesn't confine itself to expletive-deleted ARMA processes. Neat information geometry too. Expensive but worth it.]
• Albert Vexler, "Martingale Type Statistics Applied to Change Point Detection", Communications in Statistics - Theory and Methods 37 (2008): 1207--1224
• Wei Biao Wu, "Nonlinear system theory: Another look at dependence", Proceedings of the National Academy of Sciences 102 (2005): 14150--14154 ["we introduce previously undescribed dependence measures for stationary causal processes. Our physical and predictive dependence measures quantify the degree of dependence of outputs on inputs in physical systems. The proposed dependence measures provide a natural framework for a limit theory for stationary processes. In particular, under conditions with quite simple forms, we present limit theorems for partial sums, empirical processes, and kernel density estimates. The conditions are mild and easily verifiable because they are directly related to the data-generating mechanisms."]

Things modesty forbids me to recommend:
• CRS, Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata [Ph.D. thesis, UW-Madison, 2001]
• CRS and Kristina Lisa Klinkner, "Blind Construction of Optimal Nonlinear Predictors for Discrete Sequences", cs.LG/0406011 = pp. 504--511 of Uncertainty in Artificial Intelligence: Proceedings of the Twentieth Conference (UAI 2004)

To read:
• Luis A. Aguirre, Ubiratan S. Freitas, Christophe Letellier and Jean Maquet, "Structure-selection techniques applied to continuous-time nonlinear models," Physica D 158 (2001): 1--18
• Luis B. Almeida, "MISEP - Linear and Nonlinear ICA Based on Mutual Information," Journal of Machine Learning Research submitted [online]
• Eduardo G. Altmann and Holger Kantz, "Recurrence time analysis, long-term correlations, and extreme events", physics/0503056
• Shun-ichi Amari, "Estimating Functions of Independent Component Analysis for Temporally Correlated Signals," Neural Computation 12 (2000): 2083--2107
• Heather M. Anderson, "Choosing Lag Lengths in Nonlinear Dynamic Models," Monash Econometric Working Paper [online]
• Claudia Angelini, Daniela Cavab, Gabriel Katul, and Brani Vidakovic, "Resampling hierarchical processes in the wavelet domain: A case study using atmospheric turbulence", Physica D 207 (2005): 24--40
• J. A. D. Aston, "Modeling macroeconomic time series via heavy tailed distributions", math.ST/0702844
• Gopal K. Basak and Zhan-Qian Lu, "Stationarity of Switching VAR and Other Related Models", math.ST/0507267
• Ishwar V. Basawa and D. J. Scott, Asymptotic Optimal Inference for Non-ergodic Models
• Nathaniel Beck and Jonathan N. Katz
  • "What to Do (and Not to Do) with Time-Series Cross-Section Data", American Political Science Review 89 (1995): 634--647 [JSTOR]
  • Commentary by the authors, American Political Science Review 100 (2006): 676--677
• István Berkes, Lajos Horváth and Shiqing Ling, "Estimation in nonstationary random coefficient autoregressive models", Journal of Time Series Analysis 30 (2009): 395--416 ["the unit root problem does not exist in the RCA model"!]
• Alain Berlinet and Gérar Biau, "Minimax Bounds in Nonparametric Estimation of Multidimensional Deterministic Dynamical Systems", Statistical Inference for Stochastic Processes 4 (2001): 229--248 ["We consider the problem of estimating a multidimensional discrete deterministic dynamical system from the first n+1 observations. We exhibit the optimal rate function ... the near neighbor estimator achives this optimal rate.... optimal rate function is defined from multidimensonal spacings which are edge lengths of simplicies associated with a triangulation of the Voronoi cells built from the observations." Sounds very cool!]
• Alain Berlinet and Christian Francq, "On the Identifiability of minimal VARMA representations", Statistical Inference for Stochastic Processes 1 (1998): 1--15
• Arturo Berrones, "Knowledge Network Approach to Noise Reduction", physics/0609048
• Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics ["recent developments in the field of probability and statistics for dependent data... from Markov chain theory and weak dependence with an emphasis on some recent developments on dynamical systems, to strong dependence in times series and random fields. ... section on statistical estimation problems and specific applications". Full blurb, contents]
• D. Blanke, D. Bosq and D. Guegan, "Modelization and Nonparametric Estimation for Dynamical Systems with Noise", Statistical Inference for Stochastic Processes 6 (2003): 267--290
• Janet M. Box-Steffensmeier and Bradford S. Jones, Event History Modeling: A Guide for Social Scientists [Blurb]
• Noelle Bru, Laurence Despres and Christian Paroissin, "A comparison of statistical models for short categorical or ordinal time series with applications in ecology", math.ST/0702706
• Prabir Burman, Edmond Chow and Deborah Nolan, "A Cross-Validatory Method for Dependent Data", Biometrika 81 (1994): 351--358 [JSTOR]
• T. D. Carozzi and A. M. Buckley
  • "Deriving the sampling errors of correlograms for general white noise", physics/0505145
  • "Sampling errors of correlograms with and without sample mean removal for higher-order complex white noise with arbitrary mean", physics/0506030
• Alexandre X. Carvalho and Martin A. Tanner, "Mixtures-of-Experts of Autoregressive Time Series: Asymptotic Normality and Model Specification", IEEE Transactions on Neural Networks 16 (2005): 39--56
• Kung-Sik Chan and Howell H. Tong, Chaos: A Statistical Perspective
• J.-R. Chazottes, P. Collet and B. Schmitt, "Statistical Consequences of Devroye Inequality for Processes. Applications to a Class of Non-Uniformly Hyperbolic Dynamical Systems", math.DS/0412167
• J.-R. Chazottes, E. Floriani and R. Lima, "Relative Entropy and Identification of Gibbs Measures in Dynamical Systems," Journal of Statistical Physics 90 (1998): 697--725
• Zhuo Chen and Yuhong Yan, "Time Series Models for Forecasting: Testing or Combining?", Studies in Nonlinear Dynamics and Econometrics 11:1 (2007): 3
• Zhiyi Chi, "Large deviations for template matching between point processes", Annals of Applied Probability 15 (2005): 153--174 = math.PR/0503463
• P. Cizek, W. Hardle, V. Spokoiny, "Adaptive pointwise estimation in time-inhomogeneous conditional heteroscedasticity models", arxiv:0903.4620 [I'm more interested in the idea of adaptively estimating non-stationary time series here than the finance application...]
• Clements and Hendry, Forecasting Non-Stationary Economic Time Series
• P. Collet, S. Martinez and B. Schmitt, "Asymptotic distribution of tests for expanding maps of the interval", Ergodic Theory and Dynamical Systems 24 (2004): 707--722 [Kolmogorov-Smironov-type results for the empirical distribution under the invariant measure of a dynamical system]
• Jacques J. F. Commandeur, An Introduction to State Space Time Series Analysis
• Daniel Commenges and Anne Gegout-Petit, "Likelihood inference for incompletely observed stochastic processes: ignorability conditions", math.ST/0507151 ["We define a general coarsening model for stochastic processes. We decribe incomplete data by means of sigma-fields and we give conditions of ignorability for likelihood inference."]
• Colleen D. Cutler and Daniel T. Kaplan (eds.), Nonlinear Dynamics and Time Series: Building a Bridge between the Natural and Statistical Sciences
• Serguei Dachian, Yury A. Kutoyants
  • "Hypotheses Testing: Poisson Versus Self-exciting", arxiv:0903.4636 = Scandinavian Journal of Statistics 33 (2006): 391
  • "On the Goodness-of-Fit Tests for Some Continuous Time Processes", arxiv:0903.4642 ["We present a review of several results concerning the construction of the Cramer-von Mises and Kolmogorov-Smirnov type goodness-of-fit tests for continuous time processes. As the models we take a stochastic differential equation with small noise, ergodic diffusion process, Poisson process and self-exciting point processes"]
• Arnak Dalalyan and Markus Reiss, "Asymptotic statistical equivalence for ergodic diffusions: the multidimensional case", math.ST/0505053
• Youri Davydov, "Remarks on Estimation Problem for Stationary Processes in Continuous Time", Statistical Inference for Stochastic Processes 4 (2001): 1--15
• D. Dehay and Yu. A. Kutoyants, "On confidence intervals for distribution function and density of ergodic diffusion process", Journal of Statistical Planning and Inference 124 (2004): 63--73
• Miguel A. Delgado, Javier Hidalgo and Carlos Velasco, "Distribution free goodness-of-fit tests for linear processes", math.ST/0603043 = Annals of Statistics 33 (2005): 2568--2609 [i.e., goodness-of-fit for the autocorrelation function]
• Thomas G. Dietterich, "Machine Learning for Sequential Data" [PDF. Thanks to Gustavo Lacerda for a pointer.]
• Dmitry Dolgopyat, Vadim Kaloshin, Leonid Koralov, "Sample path properties of the stochastic flows," math.PR/0111011
• Randal Douc, Eric Moulines and Tobias Ryden, "Asymptotic properties of the maximum likelihood estimator in autoregressive models with Markov regime", Annals of Statistics 32 (2004): 2254--2304 = math.ST/0503681
• K. Dzhaparidze, Parameter Estimation and Hypothesis Testing in Spectral Analysis of Stationary Time Series
• Pierre Duchesne, "On Testing for Serial Correlation with a Wavelet-Based Spectral Density Estimator in Multivariate Time Series", Econometric Theory 22 (2006): 633--676
• Michael Eichler, "Graphical modelling of multivariate time series", math.ST/0610654
• Jianqing Fan and Qiwei Yao, Nonlinear Time Series: Nonparametric and Parametric Methods [Blurb]
• Yanqin Fan, Qi Li and Insik Min, "A Nonparametric Bootstrap Test of Conditional Distributions", Econometric Theoy 22 (2006): 587--613
• Enrique Figueroa-Lopez and Christian Houdre, "Nonparametric estimation for Levy processes with a view towards mathematical finance", math.ST/0412351
• D. Florens and H. Pham, "Large Deviations in Estimation of an Ornstein-Uhlenbeck Model," Journal of Applied Probability 36 (1999): 60--77
• Jurgen Franke, Jens-Peter Kreiss and Enno Mammen, "Bootstrap of Kernel Smoothing in Nonlinear Time Series", Bernoulli 8 (2002): 1--37
• Christian Francq and Jean-Michel Zakoian, "Bartlett's formula for a general class of nonlinear processes", Journal of Time Series Analysis 30 (2009): 449--465
• Cheng-Der Fuh
  • "Asymptotic operating characteristics of an optimal change point detection in hidden Markov models", Annals of Statistics 32 (2004): 2305--2339 = math.ST/0503682
  • "SPRT and CUSUM in hidden Markov models", Annals of Statistics 31 (2003): 942--977
• Philip Hans Franses and Dick Van Dijk, Non-Linear Time Series Models in Empirical Finance
• T. D. Frank, "Delay Fokker-Planck equations, perturbation theory, and data analysis for nonlinear stochastic systems with time delays", Physical Review E 71 (2005): 031106
• Roland Fried and Vanessa Didelez, "Latent variable analysis and partial correlation graphs for multivariate time series", Statistics and Probability Letters 73 (2005): 287--296
• Irene Gannaz and Olivier Wintenberger, "Adaptative density estimation with dependent observations", math.ST/0510311 [Using wavelets to estimate the invariant density of weakly-dependent processes, assumes geometric ergodicity but not stationarity]
• Tryphon T. Georgiou, "Distances between power spectral densities", math.OC/0607026
• Eric Ghysels and Denise R. Osborn, Econometric Analysis of Seasonal Time Series [Blurb]
• Basillis Gidas and Alejandro Murua, "Optimal transformations for prediction in continuous-time stochastic processes: finite past and future", Probability Theory and Related Fields 131 (2005): 479--492
• Ciprian Doru Giurcuaneanu and Jorma Rissanen, "Estimation of AR and ARMA models by stochastic complexity", math.ST/0702765
• Georg A. Gottwald and Ian Melbourne, "Testing for chaos in deterministic systems with noise", Physica D 212 (2005): 100--110
• Janez Gradisek, Silke Siegert, Rudolf Friedrich and Igor Grabec, "Analysis of time series from stochastic processes," Physical Review E 62 (2000): 3146--3155
• Grassberger and Nadal (eds.), From Statistical Physics to Statistical Inference and Back
• Grenander and Rosenblatt, Time Series
• David Gubbins, Time Series and Inverse Theory for Geophysicists
• Laszlo Gyorfi et al., Nonparametric Curve Estimation from Time Series
• Peter Hall, Soumendra Nath Lahiri and Jorg Polzehl, "On Bandwidth Choice in Nonparametric Regression with Both Short- and Long-Range Dependent Errors", Annals of Statistics 23 (1995): 1921--1936
• Wolfgang Hardle, Helmut Lutkepohl, Rong Chen, "A Review of Nonparametric Time Series Analysis", International Statistical Review 65 (1997): 49--72 [JSTOR]
• Jeffrey D. Hart, "Automated Kernel Smoothing of Dependent Data by using Time Series Cross-Validation", Journal of the Royal Statistical Society B 56 (1994): 529--542 [JSTOR]
• N. Hadyn, J. Luevano, G. Mantica and S. Vaienti, "Multifractal properties of return time statistics," nlin.CD/0108050
• Andrew Harvey et al (eds.), State Space and Unobserved Component Models: Theory and Applications
• David Hendry
  • Dynamic Econometrics
  • Econometrics: Alchemy or Science?
• David F. Hendry and Bent Nielsen, Econometric Modeling: A Likelihood Approach [Blurb, preface, ch.1 ]
• Junichi Hirukawa and Masanobu Taniguchi, "LAN theorem for non-Gaussian locally stationary processes and its applications", Journal of Statistical Planning and Inference 136 (2006): 640--688
• Jinh Hu, Wen-wen Tung, Jianbo Gao and Yinhe Cao, "Reliability of the 0-1 test for chaos", Physical Review E 72 (2005): 056207 [On Gottwald and Melbourne]
• Jianhua Z. Huang and Lijian Yang, "Identification of Non-Linear Additive Autoregressive Models", Journal of the Royal Statistical Society B 66 (2004): 463--477 [JSTOR. Proves consistency under the assumption that the data-generating process is strictly stationary and strongly mixing.]
• Stefano M. Iacus, "Statistical analysis of stochastic resonance with ergodic diffusion noise," math.PR/0111153
• Massimiliano Ignaccolo, P. Allegrini, P. Grigolini, P. Hamilton and Bruce J. West
  • "Scaling in Non-stationary time series I," physics/0301057
  • "Scaling in Non-stationary Time Series II: Teen Birth Phenomenon," physics/0301058
• Ching-Kang Ing, Jin-Lung Lin, Shu-Hui Yu, "Toward optimal multistep forecasts in non-stationary autoregressions", Bernoulli 15 (2009): 402--437 = arxiv:0906.2266 ["Optimal" assuming that you know you are facing a linear AR model.]
• Atsushi Inoue and Lutz Kilian, "In-sample or out-of-sample tests of predictability: which one should we use?", European Central Bank Working Paper [PDF]
• Akihiko Inoue and Yukio Kasahara, "Explicit representation of finite predictor coefficients and its applications", math.ST/0405051 = Annals of Statistics 34 (2006): 973--993
• D. A. Ioannides and D. P. Papanastassiou, "Estimating the distribution function of a stationary process involving measurement errors", Statistical Inference for Stochastic Processes 4 (2001): 181--198
• E. L. Ionides, C. Breto and A. A. King, "Inference for nonlinear dynamical systems", Proceedings of the National Academy of Sciences (USA) 103 (2006): 18438--18443
• S. Ishii and M.-A. Sato, "Reconstruction of chaotic dynamics by on-line EM algorithm," Neural Networks 14 (2001): 1239--1256
• George Kapetanios and Massimiliano Marcellino, "A Comparison of Estimation Methods for Dynamic Factor Models of Large Dimensions" [PDF]
• L. Kaplan, "Correlation function bootstrapping in quantum chaotic systems", Physical Review E 71 (2005): 056212
• Yan Karklin and Michael S. Lewicki, "A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals", Neural Computation 17 (2005): 397--423
• Matthew B. Kennel, "Testing time symmetry in time series using data compression dictionaries", Physical Review E 69 (2004): 056208
• Tae Yoon Kim and Sangyeol Lee, "Kernel density estimator for strong mixing processes", Journal of Statistical Planning and Inference 133 (2005): 273--284
• Jon Kleinberg, "Bursty and Hierarchical Structure in Streams" [PDF]
• D. Kleinhans, R. Friedrich, "Maximum Likelihood Estimation of Drift and Diffusion Functions", physics/0611102
• D. Kleinhans, R. Friedrich, A. Nawroth and J. Peinke, "An iterative procedure for the estimation of drift and diffusion coefficients of Langevin processes", Physics Letters A 346 (2005): 42--46 = physics/0502152 ["The analysis is based on an iterative procedure minimizing the Kullback-Leibler distance between measured and estimated two time joint probability distributions of the process."]
• M. L. Kleptsyna and A. Le Breton, "Statistical Analysis of the Fractional Ornstein-Uhlenbeck Type Process", Statistical Inference for Stochastic Processes 5 (2002): 229--248
• Rahul Konnur, "Estimation of all model parameters of chaotic systems from discrete scalar time series measurements", Physics Letters A 346 (2005): 275--280
• D. Kugiumtzis, "Statically Transformed Autoregressive Process and Surrogate Data Test for Nonlinearity," nlin.CD/0110025
• Uwe Küchler and Michael Sørensen, Exponential Families of Stochastic Processes
• Hans R. Künsch, "State Space and Hidden Markov Models", pp. 109--173 in Ole E. Barndorff-Nielsen, David R. Cox and Claudia Klüppelberg (eds.), Complex Stochastic Systems
• Y. A. Kutoyants
  • Statistical Inference for Ergodic Diffusion Processes
  • "On the Goodness-of-Fit Testing for Ergodic Diffusion Processes", arxiv:0903.4550
  • "Goodness-of-Fit Tests for Perturbed Dynamical Systems", arxiv:0903.4612
  • "On Properties of Estimators in non Regular Situations for Poisson Processes", arxiv:0903.4613
• B. Lacaze, "Errorless uniform sampling of complex stationary processes," Signal Processing 83 (2003): 913--917
• Stephen M. S. Lee and P. Y. Lai, "Improving coverage accuracy of block bootstrap confidence intervals", arxiv:0804.4361
• J. Lember and A. Koloydenko, "Adjusted Viterbi training", math.ST/0406237
• Daniel Lemire, "A Better Alternative to Piecewise Linear Time Series Segmentation", cs.DB/0605103
• N. N. Leonenko and L. M. Sakhno, "On the Whittle estimators for some classes of continuous-parameter random processes and fields", Statistics and Probability Letters 76 (2006): 781--795
• J. K. Lindsey, Statistical Analysis of Stochastic Processes in Time [old draft in Postscript; data and R code]
• Shiqing Ling and Howell Tong, "Testing for a linear MA model against threshold MA models", math.ST/0603040 = Annals of Statistics 33 (2005): 2529--2552
• Yu. N. Lin'kov, Asymptotic Statistical Methods for Stochastic Processes [Restricted to semi-martingales. blurb]
• E. Locherbach, "Likelihood Ratio Processes for Markovian Particle Systems with Killing and Jumps", Statistical Inference for Stochastic Processes 5 (2002): 153--177
• Wei Lu, Namrata Vaswani, "The Wiener-Khinchin Theorem for Non-wide Sense stationary Random Processes" ["under certain assumptions, the power spectral density (PSD) of any random process is equal to the Fourier transform of the time-averaged autocorrelation function"]
• Xiaodong Luo, Tomomichi Nakamura and Michael Small, "Surrogate data method applied to nonlinear time series", nlin.CD/0603004
• Xiaodong Luo, Jie Zhang, Junfeng Sun, Michael Small, Irene Moroz, "Asymptotically pivotal statistic for surrogate testing with extended hypothesis", nlin.CD/0701008
• Xiaodong Luo, Jie Zhang and Michael Small, "Exact nonparametric inference for detection of nonlinear determinism", nlin.CD/0507049 [More exactly, this is an exact test for linear stochasticity --- rejecting the null indicates either nonlinearity or determinism, or both.]
• Enno Mammen and Swagata Nandi, "Change of the nature of a test when surrogate data are applied", Physical Review E 70 (2004): 016121
• Heikki Mannila and Dmitry Rusakov, "Decomposition of Event Sequences into Independent Components" [short and long versions in PS]
• T. K. March, S. C. Chapman and R. O. Dendy, "Recurrence plot statistics and the effect of embedding", physics/0502042
• Pierre-Francois Marteau, "Time Warp Edit Distances with Stiffness Adjustment for Time Series Matching", cs.IR/0703033
• Norbert Marwan and Jurgen Kurths, "Nonlinear analysis of bivariate data with cross recurrence plots," physics/0201061
• Norbert Marwan, M. Thiel, N. R. Nowaczyk, "Cross Recurrence Plot Based Synchronization of Time Series," physics/0201062
• Norbert Marwan, N. Wessel, U. Meyerfeldt, A. Schirdewan, J. Kurths, "Recurrence Plot Based Measures of Complexity and its Application to Heart Rate Variability Data," physics/0201064
• Ikuo Matsuba, Hiroshi Takahashi and shinya Wakasa, "Stochastically Equivalent Dynamical System Approach to Nonlinear Deterministic Prediction", International Journal of Bifurcation and Chaos 16 (2006): 2721--2728 [I can't tell, from the abstract, if they're proposing to use stochastic systems to predict deterministic ones or vice versa; it'd be interesting either way!]
• Muneya Matsui, "A characterization of ARMA and Fractional ARIMA models with infinitely divisible innovations", math.ST/0703731
• Patrick E. McSharry and Leonard A. Smith, "Consistent nonlinear dynamics: identifying model inadequacy", nlin.CD/0401024 = Physica D 192 (2004): 1--22
• Javier R. Movellan, Paul Mineiro, and R. J. Williams, "A Monte Carlo EM Approach for Partially Observable Diffusion Processes: Theory and Applications to Neural Networks," Neural Computation 14 (20020: 1507--1544
• Eric Moulines, Pierre Priouret and Francois Roueff, "On recursive estimation for time varying autoregressive processes", math.ST/0603047 = Annals of Statistics 33 (2005): 2610--2654
• Jose M. F. Moura and Sanjoy K. Mitter, "Identification and Filtering: Optimal Recursive Maximum Likelihood Approach" [1986 technical report from MIT, found looking for something else, original URL now lost --- presumably long since published]
• George V. Moustakides, "Sequential change detection revisited", arxiv:0804.0741 = Annals of Statistics 36 (2008): 787--807
• Hans-Georg Muller and Ulrich Stadtmuller, "Generalized functional linear models", math.ST/0505638 = Annals of Statistics 33 (2005): 774--805 ["We propose a generalized functional linear regression model for a regression situation where the response variable is a scalar and the predictor is a random function. A linear predictor is obtained by forming the scalar product of the predictor function with a smooth parameter function, and the expected value of the response is related to this linear predictor via a link function. If, in addition, a variance function is specified, this leads to a functional estimating equation which corresponds to maximizing a functional quasi-likelihood. This general approach includes the special cases of the functional linear model, as well as functional Poisson regression and functional binomial regression. The latter leads to procedures for classification and discrimination of stochastic processes and functional data. ... An essential step in our proposal is dimension reduction by approximating the predictor processes with a truncated Karhunen-Loeve expansion."]
• Ursula U. Müller, Anton Schick and Wolfgang Wefelmeyer, "Estimating the innovation distribution in nonparametric autoregression", Probability Theory and Related Fields 144 (2009): 53--77 ["We prove a Bahadur representation for a residual-based estimator of the innovation distribution function in a nonparametric autoregressive model. The residuals are based on a local linear smoother for the autoregression function."]
• Tomomichi Nakamura, Yoshito Hirata, and Michael Small, "Testing for correlation structures in short-term variabilities with long-term trends of multivariate time series", Physical Review E 74 (2006): 041114
• Tomomichi Nakamura, Xiaodong Luo, and Michael Small, "Testing for nonlinearity in time series without the Fourier transform", Physical Review E 72 (2005): 055201
• Tomomichi Nakamura and Michael Small, "Small-shuffle surrogate data: Testing for dynamics in fluctuating data with trends", Physical Review E 72 (2005): 056216
• Ilia Negri, "Efficiency of a class of unbiased estimators for the invariant distribution function of a diffusion process", math.ST/0609590
• Jimmy Olsson, Olivier Cappe, Dandal Douc and Eric Moulines, "Sequential Monte Carlo smoothing with application to parameter estimation in non-linear state space models", math.ST/0609514
• Sorinel Adrian Oprisan, "An application of the least-squares method to system parameters extraction from experimental data", Chaos 12 (2002): 27--32
• Brahim Ouhbi and Nikolaos Limnios, "Nonparametric estimation for semi-Markov processes based on its hazard rate functions", Statistical Inference for Stochastic Processes 2 (1999): 151--173
• Yacine Oussar and Gérard Dreyfus, "How to be a gray box: dynamic semi-physical modeling," Neural Networks 14 (2001): 1161--1172
• P. Palaniyandi and M. Lakshmanan, "Estimation of System Parameters and Predicting the Flow Function from Time Series of Continuous Dynamical Systems", nlin.CD/0406027
• Milan Palus, "Coarse-grained entropy rate for characterization of complex time series", Physica D 93 (1996): 64--77 [Thanks to Prof. Palus for a reprint]
• Zacharias Psaradakis
  • "A sieve bootstrap test for stationarity," Statistics and Probability Letters 62 (2003): 263--274
  • "Blockwise bootstrap testing for stationarity", Statistics and Probability Letters 76 (2006): 562--570
• Zacharias Psaradakis, Martin Sola, Fabio Spagnolo and Nicola Spagnolo, "Selecting nonlinear time series models using information criteria", Journal of Time Series Analysis 30 (2009): 369--394
• N. U. Prabhu and Ishawar V. Basawa (eds.), Statistical Inference in Stochastic Processes (1991)
• B. L. S. Prakasa Rao
  • Semimartingales and Their Statistical Inference
  • Statistical Inference for Diffusion-Type Processes
• E. Racca and A. Porporato, "Langevin equations from time series", Physical Review E 71 (2005): 027101
• Ali Rahimi, Learning to Transform Time Series with a Few Examples [Ph.D. thesis, MIT dept. of electrical engineering and computer science, 2005. PDF]
• Ali Rahimi, Ben Recht and Trevor Darrell, "Learning to Transform Time Series with a Few Examples", tech report [PDF]
• M. M. Rao, Stochastic Processes: Inference Theory
• Ramiro Rico-Martinez, K. Krischer, G. Flaetgen, J.S. Anderson and I. G. Kevrekidis, "Adaptive Detection of Instabilities: An Experimental Feasibility Study," nlin.CD/0202057
• Christoph Rieke, Ralph G. Andrzejak, Florian Mormann and Klaus Lehnertz, "Improved statistical test for nonstationarity using recurrence time statistics", Physical Review E 69 (2004): 046111 [link]
• John C. Robertson, Ellis W. Tallman and Charles H. Whiteman, "Forecasting using relative entropy," Federal Reserve Bank of Atlanta Working Paper 2002-20 [PDF]
• J. W. C. Robinson, J. Rung, A. R. Bulsara and M. E. Inchiosa, "General measures for signal-noise separation in nonlinear dynamical systems," Physical Review E 63 2000: 011107
• G. G. Roussas, Contiguity of Probability Measures: Some Applications in Statistics [1972; "established a modern and elegant approach for statistical analysis of a Markov proces. Using the concept of contiguity and LAN {local asymptotic normality}, a concept which goes back to LeCam, he studied asymptotic optimality of sequences of estimators and tests. The description is systematic and mathematically rigorous." --- Taniguchi and Kakizawa, p. 63]
• Boris Ryabko and Jaakko Astola
  • "Universal Codes as a Basis for Time Series Testing", cs.IT/0602084
  • "Universal Codes as a Basis for Nonparametric Testing of Serial Independence for Time Series", cs.IT/0506094
• Nicola Scafeta, Patti Hamilton and Paolo Grigolini, "The Thermodynamics of Social Processes: The Teen Birth Phenomenon," cond-mat/0009020 [Not because I believe them about sociology, but because they claim to have new and powerful nonparametric methods for detecting and quantifying memory in time series]
• Thomas Schreiber and Andreas Schmitz, "Surrogate time series," chao-dyn/9909037
• Reiner Schulz and James A. Reggia, "Temporally Asymmetric Learning Supports Sequence Processing in Multi-Winner Self-Organizing Maps", Neural Computation 16 (2004): 535--561 [the "model presented here raises the possibility that SOMs may ultimately prove useful as visualization tools for temporal sequences and as preprocessors for sequence pattern recognition systems."]
• Xiaofeng Shao, Wei Biao Wu, "Asymptotic spectral theory for nonlinear time series", math.ST/0611029
• M. Siefert, J. Peinke and R. Friedrich, "On a quantitative method to analyse dynamical and measurement noise," physics/0108034
• Przemyslaw Sliwa and Wolfgang Schmid, "Monitoring the cross-covariances of a multivariate time series", Metrika 61 (2005): 89--115
• A. Sitz, U. Schwarz, J. Kurths, H. U. Voss, "Estimation of parameters and unobserved components for nonlinear systems from noisy time series," Physical Review E 66 (2002): 016210
• Michael Small
  • Applied Nonlinear Time Series Analysis: Applications in Physics, Physiology and Finance
  • "Optimal time delay embedding for nonlinear time series modeling", nlin.CD/0312011
• Michael Small and Kevin Judd, "Detecting periodicity in experimental data using linear modeling techniques", physics/9810021
• Vadim N. Smelyanskiy and Dmitry G. Luchinsky, "Inference of stochastic nonlinear oscillators with applications to physiological problems", physics/0403121 [They present this as a Bayesian inference issue, but the core of their work appears, from skimming, to be an efficient method for computing the likelihood, so it'd apply equally well to maximum likelihood inference, for instance.]
• V. N. Smelyanskiy, D. A. Timucin, A. Bandrivskyy and D. G. Luchinsky, "Model reconstruction of nonlinear dynamical systems driven by noise", physics/0310062 [Same as earlier paper --- was this one submitted to PRL?]
• Dmitry A. Smirnov, Vladislav S. Vlaskin and Vladimir I. Ponomarenko, "Estimation of parameters in one-dimensional maps from noisy chaotic time series", Physics Letters A 336 (2005): 448--458
• Eduardo D. Sontag, "For differential equations with r parameters, 2r+1 experiments are enough for identification," math.DS/0111135
• D. Sornette and V. F. Pisarenko, "Properties of a simple bilinear stochastic model: estimation and predictability", physics/0703217
• Tomoya Suzuki, Tohru Ikeguchi, and Masuo Suzuki, "Effects of data windows on the methods of surrogate data", Physical Review E 71 (2005): 056708
• Alexander G. Tartakovsky, "Asymptotic Optimality of Certain Multihypothesis Sequential Tests: Non-i.i.d. Case", Statistical Inference for Stochastic Processes 1 (1998): 265--295
• Marco Thiel, M. Carmen Romano and Jurgen Kurths, "How much information is contained in a recurrence plot?", Physics Letters A 330 (2004): 343--349
• Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, Michael P.H. Stumpf, "Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems", arxiv:0901.1925
• Wilson Truccolo, John P. Donoghue, "Nonparametric Modeling of Neural Point Processes via Stochastic Gradient Boosting Regression", Neural Computation 19 (2007): 672-705
• Ciprian A. Tuder and Frederi G. Viens, "Statistical Aspects of the Fractional Stochastic Calculus", math.ST/0609295
• Masayuki Uchida and Nakahiro Yoshida, "Information Criteria in Model Selection for Mixing Processes", Statistical Inference for Stochastic Processes 4 (2001): 73--98 ["The emphasis is put on the use of the asymptotic expansion of the distribution of an estimator based on the conditional Kullback-Leibler divergence for stochastic processes. Asymptotic properties of information criteria and their improvement are discusssed."]
• Aad van der Vaart and Harry van Zanten, "Donsker theorems for diffusions: Necessary and sufficient conditions", math.PR/0507412 = Annals of Probability 33 (2005): 1422--1451
• Harry van Zanten, "On Uniform Laws of Large Numbers for Ergodic Diffusions and Consistency of Estimators", Statistical Inference for Stochastic Processes 6 (2003): 199--213 ["In contrast with uniform laws of large numbers for i.i.d. random variables, we do not need conditions on the 'size' of the class [of functions] in terms of bracketing or covering numbers. The result is a consequence of a number of asymptotic properties of diffusion local time that we derive."]
• J. H. van Zanten, "On the Uniform Convergence of the Empirical Density of an Ergodic Diffusion", Statistical Inference for Stochastic Processes 3 (2000): 251--262
• P. F. Verdes, P. M. Granitto and H. A. Ceccatto, "Overembedding Method for Modeling Nonstationary Systems", Physical Review Letters 96 (2006): 118701
• Juan M. Vilar-Fernandez and Ricardo Cao, "Nonparametric Forecasting in Time Series - A Comparative Study", Communications in Statistics: Simulation and Computation 36 (2007): 311--334
• R. Vilela Mendes, R. Lima and T. Araujo, "A Process-Reconstruction Analysis of Market Fluctuations," cond-mat/0102301 [I don't care about the market, but they claim to have a new method for identifying distributions over entire sequences]
• Hiroshi Wakuya and Jacek M. Zurada, "Bi-directional computing architecture for time series prediction," Neural Networks 14 (2001): 1307--1321
• Halbert White
  • "A consistent model selection procedure based on m-testing," in C. W. J. Granger (ed.), Modelling Economic Series, pp. 369--383
  • Asymptotic Theory for Econometricians
  • Estimation, Inference and Specification Analysis [Or, what shall we do with a mis-specified model?]
• Wei Biao Wu and Jan Mielniczuk, "Kernel Density Estimation for Linear Processes", Annals of Statistics 30 (2002): 1441--1459 [PDF]
• Herwig Wendt, Patrice Abry and Stephane Jaffard, "Bootstrap for Empirical Multifractal Analysis", IEEE Signal Processing Magazine July 2007, pp. 38--48 [+ technical papers by these authors]
• A. Zeileis and G. Grothendieck, "zoo: S3 Infrastructure for Regular and Irregular Time Series", Journal of Statistical Software 14 (2005): 1--27 = math.ST/0505527 ["zoo is an R package providing an S3 class with methods for indexed totally ordered observations, such as discrete irregular time series. Its key design goals are independence of a particular index/time/date class and consistency with base R and the "ts" class for regular time series."]
• Jacob Ziv, "A Universal Prediction Lemma and Applications to Universal Data Compression and Prediction", IEEE Transactions on Information Theory 47 (2001): 1528--1532
• M. Zukovic, D. T. Hristopulos, "Spartan Random Processes in Time Series Modeling", 0709.3418

To write/finish:
• CRS, "Learning Rates and Recurrence Times" [a.k.a. "Wait and see"]
• CRS, "Algorithms for Inferring the Statistical Structure of Symbol Sequences: History and Review"