Notebooks

This is a place for EDA notebooks dealing with some concepts in applied math and statistics, dynamical systems, mathematical finance, econometrics, and machine learning. Detecting statistical shifts is important for monitoring stability in production data pipelines. Regime shifts are additionally a feature of many natural and physical phenomena worth modeling in themselves.

What are regimes?
https://regimelab.substack.com/p/what-are-regimes

Ergodic vs non-Ergodic Regimes
https://regimelab.substack.com/p/ergodic-regimes

Wikipedia
https://en.wikipedia.org/wiki/Regime_shift

Further Reading & Inspiration

Haven't read all of these but want to keep them for follow-up.

Climate, Ecosystem Regimes & Critical Slowing Down

I. The Theory of Parallel Climate Realizations Journal of Statistical Physics

II. Irreversibility of regime shifts in the North Sea Frontiers in Marine Science

III. Evidence of human influence on Northern Hemisphere snow loss Nature, Alexander R. Gottlieb, Justin S. Mankin

IV. Variational Bayes Estimation of Hidden Markov Models for Daily Precipitation with Semi-Continuous Emissions Department of Mathematics and Statistics, Joint Center for Earth Systems Technology, University of Maryland

V. A Bayesian Deep Learning Approach to Near-Term Climate Prediction Xihaier Luo, Balasubramanya T. Nadiga, Ji Hwan Park, Yihui Ren, Wei Xu, Shinjae Yoo, Journal of Advances in Modeling Earth Systems

VI. Variational inference at glacier scale Douglas J. Brinkerhoff

VII. Mapping Fire Regime Ecoregions in California USGS

VIII. Critical slowing down in dynamical systems driven by nonstationary correlated noise Christopher Boettner, Niklas Boers

IX. Spectral analysis of climate dynamics with operator-theoretic approaches Gary Froyland, Dimitrios Giannakis, Benjamin R. Lintner, Maxwell Pike & Joanna Slawinska

X. The most at-risk regions in the world for high-impact heatwaves Vikki Thompson, Dann Mitchell, Gabriele C. Hegerl, Matthew Collins, Nicholas J. Leach & Julia M. Slingo

XI. Maintaining human wellbeing as socio-environmental systems undergo regime shifts Andrew R. Tilman, Elisabeth H. Krueger, Lisa C. McManus, & James R. Watson

XII. Flickering as an early warning signal Vasilis Dakos, ETH Zurich, Egbert H. Van Nes

XIII. Non-equilibrium early-warning signals for critical transitions in ecological systems Li Xu, Denis Patterson, Simon Asher Levin, Jin Wang

XIV. Critical slowing down theory provides early warning signals for sandstone failure Yao Tang, Xing Zhu, Chunlei He, Jiewei Hu, Jie Fan

XV. Critical slowing down associated with regime shifts in the US housing market James Peng Lung Tan, Siew Siew Ann Cheong

XVI. Early Warning Signals for Critical Transitions Marten Scheffer, Jordi Bascompte, William A. Brock, Victor Brovkin, Stephen R. Carpenter, Vasilis Dakos, Hermann Held, Egbert H. van Nes, Max Rietkerk & George Sugihara

XVII. A ‘regime shift’ is happening in the Arctic Ocean, Stanford Scientists Say Josie Garthwaite

XVIII. A Brief Overview of the Regime shift Detection Methods NOAA, Sergei Rodionov, Joint Institute for the Study of the Atmosphere and Ocean, University of Washington

XIX. Researchers Study Effects of Harmful Algal Blooms NOAA

XX. Nonlinear El Niño impacts on the global economy under climate change Nature Communications

XXI. Significantly wetter or drier future conditions for one to two thirds of the world’s population Nature Communications

XXII. Food System Transformation: Integrating a Political–Economy and Social–Ecological Approach to Regime Shifts Intl Journal of Environmental Research & Public Health

XXIII. Precipitation regime change in Western North America: The role of Atmospheric Rivers Scientific Reports

XXIV. Atmospheric Rivers: What are they and how does NOAA study them? NOAA

XXV. Combined effects of the Pacific Decadal Oscillation and El Niño-Southern Oscillation on Global Land Dry–Wet Changes Scientific Reports

XXVI. Ancient Underwater Mountains in the Bay SF Baykeeper

XXVII. Biological communities in San Francisco Bay track large-scale climate forcing over the North Pacific Geophysical Research Letters

XXVIII. The Impact of Regime Shifts on Long-Range Persistence and the Scaling of Sea Surface Temperature Off the Coast of California Breaker, L. C., & Carroll, D. (2019), Journal of Geophysical Research

XXIX. Historical and future maximum sea surface temperatures Science Advances

XXX. A review of possible pathways of marine microplastics transport in the ocean Yanfang Li, Hua Zhang, Cheng Tang

XXXI. Microplastics in aquatic systems, a comprehensive review: origination, accumulation, impact, and removal technologies Antonio Tursi, Mariafrancesca Baratta, Thomas Easton, Efthalia Chatzisymeon, Francesco Chidichimo, Michele De Biase, Giovanni De Filpo

XXXII. Risks of hydroclimatic regime shifts across the western United States Subhrendu Gangopadhyay, Gregory McCabe, Gregory Pederson, Justin Martin & Jeremy S. Littell

XXXIII. Turning back from the brink: Detecting an impending regime shift in time to avert it Reinette Biggs, Stephen R. Carpenter, William A. Brock

XXXIV. Physics-based early warning signal shows that AMOC is on tipping course Rene M. Van Westen, Michael Kliphuis, Henk A. Dijkstra

XXXV. What is happening in the Atlantic Ocean to the AMOC? realclimate.org

XXXVI. Warning of a forthcoming collapse of the Atlantic meridional overturning circulation Peter Ditlevsen, Susanne Ditlevsen

XXXVII. A Generative Adversarial Network for Climate Tipping Point Discovery (TIP-GAN) Johns Hopkins University Applied Physics Laboratory, Duke University, Jennifer Sleeman, David Chung, Anand Gnanadesikan, Jay Brett, Yannis Kevrekidis, Marisa Hughes, Thomas Haine, Marie-Aude Pradal, Renske Gelderloos, Chace Ashcraft, Caroline Tang, Anshu Saksena, Larry White

XXXVIII. Predicting changes in bee assemblages following state transitions at North American dryland ecotones Melanie R. Kazenel, Karen W. Wright, Julieta Bettinelli, Terry L. Griswold, Kenneth D. Whitney, Jennifer A. Rudgers

XXXIX. In places most prone to wildfires and hurricanes, state “insurers of last resort” are absorbing trillions of dollars in risk Bloomberg

XXXX. Global warming and heat extremes to enhance inflationary pressures Maximilian Kotz, Friderike Kuik, Eliza Lis, Christiane Nickel

XXXXI. GraphCast AI Model for Weather Forecasting DeepMind

XXXXII. Where are the Coexisting Parallel Climates? M. Herein, T. Tél, T. Haszpra

Miscellaneous (ML, Bayesian stats, Physics, Generative Modeling)

I. A Bayesian perspective on severity: risky predictions and specific hypotheses Noah van Dongen, Jan Sprenger & Eric-Jan Wagenmakers

II. Popper’s Critical Rationalism as a Response to the Problem of Induction: Predictive Reasoning in the Early Stages of the Covid-19 Epidemic Tuodmo Peltonen

III. Transformers Can Do Bayesian Inference Samuel Muller, Noah Hollmann, Sebastian Pineda, Josif Grabocka, Frank Hutter

IV. The Algebra of Probable Inference Richard T. Cox

V. Online Variational Filtering and Parameter Learning Andrew Campbell, Yuyang Shi, Tom Rainforth, Arnaud Doucet

VI. Particle Mean Field Variational Bayes Minh-Ngoc Tran, Paco Tseng, Robert Kohn

VII. Attention is Kernel Trick Reloaded Gokhan Egri, Xinran (Nicole) Han

VIII. Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli

IX. Spectrum Estimation from Samples Weihao Kong, Gregory Valiant

X. Variational inference with a quantum computer Marcello Benedetti, Brian Coyle, Mattia Fiorentini, Michael Lubasch, Matthias Rosenkranz

XI. Generative modeling for time series via Schrödinger bridge Mohamed Hamdouche, Pierre Henry-Labordere, Huyên Pham

XII. EM Algorithm Tengyu Ma and Andrew Ng

XIII. Infinite Mixture of Global Gaussian Processes Fernando Perez-Cruz, Melanie Pradier

XIV. Dirichlet Process Yee Whye Teh, University College London

XV. Variational Inference David M. Blei

XVI. Graphical Models, Exponential Families, and Variational Inference Martin J. Wainwright, Michael I. Jordan, University of California, Berkeley

XVII. The Elements of Statistical Learning Trevor Hastie, Robert Tibshirani, Jerome Friedman

XVII. A New Approach to Data Driven Clustering Arik Azran, Gatsby Computational Neuroscience Unit, University College London, Zoubin Ghahramani, Department of Engineering, University of Cambridge, Cambridge

XIX. Particle Learning for Bayesian Non-Parametric Markov Switching Stochastic Volatility Model Bayesian Analysis

XXI. A Guide To Monte Carlo Simulations In Statistical Physics David P. Landau, Kurt Binder

XXII. Score-Based Generative Modeling through Stochastic Differential Equations Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole

XXIII. Denoising Diffusion Probabilistic Models Jonathan Ho, Ajay Jain, Pieter Abbeel

XXIV. B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data Liu Yanga, Xuhui Menga, George Em Karniadakisa

XXV. Deep Signature Transforms Patric Bonnier, Patrick Kidger, Imanol Perez Arribas, Cristopher Salvi, Terry Lyons

XXVI. A Wasserstein-type distance in the space of Gaussian Mixture Models Julie Delon, Agnès Desolneux

XXVII. Gaussian Processes for ML Rasmussen et al

XXVIII. Advances in Variational Inference IEEE

XXIX. Stochastic Optimization James C. Spall, The Johns Hopkins University, Applied Physics Laboratory

XXX. Bayesian estimation of information-theoretic metrics for sparsely sampled distributions Angelo Piga, Lluc Font-Pomarol, Marta Sales-Pardo, Roger Guimerà

XXXI. Variational Inference for Model-Free and Model-Based Reinforcement Learning Felix Leibfried

XXXII. Deep Kernel Learning Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing, CMU, University of Toronto

XXXIII. Exact Gaussian processes for massive datasets via non-stationary sparsity-discovering kernels Nature, Marcus M. Noack, Harinarayan Krishnan, Mark D. Risser & Kristofer G. Reyes

XXXIV. Non-Stationary Spectral Kernels Sami Remes, Markus Heinonen, Samuel Kaski

XXXV. Residual Diffusion Modeling for Km-scale Atmospheric Downscaling Morteza Mardani, Noah Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng-Chin Liu, Arash Vahdat, Karthik Kashinath, Jan Kautz, Mike Pritchard

XXXVI. Attention is All You Need Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

fBm & Hurst Effect/Long Memory

I. Basic properties of the Multivariate Fractional Brownian Motion Pierre-Olivier Amblard, Jean-François Coeurjolly, Frédéric Lavancier, Anne Philippe. Basic properties of the Multivariate Fractional Brownian Motion. Séminaires et congrès, 2013, 28, pp.65-87. ffhal00497639v2f

II. Learning Fractional White Noises in Neural Stochastic Differential Equations Anh Tong, Thanh Nguyen-Tang (Johns Hopkins University), Toan Tran (VinAI Research, Vietnam), Jaesik Choi

III. A Dynamical Systems Explanation of the Hurst Effect and Atmospheric Low-Frequency Variability Christian L. E. Franzke, Scott M. Osprey, Paolo Davini & Nicholas W. Watkins

IV. Long memory and regime switching Francis X. Diebold, Atsushi Inoue

V. On the continuing relevance of Mandelbrot’s non-ergodic fractional renewal models of 1963 to 1967 Nicholas W. Watkins, Centre for the Analysis of Time Series, London School of Economics and Political Science, London, UK, Centre for Fusion, Space and Astrophysics, University of Warwick, Coventry, UK, Faculty of Science, Technology, Engineering and Mathematics, Open University, Milton Keynes, UK

VI. The Zumbach effect under rough Heston Rados Radoicic, Mathieu Rosenbaum, Omar El Euch, Jim Gatheral, Baruch College, CUNY, Ecole Polytechnique

VII. Variational inference of fractional Brownian motion with linear computational complexity Hippolyte Verdier, François Laurent, Alhassan Cassé, Christian L. Vestergaard, and Jean-Baptiste Masson

VIII. Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence Mattes Mollenhauer, Stefan Klus, Christof Schutte, Peter Koltai

IX. Efficiently Implementing the Maximum Likelihood Estimator for Hurst Exponent Yen-Ching Chang

X. Determination of Pipeline Leaks Based on the Analysis the Hurst Exponent of Acoustic Signals Department of Industrial Heat Power and Heat Supply Systems, Kazan State Power Engineering University, Kazan, Russia

XI. Determination of the Hurst exponent by use of wavelet transforms Ingve Simonsen, Alex Hansen, and Olav Magnar Nes, American Physical Society

XII. Long-Range Dependence in Financial Markets: a Moving Average Cluster Entropy Approach Pietro Murialdo, Linda Ponta, Anna Carbone

XII. An Introduction to Long-memory time series models and Fractional Differencing Granger, Joyeux

XIV. Dynamic Hurst Exponent in Time Series Carlos Arturo Soto Campos, Leopoldo Sánchez Cantú, Zeus Hernández Veleros

XV. Markov Chains and Metastability FU Berlin

XVI. Fractional Brownian motion, the Matérn process, and stochastic modeling of turbulent dispersion Jonathan M. Lilly, Adam M. Sykulski, Jeffrey J. Early, Sofia C. Olhede

XVII. Fractional Brownian Motion and Rao Geodesic Distance for Bone X-Ray Image Characterization Mohammed El Hassouni, Abdessamad Tafraouti, Hechmi Toumi, Eric Lespessailles, Rachid Jennane

XVIII. Fractional brownian motion and fractal analysis of brain mri images: A review Gokilavani Chinnasamy, Vanitha S

XIX. A Multifractal Model of Asset Returns Benoit Mandelbrot, Adlai Fisher, Laurent Calvet

Business Cycles

I. The Recession and Recovery of 1973-1976 NBER

II. The Anatomy of Double-digit Inflation in the 1970s NBER

Mathematical Finance, Econometrics

I. Advances in Financial Machine Learning Marcos Lopez de Prado

II. Tactical Investment Algorithms Marcos Lopez de Prado

III. Statistical Arbitrage in the U.S. Equities Market Marco Avellaneda & Jeong-Hyun Lee

IV. Principal Eigenportfolios for U.S Equities Marco Avallaneda, Brian Healy, Andrew Papanicolaou, George Papanicolaou

V. Managing Risks in a Risk-On/Risk-Off Environment Marcos Lopez de Prado, Lawrence Berkeley National Laboratory

VI. Market Regime Detection via Realized Covariances Andrea Bucci, Vito Ciciretti, Department of Economics, Universita degli Studi ”G. d’Annunzio” Chieti-Pescara, Independent Researcher

VII. Can Factor Investing Become Scientific? Marcos Lopez de Prado

VIII. Rational Expectations Econometric Analysis Of Changes in Regime James D. Hamilton

IX. Macroeconomic Regimes and Regime Shifts James D. Hamilton

X. A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle James D. Hamilton

XI. The Volume Clock Marcos Lopez de Prado, David Easley, Maureen O'Hara

XII. Pseudo-factors and Factor Investing Marcos Lopez de Prado

XIII. Regime Switching with Time-Varying Transition Probabilities Francis X. Diebold, Joon-Haeng Lee, Gretchen C. Weinbach

XIV. The Inverted Parabola World of Classical Quantitative Finance: Non-Equilibrium and Non-Perturbative Finance Perspective Igor Halperin

XV. On The Distribution of Stock Price Differences Mandelbrot, Taylor

XVI. Volatility is mostly Path-Dependent Julien Guyon, Jordan Lekeufack

XVII. Principal Component Analysis for Nonstationary Series NBER, James D. Hamilton, Jin Xi

XVIII. Prospect Theory: An Analysis of Decision under Risk Daniel Kahneman and Amos Tversky

XIX. Time-Varying Gaussian-Cauchy Mixture Models for Financial Risk Management Shuguang Zhang, Minjing Tao, Xu-Feng Niu, Fred Huffer, Department of Statistics, Florida State University

Ergodicity & Diffusion

I. The ergodicity problem in economics Ole Peters, Nature Physics

II. Time to move beyond average thinking Nature Physics

III. Ergodicity-breaking reveals time optimal decision making in humans David Meder, Finn Rabe, Tobias Morville, Kristoffer H. Madsen, Magnus T. Koudahl, Ray J. Dolan, Hartwig R. Siebner, Oliver J. Hulme

IV. Autocorrelation functions and ergodicity in diffusion with stochastic resetting Viktor Stojkoski, Trifce Sandev, Ljupco Kocarev, Arnab Pal

V. Self-fulfilling Prophecies, Quasi Non-Ergodicity & Wealth Inequality NBER Working Paper

VI. A misconception in ergodicity: Identify ergodic regime not ergodic process Mehmet Süzen

VII. Effective ergodicity in single-spin-flip dynamics Mehmet Süzen

VIII. Wealth Inequality and the Ergodic Hypothesis: Evidence from the United States Yonatan Berman, London Mathematical Laboratory, Ole Peters, London Mathematical Laboratory; Santa Fe Institute, Alexander Adamou, London Mathematical Laboratory

IX. Non-ergodic extended regime in random matrix ensembles: insights from eigenvalue spectra Scientific Reports, Nature, Wang‐Fang Xu, W. J. Rao

X. Spectral Analysis and L2 Ergodicity Carnegie Mellon University

Causal Inference

I. Causal Inference for Time series Analysis: Problems, Methods and Evaluation Raha Moraffah, Paras Sheth, Mansooreh Karami, Anchit Bhattacharya, Qianru Wang, Anique Tahir, Adrienne Raglin, Huan Liu

II. Dormant Independence Ilya Shpitser, Judea Pearl, Bloomberg School of Public Health

III. An Introduction to Causal Inference Judea Pearl

IV. Causality: Models, Reasoning, and Inference Judea Pearl

V. Measuring Cause-Effect with the Distribution of the Largest Eigenvalue Alejandro Rodriguez Dominguez, Irving Ramirez Carrillo, David Parraga Riquelme

VI. On the top eigenvalue of heavy-tailed random matrices Giulio Biroli, Jean-Philippe Bouchaud, Marc Potters

VII. Variable-lag Granger Causality for Time Series Analysis Chainarong Amornbunchornvej, Elena Zheleva, Tanya Y. Berger-Wolf

VIII. Post-COVID Inflation & the Monetary Policy Dilemma: An Agent-Based Scenario Analysis

IX. Causal Effects of Monetary Shocks Joshua D. Angrist, Guido M. Kuersteiner

X. Expectations, lags, and the transmission of monetary policy Catherine L. Mann

Sleep, Brain, Health

I. Wavelet Based Performance Analysis of SVM and RBF Kernel for Classifying Stress Conditions of Sleep EEG Prabhat Kumar Upadhyay, Chetna Nagpal

II. Discovery of key whole-brain transitions and dynamics during human wakefulness and non-REM sleep Nature Communications

III. Dynamics of sleep: Exploring critical transitions and early warning signals Susanne M.M. de Mooij a, Tessa F. Blanken b, Raoul P.P.P. Grasman a, Jennifer R. Ramautar b, Eus J.W. Van Someren b c d, Han L.J. van der Maas a

IV. Automatic Segmentation of Sleep Spindles: A Variational Switching State-Space Approach IEEE

V. Critical slowing down as early warning for the onset and termination of depression PNAS

VI. Dynamic Treatment Regimes Bibhas Chakraborty, Susan A. Murphy

Microbiome

I. Two dynamic regimes in the human gut microbiome Sean M Gibbons, Sean M Kearney, Chris S Smillie, Eric J Alm

II. Diversity, stability and resilience of the human gut microbiota Catherine A. Lozupone, Jesse I. Stombaugh, Jeffrey I. Gordon, Janet K. Jansson, Rob Knight

III. The gut microbiome as a modulator of healthy ageing Nature

IV. The gastrointestinal microbiome in dairy cattle is constrained by the deterministic driver of the region and the modified effect of diet Microbiome Journal

V. Timescales of gut microbiome dynamics Brandon H. Schlomann, Raghuveer Parthasarathy

VI. Ecological dynamics of the gut microbiome in response to dietary fiber The ISME Journal

VII. Resource Competition and Host Feedbacks Underlie Regime Shifts in Gut Microbiota University of Chicago Press, John Guittar, Thomas Koffel, Ashley Shade, Christopher A. Klausmeier, Elena Litchman

VIII. Microbiota in health and diseases Signal Transduction and Targeted Therapy, Kaijian Hou, Zhuo-Xun Wu, Xuan-Yu Chen, Jing-Quan Wang, Dongya Zhang, Chuanxing Xiao, Dan Zhu, Jagadish B. Koya, Liuya Wei, Jilin Li & Zhe-Sheng Chen

IX. Gut microbiome diversity is associated with sleep physiology in humans National Library of Medicine

Chaos & Statistical Mechanics

I. Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach Jaideep Pathak, Brian Hunt, Michelle Girvan, Zhixin Lu, and Edward Ott

II. The Dripping Faucet As a Model Chaotic System Robert Shaw

III. Elementary Principles of Statistical Mechanics JW Gibbs

IV. Hamiltonian Systems and Transformation in Hilbert Space B. O. Koopman

V. Regimes in Simple Systems Edward Norton Lorenz

VI. Designing Chaotic Models Edward Norton Lorenz

Path Integrals

I. Path Integrals in Quantum Physics R. Rosenfelder, ETH Zurich

II. Path integrals, particular kinds, and strange things Karl Friston, Lancelot Da Costa, Dalton A.R. Sakthivadivel, Conor Heins, Grigorios A. Pavliotis, Maxwell Ramstead, Thomas Parr

III. Covariant path integrals for quantum fields back-reacting on classical space-time Jonathan Oppenheim, Zachary Weller-Davies

IV. Path integrals and stochastic calculus Thibaut Arnoulx de Pireya, Leticia F. Cugliandolob, Vivien Lecomted, Frederic van Wijlande

Energy Usage, Climate & ML

I. Perseus: Removing Energy Bloat from Large Model Training Jae-Won Chung, Yile Gu, Insu Jang, Luoxi Meng, Nikhil Bansal, Mosharaf Chowdhury

II. Climate change from Large Language Models Hongyin Zhu, Prayag Tiwari, IEEE

III. Estimating the Carbon Footprint of Bloom, A 176B Parameter Language Model Alexandra Sasha Luccioni, Sylvain Viguier, Anne-Laure Ligozat

Ideology, Normative beliefs & Morality in LLMs

I. Evaluating the Moral Beliefs Encoded in LLMs Nino Scherrer, Claudia Shi, Amir Feder, David M. Blei

II. LLMs Grasp Morality in Concept Mark Pock, Andre Ye, Jared Moore

III. Exploring the psychology of GPT-4's Moral and Legal Reasoning Guilherme F. C. F. Almeida, José Luiz Nunes, Neele Engelmann, Alex Wiegmann, Marcelo de Araújo

IV. Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity Gabriel Simmons III

V. Who Are All The Stochastic Parrots Imitating? They Should Tell Us! Sagi Shaier, Lawrence E. Hunter, Katharina von der Wense