Espartaco

Marxist Philosophy of Science

“Is that to say we are against Free Trade? No, we are for Free Trade, because by Free Trade all economical laws, with their most astounding contradictions, will act upon a larger scale, upon the territory of the whole earth; and because from the uniting of all these contradictions in a single group, where they will stand face to face, will result the struggle which will itself eventuate in the emancipation of the proletariat.”

Karl Heinrich Marx · Marx-Engels Collected Works, Vol. VI, p. 290

EnglishEspañol

Tag: PSIS-LOO

  • EconCausal: An R Package That Takes Causal Inference in Time Series Seriously

    EconCausal: An R Package That Takes Causal Inference in Time Series Seriously

    You can also find this library at CRAN and download it directly from R and RStudio.

    LISTEN TO THIS POST AS A PODCAST:

    You found a relationship in your data. It’s statistically significant. The coefficients are clean. But here’s the question that keeps rigorous researchers up at night: will this relationship still hold six months from now?

    That question — whether a relationship between economic variables is real and durable, not just a fluke of the particular sample you happened to look at — is exactly what the EconCausal R package was built to answer.

    What Problem Does EconCausal Solve?

    Causal inference in time series data is notoriously tricky. Standard regression can tell you that two variables move together. It cannot tell you why, or whether that co-movement will persist when the economic regime shifts — which it inevitably will.

    EconCausal addresses this challenge by combining three well-established econometric methodologies into a single, reproducible pipeline. What makes it distinctive is not any single technique — each component has solid academic pedigree — but rather the orchestration: a standardized protocol that forces every candidate relationship through rigorous pre-testing, proper temporal validation, and explicit decision rules before declaring a finding.

    Think of it as a quality-control assembly line for causal claims in time series.

    The Three Approaches

    EconCausal implements three methodological frameworks, each capturing a different aspect of what “causality” can mean in temporal data. Using all three gives you complementary perspectives rather than relying on a single method’s assumptions.

    1. Error Correction Models with MARS (ECM-MARS)

    The idea: Many economic variables share a long-run equilibrium relationship but deviate from it in the short run. An Error Correction Model (ECM) captures both dimensions — the long-run cointegrating relationship and the short-run adjustment dynamics.

    EconCausal enhances the classical ECM by replacing its usual linear regression engine with Multivariate Adaptive Regression Splines (MARS), a flexible non-parametric method developed by Jerome Friedman in the early 1990s. This matters because real-world adjustment mechanisms are often nonlinear: variables may correct slowly when deviations are small but snap back aggressively when deviations cross certain thresholds. MARS can detect these threshold effects automatically without you having to specify them in advance.

    What the protocol does in practice:

    • Tests whether the variables are integrated of order one (I(1)) — a prerequisite for cointegration analysis
    • Runs both the Engle-Granger and Johansen cointegration tests, applying an “either” rule: if either test finds evidence of cointegration, the analysis proceeds
    • Controls for serial correlation and heteroskedasticity using HAC-consistent standard errors
    • Fits the ECM with MARS as the regression engine
    • Validates the relationship using rolling-origin cross-validation
    • Evaluates the model using a dual criterion (more on this below)

    Why two cointegration tests? Because Engle-Granger and Johansen have complementary strengths. Engle-Granger is intuitive and straightforward but can miss cointegrating relationships in multivariate settings. Johansen’s approach is more powerful in higher dimensions but relies on stricter distributional assumptions. Requiring only one of them to flag significance is a pragmatic compromise that reduces false negatives without being reckless about false positives.

    2. Bayesian Structural Time Series (BSTS)

    The idea: Decompose a time series into interpretable structural components — trend, seasonality, regression effects — and do so within a fully Bayesian framework that quantifies uncertainty at every level.

    BSTS, originally popularized by Google’s CausalImpact package and the work of Scott and Varian (2014), models the data-generating process as a state-space system. The key innovation in EconCausal’s implementation is the use of spike-and-slab priors for automatic variable selection. This is a Bayesian regularization technique that effectively asks: “Which candidate predictors actually belong in this model?” — and it does so probabilistically, rather than relying on stepwise procedures or arbitrary p-value cutoffs.

    What makes it useful for causal inference:

    • The structural components absorb confounding patterns (trends, seasonality) that might otherwise masquerade as causal relationships
    • Spike-and-slab priors guard against overfitting by aggressively shrinking irrelevant predictors toward zero
    • The Bayesian framework produces probabilistic forecasts — not just point predictions, but full posterior distributions — which naturally give you calibrated prediction intervals
    • It handles the “what would have happened without the intervention” question, which is central to impact evaluation

    3. Bayesian GLM with AR(1) Errors (BGLM-AR1)

    The idea: Fit a Bayesian generalized linear model but explicitly account for temporal dependence in the residuals through a first-order autoregressive error structure.

    This might sound like a niche technical detail, but it addresses one of the most common mistakes in applied econometrics: ignoring autocorrelation in regression residuals. When residuals are correlated over time, standard errors are wrong, confidence intervals are too narrow, and you end up being more confident in your results than you should be.

    By modeling the residual autocorrelation directly with an AR(1) structure and using Hamiltonian Monte Carlo (HMC) sampling via Stan, the BGLM-AR1 approach produces valid inference even when the data have strong temporal dependence. The priors are weakly informative and calibrated for standardized variables, which means they provide regularization without dominating the data.

    The dual evaluation criterion: This is where EconCausal’s protocol really distinguishes itself. A relationship is only accepted if it improves both:

    1. Predictive density (measured by Expected Log Predictive Density, or ELPD, estimated via Pareto-Smoothed Importance Sampling Leave-One-Out cross-validation — PSIS-LOO)
    2. Point forecast accuracy (measured by RMSE)

    Requiring improvement on both dimensions simultaneously is stricter than most practitioners apply. A model might improve point forecasts while worsening the calibration of its uncertainty estimates, or vice versa. EconCausal demands both.

    What Makes EconCausal Different: The Protocol

    You might be thinking: “Most of these individual techniques already exist in other R packages — brms and Stan for Bayesian modeling, the ecm package for error correction models, bsts and CausalImpact for structural time series.” And you’d be right.

    The novelty of EconCausal is not in inventing new statistical methods. It lies in integrating them into a single, standardized protocol with explicit decision rules, thresholds, and validation procedures that run automatically.

    Here’s what that means concretely:

    Temporal Stability Validation

    Rather than evaluating a model on a single train-test split, EconCausal uses Leave-Future-Out (LFO) cross-validation with configurable windows and horizons. The system repeatedly:

    1. Trains on data up to a cutoff point
    2. Forecasts into the future
    3. Evaluates forecast accuracy
    4. Slides the cutoff forward
    5. Repeats

    A relationship is only declared “stable” if it performs consistently across multiple temporal folds — not just one lucky window. The package reports a support metric: the proportion of temporal folds where the relationship holds up.

    Dual Decision Rules with Explicit Thresholds

    Every candidate relationship must clear both the Bayesian bar (ELPD improvement) and the frequentist bar (RMSE reduction). These are not vague guidelines — the protocol applies explicit numerical thresholds and includes tie-breaking rules for ambiguous cases.

    A Single Reproducible Pipeline

    From data loading through pre-testing, model fitting, temporal validation, and final go/no-go decisions — everything runs in one pipeline. This eliminates the researcher degrees of freedom that plague so many applied studies, where hundreds of analytical choices are made invisibly along the way.

    An independent novelty assessment included in the package’s documentation compared this protocol against both the academic literature and existing software frameworks. The finding: while all the building blocks are state-of-the-art and well-documented in the literature, no existing academic paper or software package was found that implements the same end-to-end protocol with the same combination of pre-tests, dual decision rules, support thresholds, and temporal validation.

    ApproachComponents = State of the Art?Identical Protocol in Literature?Identical Protocol in Existing Software?
    BGLM-AR(1)YesNoNo
    ECM-MARSYesNoNo (though ecm + earth approximates the model-fitting step)
    BSTSYesNoNo

    Who Should Use EconCausal?

    The package was originally designed for economic research — specifically, examining production-circulation relationships — but its applicability extends to any domain where you need to answer questions like:

    • Does X cause Y, or is it the other way around? The temporal structure of the methods naturally distinguishes between “X leads Y” and “Y leads X.”
    • Is this relationship robust across time? The rolling-origin validation directly tests temporal stability.
    • Are there nonlinear dynamics at play? The MARS component in ECM-MARS can detect threshold effects and nonlinear adjustment speeds.
    • How certain can we be? The Bayesian components (BSTS and BGLM-AR1) provide full uncertainty quantification, not just point estimates.

    This makes it useful for:

    • Academic researchers conducting empirical macroeconomics, financial economics, or applied econometrics
    • Policy analysts evaluating whether proposed policy relationships will hold across different economic conditions
    • Data scientists in finance who need to move beyond correlation to directionality and stability
    • Graduate students learning causal inference methods who want a pedagogical tool that enforces methodological discipline

    Getting Started

    Installation is straightforward:

    # Install from GitHub
    remotes::install_github("IsadoreNabi/EconCausal")
    # On Windows, if vignette building fails:
    remotes::install_github("IsadoreNabi/EconCausal", build_vignettes = FALSE)

    The package depends on a modern Bayesian computing stack — cmdstanr for Hamiltonian Monte Carlo sampling, loo for PSIS-LOO computation, and bsts for structural time series components — so make sure your R environment is up to date.

    A Note on What EconCausal Is Not

    It’s worth being explicit about the boundaries:

    • It is not a magic wand. If your data are fundamentally unsuitable for causal inference (too few observations, structural breaks that dominate the signal, measurement error in the variables), EconCausal will not rescue you. Its value lies in applying rigorous methods correctly and transparently — not in overcoming the limitations of your data.
    • It does not replace domain knowledge. Statistical evidence for a stable, directional relationship is necessary but not sufficient for a causal claim. You still need a theoretical mechanism, an understanding of potential confounders, and judgment about the plausibility of the identifying assumptions.
    • It does not invent new mathematics. Its contribution is the protocol — the disciplined, reproducible integration of established techniques into a pipeline with explicit, auditable decision rules.

    The Bottom Line

    EconCausal occupies a valuable niche in the R ecosystem. While individual components of its methodology are available elsewhere — Bayesian GLMs in brms, BSTS in the bsts package, ECM in the ecm package — no other tool packages them into a single protocol that systematically tests temporal stability, applies dual evaluation criteria, and produces reproducible, auditable results.

    If your work involves making causal claims from time series data — and you care about whether those claims will survive contact with the future — EconCausal deserves a serious look.

    Repository: github.com/IsadoreNabi/EconCausal

    References for the underlying methodologies: Engle & Granger (1987), Johansen (1991), Friedman (1991), Scott & Varian (2014), Vehtari et al. (2017), Hyndman & Athanasopoulos (2021). See the package documentation for the complete bibliography.

  • When the Whole Is All You See: A Bayesian Way to Recover the Parts with BayesianDisaggregation

    When the Whole Is All You See: A Bayesian Way to Recover the Parts with BayesianDisaggregation

    You can also find this library at CRAN and download it directly from R and RStudio.

    LISTEN TO THIS POST AS A PODCAST:

    You have a national Consumer Price Index. It is a single number per year. What you actually want is the price index for each sector of the economy — manufacturing, services, agriculture, construction — because those individual paths are what your model, your policy analysis, or your investment thesis really needs. You know how the sectors combine into the whole: the weights are public, or at least knowable. What you do not know is the sectoral numbers themselves. You only ever see their weighted sum.

    This is the disaggregation problem. It sounds like bookkeeping. It is actually a quietly profound statistical question, and the R package BayesianDisaggregation — built by José Mauricio Gómez Julián — tackles it with a degree of intellectual honesty that is rare in software. This post is a deep, plain-language tour of what the package does, why it works the way it does, and what it teaches about building statistical tools that tell the truth.

    If you want to follow along with the code, the project’s wiki has installation instructions, function references, and worked examples. This post deliberately stays code-free so the ideas come through.

    The problem, precisely

    Let’s make it concrete. You observe an aggregate index — call it the CPI — over T years. You also have a matrix of weights W: for each year and each of K sectors, W tells you that sector’s share of the total. The weights sum to one within each year. The aggregate is, up to measurement noise, the weighted sum of the latent sectoral indices:

    CPI at year t ≈ the weighted sum of the K sectoral indices, using that year’s weights.

    The goal: recover the K sectoral indices — call them φ — from the single aggregate series and the known weights.

    Here is the catch, and it is the entire heart of the matter. At every single year, the aggregate pins down one linear combination of the K sectors. The remaining K−1 directions are completely unconstrained by the data. With four sectors, you have one equation and four unknowns per year. The system is, in a precise mathematical sense, under-determined.

    This is not a numerical annoyance you can engineer away. It is structural. Any method that hands you a single, sharp sectoral path from an aggregate alone is, whether it admits it or not, smuggling in assumptions to fill the gap — and most methods do not tell you how much of the answer is data and how much is assumption.

    The question is not can you disaggregate. You always can. The question is: can you do it honestly, showing your work, and carrying the right amount of uncertainty forward?

    The wrong way: a cautionary tale

    The first version of the package — the 0.1.x line — advertised “MCMC-free Bayesian disaggregation.” The pitch was appealing: no slow Markov chain Monte Carlo sampling, just clean closed-form math. The implementation used a family of deterministic update rules — weighted, multiplicative, Dirichlet, adaptive — over the prior weight matrix.

    It did not work. Not in the sense of crashing. In the much more dangerous sense of appearing to work while silently not doing the one thing it claimed to do.

    The package’s own author, in a moment of radical honesty that is worth pausing on, audited the 0.1.x family and catalogued six foundational defects, labeled F1 through F6:

    • F1 — the aggregate never entered the computation. The “posterior” was derived entirely from the prior weight matrix. The actual observed CPI — the one piece of real evidence — was never used. The method was not conditioning on data; it was rearranging priors.
    • F2 — the Dirichlet concentration cancelled on renormalization. A parameter that was supposed to control how concentrated the sectoral estimates were simply vanished in the algebra when weights were normalized to sum to one.
    • F3 — the temporal pattern cancelled too. A component meant to encode smoothness over time also disappeared in the renormalization.
    • F4 — the “efficiency” term was a fixed constant. It looked like a data-dependent quality score; it was actually invariant.
    • F5 — there were no recovery tests. No one had ever generated synthetic data with a known truth and checked whether the method got it back.
    • F6 — a correlation helper cheated. It computed both Pearson and Spearman correlation and reported whichever was larger, a form of silent data-snooping.

    The most damning of these is F1. A Bayesian method that does not condition on data is not a Bayesian method. It is a deterministic transformation dressed in Bayesian vocabulary. And the worst part is that it looked reasonable — it returned numbers, they moved in plausible directions, and nothing crashed.

    The author’s response is, I think, the most important thing about this whole project. Rather than patching the defects one by one — adding the CPI here, fixing the concentration there — the author recognized that the foundational problem (not using the data) cannot be fixed within a deterministic re-weighting framework. The fix is not a patch. The fix is a fundamentally different model: one in which the aggregate enters as genuine evidence.

    So the entire 0.1.x family was deleted. Every function — bayesian_disaggregate(), compute_L_from_P(), spread_likelihood(), the four update rules, the grid search, the save function, the cheating correlation helper — all of it, gone. In its place: two Bayesian engines that actually condition on the data. The package version jumped to 0.2.0, the DESCRIPTION was edited to remove any claim of novelty, and the documentation was rewritten to be explicit about what was removed and why.

    This is what intellectual honesty in software looks like. It is not common. We should notice it when it happens.

    The right idea: the aggregate as evidence

    The conceptual move is simple to state and deep in consequence. Instead of treating the relationship between the aggregate and the sectors as a renormalization identity — a bit of algebra you apply after the fact — treat it as an observation density. The aggregate CPI is data. It is evidence about the latent sectors. The model should condition on it.

    In the package’s canonical engine, this means: the latent sectoral indices φ evolve over time as a random walk with drift, and the observed CPI is generated from the weighted sum of the sectors with some observation noise. The aggregate is not a constraint imposed after estimation; it is the likelihood. The sectors come out the other side as a posterior distribution — not a single number, but a full cloud of plausible values with credible intervals.

    This is the difference between solving an equation and updating beliefs in light of evidence. The first gives you a point. The second gives you a distribution. And as we will see, the distribution is the whole point.

    Two engines, one trade-off

    The package offers two ways to do the Bayesian inference, and the choice between them is a clean, well-explained trade-off between richness and exactness.

    The state-space MCMC engine

    This is the canonical, full-featured model. The sectoral indices live in log space, which guarantees they stay positive — a natural constraint for price indices that a model in raw levels could violate. Each sector’s log-index follows a random walk with its own drift and its own innovation scale (the amount of jitter per period).

    Two layers of hierarchical structure make this more than K independent random walks:

    • Partial pooling on the drift. Each sector has its own drift, but the drifts are drawn from a common distribution. This means sectors share information about their average growth rate without being forced to be identical — the classic shrinkage trade-off.
    • Partial pooling on the innovation scale. Similarly, each sector’s volatility is drawn from a shared distribution. Sectors borrow strength in estimating how jittery they are.

    The initial cross-section — the starting levels of the sectors at the first period — is anchored at the aggregate level with an estimable dispersion parameter. This is a subtle but important point. In the old, broken 0.1.x family, the “concentration” parameter was supposed to control how spread out the sectors were, but it cancelled in the algebra and had no effect. In the new model, the dispersion is a genuine parameter that the data and priors can actually estimate. It does not cancel. It does real work.

    Finally, the observation: the CPI is modeled as coming from a Student-t distribution centered at the weighted sum of the sectors, with an estimable scale. The Student-t (rather than a Gaussian) makes the model robust to outliers in the aggregate — a heavy-tailed observation can be accommodated without wrecking the fit. If you prefer, you can switch to a plain Gaussian observation.

    Because this model is not conjugate — the log transform, the Student-t, and the hierarchical structure break the neat algebra that would allow a closed-form solution — it is fit by Hamiltonian Monte Carlo via Stan (using either the cmdstanr or rstan backend). HMC is the gold standard for this kind of model: it handles the correlated, high-dimensional parameter space efficiently and comes with reliable diagnostics. The package runs four chains by default, checks the R-hat convergence statistic and the number of divergent transitions, and returns posterior draws of every sectoral index at every period.

    The closed-form conjugate engine

    The second engine is the linear-Gaussian counterpart. The sectoral indices evolve as a random walk in levels (not logs), and the aggregate is observed with Gaussian noise. This model is conjugate, which means its exact posterior can be computed in closed form — no MCMC, no sampling, no convergence diagnostics. The tool is the Kalman filter combined with the Rauch-Tung-Striebel smoother: the filter passes forward through time, updating beliefs about the sectors given each new CPI observation, and the smoother passes backward, refining those beliefs using future information.

    If you want joint posterior draws — not just the smoothed means and variances, but actual correlated samples from the full posterior — the package uses the Durbin-Koopman simulation smoother, a elegant technique that produces draws with the correct cross-time and cross-sector covariance structure. These draws are not marginal approximations; they are genuine samples from the joint posterior.

    This engine is the “correct realization of the original MCMC-free posterior idea.” The 0.1.x family wanted a closed-form Bayesian answer; the problem was not that closed-form is un-Bayesian (conjugacy is perfectly Bayesian), but that the old method did not use the data. This engine uses the data — the aggregate enters as the observation equation — and it does so in exact, closed form.

    The trade-off is explicit and documented. The closed-form engine buys you speed and mathematical exactness. It costs you three things: positivity (levels can drift slightly negative, which logs prevent), robustness (Gaussian observations are sensitive to outliers, which the Student-t handles), and the cross-sector hierarchy (there is no partial pooling in the linear-Gaussian model). The MCMC engine buys you all three of those, at the cost of sampling.

    Both engines return the same thing: an array of posterior draws of the sectoral indices, with dimensions [periods × sectors × draws], plus summary tables of medians and 95% credible bands.

    The honesty at the center: weak identification

    Here is where the package earns its deepest respect. It would be tempting, after building a real Bayesian model that conditions on the data, to declare victory and hand users sharp sectoral estimates. The package does not do this. Instead, it is explicit about a fact that most disaggregation methods gloss over: the sectoral split is only weakly identified.

    Remember the under-determination: at each period, the aggregate pins down one linear combination of the K sectors. The remaining K−1 directions are governed by the prior — the cross-sectional dispersion and the temporal smoothness — not by the data. This means the posterior intervals for individual sectors are wide, and they are influenced by the prior. This is not a bug. It is not a limitation to be engineered away. It is the correct representation of what the data can and cannot tell you.

    The package’s own recovery tests — which generate synthetic data from the model’s own data-generating process, where the true sectoral paths are known — confirm this directly. The aggregate is recovered essentially perfectly: the correlation between the fitted aggregate and the true aggregate is above 0.95, often essentially 1.0. The aggregate is strongly identified. But the sectoral coverage — the fraction of times the true sectoral path falls within the 95% credible band — is around 0.84, with the bands being deliberately wide. The test asserts that coverage exceeds 0.70, a conservative threshold, because the package refuses to claim sectoral precision that the data cannot deliver.

    This is “rigour by layers”: assert tightly what is identified, assert conservatively what is not. It would be easy to tune the prior to produce narrower, more impressive-looking bands. The package deliberately does not.

    Why the full posterior matters: propagating uncertainty

    If the sectoral estimates are uncertain — wide bands, prior-influenced — then what good are they? This is where the design of the package reveals its purpose. The sectoral indices are not the final product. They are input to a downstream model. In the author’s research program, they feed a nested Ornstein-Uhlenbeck model of price gravitation. But the principle is general: any time a disaggregated estimate feeds a second-stage analysis, the uncertainty in the first stage should flow into the second.

    The package handles this by multiple imputation, following Rubin’s rules. Each posterior draw of the sectoral indices is treated as one imputation — one plausible version of the truth. The downstream model is fit once per imputation, and the results are combined. The effect is that the weak per-sector identification — the wide bands, the prior influence — is carried forward into the downstream uncertainty intervals rather than being discarded. You do not plug in a point estimate and pretend it is the truth. You plug in the whole cloud and let the cloud’s shape propagate.

    The package’s documentation is candid about a consequence: because disaggregation is under-determined, the random-walk smoother prior dilutes the reversion signal, biasing the downstream reversion speed toward slowness by a modest, quantified amount (roughly 13–26%). Crucially, the direction is conservative — the true gravitation is at least as fast as reported — and the fraction of missing information that is propagated is about 0.4. This is not hidden. It is measured, reported, and flagged as a property of honest under-determination.

    A sharp contrast: an ad-hoc method that simply added noise to a point estimate did not produce proper imputations and gave sub-nominal coverage when routed through Rubin’s rules. The coherent posterior from the conjugate engine — the one that actually conditions on the data — did. The mathematics of multiple imputation demands proper posterior draws; garbage in, garbage out.

    How it is validated

    The package’s validation strategy is worth studying because it embodies a philosophy: test the identified quantity tightly, test the unidentified quantity conservatively, and test the computation itself exactly.

    Three layers, kept deliberately separate:

    Smoke tests run always, on every check. They confirm that both engines compile, sample, and return the correct [periods × sectors × draws] array structure on synthetic data. They catch breakage.

    Recovery tests are gated behind an environment flag because they require actually compiling and sampling the Stan model, which is slow. They generate data from the model’s own data-generating process — the same random walk with drift, the same partial pooling, the same aggregate observation — so the true sectoral paths are known. Then they check: does the aggregate come back essentially perfectly? (Yes, correlation above 0.95.) Do the sectoral bands cover the truth at a reasonable rate? (Yes, above 0.70, honestly wide.) The recovery test is well-posed because the simulator uses the same process as the model. If the model cannot recover its own data, something is wrong with the sampler or the implementation. If it can, you have a meaningful baseline.

    Golden tests run always and are the most stringent. They use Stan’s generate_quantities function — which deterministically recomputes derived quantities from frozen parameter draws, with no random number generation involved — and demand a bit-for-bit match against a frozen reference output. This catches any change to the model’s computed quantities: if someone edits the Stan code and the log-likelihood values shift by even one bit, the test fails. The reference fixture is generated by the same code path, isolating the CSV serialization so the comparison is exact.

    This is not the “does it run?” school of testing. It is the “does it compute the right thing, and does it compute exactly the same thing tomorrow?” school.

    Where it sits among existing methods

    The package is careful — almost unusually careful — about situating itself relative to the existing literature. It makes no claim of being the first or only solution to this problem. The documentation uses the phrase “we did not find” rather than “we are the first,” and the DESCRIPTION file was explicitly edited to remove any “novel/original” claim.

    The adjacent traditions, and what each misses:

    Biproportional balancing (RAS, IPF) iteratively scales a matrix to match new margins. It is deterministic: no posterior, no credible intervals, no treatment of the aggregate as evidence. It is a useful accounting tool, not an inference method.

    Temporal disaggregation (Denton, Chow-Lin, Fernández) distributes a low-frequency aggregate to higher frequency using an indicator series. This is a temporal problem — splitting annual into quarterly — not a cross-sectional one. It assumes you already have the sectoral decomposition and just need finer time resolution.

    Forecast reconciliation (MinT and related methods) projects inconsistent hierarchical forecasts onto a coherent subspace. It is forecast-centric and linear-algebraic: it corrects forecasts that do not add up, rather than recovering latent components from an aggregate by Bayesian updating.

    Compositional or Dirichlet state-space models evolve simplex weights over time. They model how shares move, not how the components themselves are recovered conditioned on their weighted sum.

    Each tradition addresses a real problem. None, as far as the package’s author could find, does exactly this: recover latent cross-sectional components from a single aggregate by conditioning on it as a genuine observation density and returning a posterior that can be propagated downstream. The claim is narrow and checkable, not sweeping.

    The data pipeline

    The package is not just a model; it is a usable tool. It includes hardened readers for the real inputs. A CPI reader pattern-matches on column headers (in English or Spanish — it recognizes “date,” “fecha,” “year,” “año” for the time column and “cpi,” “indice,” “price” for the value column), parses localized number formats (European-style decimals and thousands separators), collapses duplicate years by averaging, and returns a clean, sorted data frame. A weights reader loads a sector-by-year table, normalizes weights to the simplex within each year, and handles missing entries gracefully.

    An alignment function intersects the years covered by the CPI and the weights, ensuring both cover the same periods before either engine runs. A convenience wrapper reads both files and runs the disaggregation in one call. And a simulator generates synthetic data from the model’s own data-generating process — the same random walk with drift, the same partial pooling, the same aggregate observation — so that recovery tests, examples, and exploratory analysis are always well-posed.

    One data note worth flagging, because it is a common error: the model works in index levels, not rates of change. Feeding a percent-change series (inflation rate) instead of a level series (the CPI itself) is a category error — the aggregate would not be on the same scale as the weighted sum of the sectors. The CPI must be a level series, re-indexed to the same base as whatever the sectors will be compared against.

    The bigger lesson

    You could read this package as a technical contribution: a Bayesian state-space model for disaggregation with two engines, honest uncertainty, and a propagation contract. That reading is correct but incomplete.

    The deeper lesson is about how to build statistical software that tells the truth. The 0.1.x family did not fail by crashing. It failed by producing plausible-looking output that did not depend on the data. That is the most dangerous failure mode in statistics, because there is no error message. The numbers look reasonable. The plots look smooth. Nothing warns you that the entire computation is a rearrangement of priors.

    The author caught it — caught it in their own work, which is harder than catching it in someone else’s — by doing the unglamorous thing: generating data with a known truth and checking whether the method recovered it. When it did not, they did not patch. They deleted and rebuilt. And then they documented the deletion, in public, with the defects labeled and explained, so that anyone reading the history would understand not just what changed but why.

    The resulting package has a quality that is hard to name but easy to feel when you read the source: every design choice has a reason, every reason is documented, and the documentation is honest about what the method can and cannot do. The aggregate is strongly identified; the sectors are weakly identified; the uncertainty is wide and prior-influenced; and all of that is surfaced, not hidden, because the whole point is to carry that uncertainty forward rather than fake it away.

    In a field where it is tempting to claim sharp results from sparse data, this is a quiet act of integrity. The package does not solve the under-determination. Nothing can. It does something better: it honors it, by returning the wide, honest, propagatable posterior that the data actually supports.

    The package, its source code, installation instructions, and full function reference are on GitHub, with extended documentation in the wiki. It is MIT-licensed and written in R, with the MCMC engine powered by Stan.

  • When the Textbook Test Fails: How HTDV Brings Rigor to Dependent, Unbalanced Data

    When the Textbook Test Fails: How HTDV Brings Rigor to Dependent, Unbalanced Data

    You can also find this library at CRAN and download it directly from R and RStudio.

    LISTEN TO THIS POST AS A PODCAST:

    The problem hiding in your data

    Picture a straightforward question: Is the average inflation rate in the United States meaningfully different from zero? You have monthly data going back decades. A classical t-test would seem like the natural tool — and it would be quietly, systematically wrong.

    The reason is that inflation figures do not bounce around independently. January’s number carries information about February’s. This autocorrelation corrupts the standard error that the t-test relies on, inflating the false-positive rate well beyond the nominal 5% you think you are signing up for. The same problem afflicts yield spreads, stock returns, sectoral profitability, regional employment — virtually any real-world time series you might want to compare.

    Now make it harder. The two groups you are comparing have different sample sizes — one sector has twenty years of data, another only five. The data may have heavy tails you cannot rule out. And your sample is finite, which means the asymptotic guarantees printed in your econometrics textbook are promises that may not have been kept yet.

    This is the terrain HTDV was built for. Short for Hypothesis Testing for Dependent Variables with Unbalanced Data, HTDV is an R package that answers a deceptively simple question — do these dependent, possibly unequally-sized samples come from the same population? — under the worst combination of conditions an applied statistician is likely to encounter.

    The central idea: triangulation, not trust

    The most common approach to dependent data is to reach for a single robust method — a heteroskedasticity-and-autocorrelation-consistent (HAC) standard error, say, or a block bootstrap — and hope it is calibrated. HTDV takes a structurally different stance: run three independent inferential methods in parallel and expose the disagreement between them as a signal.

    The three layers are:

    1. A hierarchical Bayesian fit via Hamiltonian Monte Carlo (HMC), implemented in Stan. This layer builds a full probability model of the data-generating process, places weakly informative priors on the dependence parameters, and produces a posterior distribution for the quantity of interest.
    2. A fixed-bandwidth HAR Wald test in the frequentist tradition of Kiefer and Vogelsang (2005). Rather than letting the bandwidth grow with the sample in the usual way, it holds the bandwidth at a fixed fraction of the sample size. This produces a non-standard asymptotic distribution that is better calibrated in finite samples than the conventional chi-square approximation.
    3. A stationary block bootstrap (Politis and Romano, 1994) with automatic block-length selection (Patton, Politis, and White, 2009). This resamples the data in blocks long enough to preserve the dependence structure, then constructs confidence intervals from the resampled distribution.

    A fourth, distribution-free layer — adaptive conformal inference (Gibbs and Candès, 2021) — is available for online prediction settings where no parametric assumption is palatable.

    The logic is forensic. Where all three layers agree, your conclusion is robust. Where they disagree, the pattern of disagreement tells you something specific about your data. If the Bayesian interval is dramatically wider than the HAR or bootstrap interval, your series likely has strong temporal persistence, and the asymptotic critical values that HAR and bootstrap rely on are losing their reliability. That gap is not a bug — it is the most informative thing the framework can show you.

    Why a single method is not enough

    It is fair to ask: if the Bayesian layer is the most reliable, why not just use it and discard the others? The answer is that each layer has a regime where it is the appropriate tool, and the framework’s job is to make the regime visible.

    HAR inference is computationally cheap — sub-second on typical data — and well-calibrated when persistence is low to moderate and sample sizes are large enough for asymptotics to bite. The block bootstrap shares those advantages while making fewer distributional assumptions. The Bayesian layer is the most computationally expensive (each fit can take tens of seconds) but is the only one that maintains nominal calibration under strong persistence at finite sample sizes, because it models the dependence explicitly rather than relying on asymptotic corrections.

    The package ships with a pre-registered factorial Monte Carlo study — 1,024 cells crossing sample size, autocorrelation, tail heaviness, imbalance ratio, and location shift, with 500 replications per cell across all three inferential layers — and the results are unambiguous. The Bayesian layer holds nominal size (mean 0.056 against a target of 0.05) and nominal coverage (mean 0.944 against a target of 0.95) across the entire grid. HAR and bootstrap, by contrast, inflate dramatically in the worst corners: under strong persistence and small samples, HAR’s empirical rejection rate under the null reaches 0.60, and its coverage drops to 0.29.

    The narrowness of the HAR and bootstrap intervals in those corners is not a sign of precision. It is a sign of miscalibration — the intervals are confidently wrong.

    The theory that holds it together

    Running three different methods on the same data and comparing the answers is sound practice, but it raises a mathematical question: under what conditions are the three methods even addressing the same inferential target? A Bayesian posterior on a triangular-array likelihood and a HAR-Wald statistic on a mixingale process are, on their face, different objects.

    HTDV’s theoretical backbone is a metric equivalence theorem that resolves this concern. The framework identifies three structurally distinct ways real-world data can violate the independence assumption — each corresponding to a different law-of-large-numbers regime:

    • Triangular Arrays Convergence (TAC): information accumulates through hierarchical aggregation. Think of input-output tables disaggregated into ever-finer sectors, where each “row” of the array adds more observations.
    • Weighted Sums with Correlation (WSC): the observations share a cross-sectional covariance structure. Regional markets that move together, trade flows between linked economies.
    • Mixingale Process Convergence (MPC): temporal memory that decays smoothly over time. Forecast errors, model residuals, prediction intervals that gradually lose contact with the past.

    The theorem proves that, under α-mixing with polynomial decay rate γ > 1 and finite moment conditions, these three regimes induce strictly equivalent metrics on the space of hypothesis-testing problems. The equivalence comes with explicit, computable finite-sample constants — exposed by the function htdv_equivalence_constants() — that tell you the maximum slack when translating a conclusion from one regime to another. For typical parameter values (γ = 2, q = 6, n = 500), the conversion slack is about 18%, a margin that is usually irrelevant for a hypothesis-testing decision.

    This is what makes the three-layer architecture mathematically well-defined rather than merely pragmatic. Without the equivalence theorem, comparing a Bayesian result on a TAC dataset with a HAR result on a WSC dataset would be comparing apples and oranges. The theorem certifies that the metrics are coercible to one another with computable error.

    The dependence assumption, plainly

    The framework assumes that the data are α-mixing with polynomial decay — meaning that the statistical dependence between observations dies off as they get farther apart in time, and it does so fast enough (at a rate faster than 1/k) that the long-range correlations are summable. This is a mild condition satisfied by most stationary time series in econometrics and finance, including ARMA processes, GARCH models, and a broad class of Markov chains.

    It is not satisfied by long-memory processes (where dependence decays more slowly than 1/k) or by unit-root processes (where dependence does not decay at all). The framework is honest about these limitations: it will fit near-unit-root data, but the posterior will widen correspondingly — which is the correct answer. For explicit unit-root testing, the standard ADF or Phillips-Perron tools remain the right choice.

    The Bayesian engine

    The hierarchical Bayesian core fits Stan models via the No-U-Turn Sampler (NUTS), the state-of-the-art Hamiltonian Monte Carlo variant. The models are parameterized around an AR(1) structure — the mean θ, the autocorrelation φ, and the innovation scale σ — with hierarchical priors on the dependence nuisance parameters that are weakly informative enough to respect admissible ranges without overwhelming the data.

    Five likelihood backends are available, corresponding to the three convergence regimes plus two parametric likelihood families: the Whittle likelihood (which works in the frequency domain, comparing the observed periodogram to a theoretical spectral density) and the composite likelihood (which works in the time domain, combining conditional densities over short blocks). Both are well-established in the time-series literature; the choice between them depends on whether you have more confidence in your spectral model or your conditional density model.

    A distinctive feature is the Berger-robust envelope — a method for combining posteriors across multiple fitted models into a single, wider posterior that hedges against the worst-likelihood-specification scenario. If you are unsure whether the Whittle or composite likelihood better describes your data, the envelope gives you an inferential answer that is honest about that uncertainty rather than forcing an arbitrary choice.

    After sampling, every fit must pass a five-gate diagnostic check before its posterior is admissible: split-R̂ below 1.01, bulk and tail effective sample sizes above 400, zero post-warmup divergences, and energy Bayesian fraction of missing information (E-BFMI) above 0.3. These are the standard HMC convergence diagnostics from the Stan ecosystem, enforced as a gate rather than offered as a suggestion.

    The validation: visible in the data

    The most compelling aspect of HTDV is that it does not merely claim to be well-calibrated — it ships the evidence. Two validation datasets are bundled with the package.

    The first is the factorial simulation described above, with its 3,069-row summary table accessible as a package dataset. The headline finding — that the Bayesian layer is the only one maintaining nominal calibration across the full design — is not an assertion but a reproducible fact. The full study took 31 hours on a 16-core workstation; the scripts to regenerate it from scratch are shipped in the package repository.

    The second is a set of three external benchmarks against published references on public-source data:

    • Post-1984 US CPI inflation, compared against Stock and Watson (2007).
    • Shiller’s log-CAPE ratio, compared against Campbell and Shiller (1998).
    • The US–Canada 10-year yield differential, compared against the naive iid Welch baseline.

    All three layers reproduce all three references with agreement in every case. But the width of the agreement tells the real story. The interval widths scale monotonically with the persistence of the underlying series. At moderate persistence (φ ≈ 0.45, the inflation series), the Bayesian interval is actually narrower than HAR — 0.81 times its width. At high persistence (φ ≈ 0.97, the CAPE series), the Bayesian interval is 2.8 times wider. At near-unit-root persistence (φ ≈ 0.99, the yield differential), it is 15 times wider.

    This gradient is the framework’s central empirical finding. Both layers are technically asymptotically valid. Only the Bayesian layer accounts honestly for the finite-sample uncertainty inflation that occurs as φ approaches 1. The HAR and bootstrap intervals do not widen because they know more — they fail to widen because their asymptotic critical values have not yet caught up with the data.

    When to use it — and when not to

    HTDV is the right tool when your data are time-dependent or spatially dependent, when your samples are of unequal size, when you suspect heavy tails but cannot rule them out, and when you need an inferential answer (a test or an interval) rather than a prediction. It is particularly valuable when the stakes are high enough that you want your conclusion to survive methodological scrutiny — the framework ships its own validation evidence precisely so that a reviewer can interrogate the calibration claims rather than taking them on faith.

    It is the wrong tool when your data are genuinely independent with finite variance — classical methods are simpler, equivalent, and faster. It is also not designed for long-memory processes, explicit unit-root testing, structural breaks (unless you segment the sample first), or forecasting. The framework is built for hypothesis testing and parameter estimation under uncertainty, not for predictive accuracy.

    An open architecture

    The package exposes its full infrastructure: the simulation engine (htdv_simstudy()), the equivalence constants calculator, the diagnostic suite, the posterior-predictive checks on dependence statistics, and the decision tools — ROPE-based decisions (Kruschke, 2018), bridge-sampling Bayes factors, WAIC and leave-future-out cross-validation, and predictive stacking (Yao, Vehtari, Simpson, and Gelman, 2018). Every function is documented with its underlying reference, so the user can trace any method back to its source.

    The complete function reference, mathematical foundations, tutorial walkthroughs (oriented toward novices, applied statisticians, and mathematicians respectively), and the full validation narrative are in the HTDV Wiki on GitHub. The package is installed with a single command — remotes::install_github("IsadoreNabi/HTDV") — and requires rstan as its only hard dependency.

    The larger point

    HTDV embodies a methodological philosophy worth stating explicitly: when no single inferential method is universally valid in the finite-sample regime, the honest response is not to pick the best one and hide its limitations, but to run several and make the disagreement visible. The framework’s value is not that it always gives you a narrower interval or a more powerful test. Its value is that it shows you — concretely, quantitatively — where your inference is on solid ground and where it is standing on asymptotic ice.

    The validation evidence makes this concrete. In 98% of the simulation cells, the Bayesian layer alone passes the calibration benchmarks. The HAR and bootstrap layers pass in the regime where asymptotics have bitten — low persistence, large samples — and fail predictably outside it. The framework does not hide that failure. It turns it into a signal.

    That signal is the product.

    HTDV is released under the MIT license. The companion paper, full validation vignette, and reproducibility scripts are available at github.com/IsadoreNabi/HTDV. For the complete mathematical foundations, function reference, and tutorials, see the project wiki.

  • bayesianOU: Exploring Market Price Gravitation via Ornstein-Uhlenbeck Process

    bayesianOU: Exploring Market Price Gravitation via Ornstein-Uhlenbeck Process

    You can also find this library at CRAN and download it directly from R and RStudio.

    LISTEN TO THIS POST AS A PODCAST:

    When Market Prices Gravitate: A Bayesian Look at an Old Question in Economics

    An old question, asked again — properly

    There is a question in economics that is older than most of the academic disciplines that border it. Do market prices — the noisy, day-to-day, here-and-now prices at which goods actually change hands — tend to settle toward some underlying center of gravity? And if they do, how fast, how violently, and through what mechanism?

    Classical political economy, from Smith and Ricardo through Marx, thought they do. The idea was that behind the churning surface of market prices there sit “prices of production”: long-run, cost-anchored prices toward which actual prices are pulled, the way a spring pulls a weight back toward its rest position. In the Marxian version, there is one more layer underneath: those prices of production themselves are supposed to gravitate around “values,” the labour embodied in commodities. Whether any of this is true is an empirical question, and for a long time the empirical tools to answer it were not really up to the job.

    A small R package called bayesianOU, written by José Mauricio Gómez Julián and hosted on GitHub, takes a serious swing at that question. It is not the first attempt to test price gravitation statistically, but it is one of the most technically careful I have seen, and it is built in a way that is instructive far beyond the Marxian debate that motivates it. What follows is a walkthrough of what the package does, why it is interesting, and — just as importantly — where it honestly admits its own limits.

    The tool that makes it possible: the Ornstein-Uhlenbeck process

    Strip the economics away for a moment and the statistical core of the package is a workhorse object from physics: the Ornstein-Uhlenbeck (OU) process. Imagine a particle moving in a fluid, attached to a spring. Brownian motion jiggles it randomly; the spring pulls it back toward a fixed point. The further it drifts away, the harder the pull. The result is a wiggly series that never settles but always tends to settle — a mean-reverting random walk.

    The OU process is exactly the mathematical object you want when you suspect a variable is noisy but anchored. It has a “speed of reversion” (how hard the spring pulls) and an “equilibrium level” (where the spring’s rest point is). Estimate those, and you can say something quantitative about gravitation: not just “yes, prices come back,” but “they come back with a half-life of about nine years.”

    That number — the half-life — is the prize. It is the difference between “market prices eventually settle” (which could mean anything) and “market prices settle on a timescale comparable to a business cycle” (which is a falsifiable, interpretable claim).

    What the package actually builds

    The package fits, by Bayesian inference, a family of models built on the OU process but considerably richer than the textbook version. There are two first-class models, sharing one inference engine.

    The single-level model

    The first model asks: do market prices revert toward an equilibrium that is a function of the prices of production, and what does that reversion look like once we let it be nonlinear, volatile, heavy-tailed, and structurally heterogeneous across sectors?

    Each of those adjectives is doing real work, and each corresponds to a feature that simpler approaches handle poorly or not at all:

    • Nonlinear drift. A plain OU process pulls back with a force proportional to the deviation. The package allows a cubic correction, so the restoring force can strengthen super-linearly when prices are far from equilibrium. This matters: real markets may behave gently near the center and violently at the extremes, and a linear model cannot represent that.
    • Stochastic volatility. Financial data, and economic data generally, go through quiet stretches and turbulent ones. The package does not assume a single noise level; it lets the volatility itself wander over time, following its own mean-reverting process on the log-variance. This is the same idea that powers modern stochastic-volatility models in finance, and it is essential for not fooling yourself about the precision of your estimates.
    • Heavy tails. Economic shocks are not Gaussian. Crashes, booms, and policy shocks produce outliers that a normal distribution would call essentially impossible. The package uses Student-t innovations and estimates the degrees of freedom from the data, so the model can discover for itself just how fat-tailed the world is.
    • Hierarchical structure across sectors. An economy has dozens of sectors, and each one presumably has its own reversion speed, its own equilibrium, its own noise. Estimating each sector in isolation throws away the information that they are all part of the same economy. Estimating them all with one set of parameters pretends they are identical. The package takes the middle path — hierarchical, or “partial pooling,” priors — where each sector’s parameters are drawn from a shared distribution whose properties the model also estimates. Sectors borrow strength from one another without being forced into lockstep.
    • A time-varying coupling. This is the most economically loaded feature. The strength with which market prices track prices of production is allowed to depend on the aggregate profit rate (what the package calls TMG). When the general rate of profit is high, the pull of production prices on market prices may be one thing; when it is low, another. Whether that modulation exists, and in which direction, is a hypothesis the model can test rather than assume.

    All of this is estimated jointly, with full Bayesian uncertainty, using Stan’s Hamiltonian Monte Carlo sampler. You do not get a point estimate of the reversion speed; you get a posterior distribution, and from it a credible interval and a probability statement like “there is a 95% chance the half-life is between six and eighteen years.”

    The nested cascade

    The second model is the more ambitious one, and it is where the package earns its “nested” branding. Instead of market prices reverting to a fixed equilibrium, they revert to a latent production price — a hidden, unobserved series that itself evolves over time according to its own OU process, driven by the general profit rate. And, if you turn on the third level, that latent production price in turn gravitates around an observed “value” index built directly from labour-content accounting.

    So the full structure is a cascade: market price → latent production price → value. Each arrow is an OU reversion, each with its own speed, and the speeds are constrained so that the outer (market) layer reverts faster than the inner (production) layer — an economically natural separation of timescales, enforced softly so the data can push back.

    The reason this matters is that it converts a slogan — “prices of production gravitate around values” — into a literal statistical hierarchy that can be fit to data and compared against alternatives. The headline empirical result, from a fit to 37 US sectors over 1960–2020, is a value-coupling coefficient essentially equal to one, with the posterior probability of it being positive effectively equal to one. In plain terms: in standardized units, prices of production track labour values almost one-for-one. That is a found result, not an assumed one — the prior on the coupling was centred at zero, deliberately neutral.

    The inference engine, and why it is not a footnote

    It would be easy to glance at the model description, nod, and move on. But how these quantities are estimated is half of what makes the package serious, and it is worth a paragraph for readers who do not think about MCMC every day.

    Bayesian inference works by exploring the space of all parameter values consistent with both the data and the prior, and characterizing that space as a probability distribution. For models this complex — with latent volatility paths, hierarchical structure, and hundreds of parameters — you cannot do that with pencil and paper. You use a Markov chain Monte Carlo sampler, specifically Hamiltonian Monte Carlo, which borrows an idea from physics: give the parameter space a “potential energy” (the log-posterior) and a “kinetic energy” (a randomly chosen momentum), and let the system glide around the posterior like a ball rolling over a landscape.

    Stan’s NUTS sampler automates this about as well as it can be automated, and the package uses it with within-chain parallelism (via Stan’s reduce_sum) to handle the fact that the likelihood must be summed over many timepoints and sectors. The diagnostics — R-hat for chain agreement, effective sample size, divergence counts — are surfaced through a validate_ou_fit function, and the package is explicit that you should look at them before believing anything.

    Model comparison is done with PSIS-LOO, a clever technique that approximates leave-one-out cross-validation without refitting the model dozens of times, by reweighting the posterior draws using importance sampling. It is the modern standard, and the package is appropriately cautious about it: because the model has a latent volatility state at every observation, plain LOO is known to be optimistic, and the documentation says so plainly.

    The honesty that makes it credible

    Here is where the package surprised me, and here is why I think it deserves a wider audience than the Marxian-economics niche it lives in.

    A naïve reading of the results would be triumphant: the value coupling is one-to-one, the reversion exists, the half-life is about nine years. But the package’s own validation section does something rare. It runs the model against legitimate rivals on genuinely held-out data — a full decade, 2011 to 2020 — and reports, without spin, that a random walk beats the OU model at forecasting, that a no-gravitation restriction ties or beats it, and that the value term adds no detectable predictive density.

    That sounds like a refutation. The package argues, carefully, that it is nothing of the sort — and the argument is the most intellectually interesting thing here.

    The key move is to distinguish two different questions. One is structural: does a reversion mechanism exist, and how fast is it? The other is predictive: can you forecast next year’s price better than a naïve benchmark? These are related but not identical, and for a slow process they come apart in a specific, predictable way.

    If gravitation is real but slow — a half-life of nine years on a dataset whose test window is a decade — then over the forecast horizon the process looks, to first order, like a random walk. The reversion is there, but it is too weak to show up in a one-step or few-step prediction. The random walk, which assumes no reversion, will forecast almost as well, because over short horizons a barely-reverting process and a non-reverting one are nearly indistinguishable. So the random walk winning the forecasting horse race is not evidence against gravitation; it is evidence consistent with gravitation being slow.

    This is not special pleading. It is a logical point about what different functionals of a model can and cannot tell you. The structural parameters — estimated from the joint likelihood over the whole panel, borrowing strength across 37 sectors and 61 years — use far more information than any single-series forecast. They can pin down a central tendency that a univariate test cannot. And the package shows, through simulation-based calibration and adversarial negative controls, that the estimation pipeline does not manufacture gravitation when none is present: feed it a true random walk and it reports a half-life of about fifty years; feed it a null value-coupling and the posterior honestly covers zero.

    The low-kappa trap, and why it matters to everyone

    The package names a difficulty it calls the low-kappa trap, and it is worth understanding because it is a trap that catches far more than Marxian price theory.

    Kappa is the reversion speed. As kappa shrinks toward zero, the OU process approaches a pure random walk. The trouble is that there is no bright line separating “slow mean reversion” from “no mean reversion.” It is a continuum, and three distinct problems stack up exactly there:

    • Algebraically, reversion speed and discrete-time persistence are two sides of the same coin; kappa going to zero is the same as the autocorrelation going to one. There is no internal frontier.
    • Statistically, the power of a unit-root test — the standard tool for asking “is this a random walk?” — collapses exactly as the truth approaches the random walk boundary. With a finite sample and a half-life comparable to the sample length, the test simply cannot tell. This is a well-known result in econometrics, and it is why decades of “is the real exchange rate stationary?” papers argued past one another.
    • Numerically, if the reversion speed is parameterized to be strictly positive (as it must be, for the sampler to behave), then “the probability that kappa is greater than zero” is trivially one — it tells you nothing. The informative quantity is the half-life, and the probability that the half-life exceeds some sensible horizon.

    The package’s response to the trap is instructive. It does not pretend the trap is not there. It states all three layers explicitly, reports the slow tail honestly (one sector has a non-trivial posterior probability of a half-life beyond forty years), and argues that the joint hierarchical posterior — which pools information across the whole panel — is a more powerful discriminator than any univariate test. That is a defensible position, and it is stated with the caveat attached rather than buried in a footnote.

    This is the broader lesson. Anyone working with time series that might be slowly mean-reverting — interest rates, real exchange rates, commodity prices, climate variables, pollutant concentrations — runs into exactly this trap. The package’s framing of it, in three layers, is one of the clearest expositions I have read, and it would travel well into any of those domains.

    What I appreciate, and what I would watch for

    A few things stand out as genuinely good practice, and they are worth naming because they are rarer than they should be.

    The separation of economic and sampler convergence. The package is scrupulous about not confusing two senses of “convergence.” Economic convergence — does the price revert? — is a statement about kappa and the half-life. Sampler convergence — did the MCMC chains agree? — is a statement about R-hat and divergences. These share a word and nothing else, and conflating them is a classic source of muddled reasoning. The documentation keeps them lexically distinct throughout.

    Neutral priors on the load-bearing hypotheses. The prior on the profit-rate modulation is centred at zero. The prior on the value coupling is centred at zero. The package does not bake the answer into the question. When the posterior then moves clearly away from zero, that means something.

    Out-of-sample integrity by construction. A subtle and common error in time-series work is “leakage”: accidentally letting future information contaminate the training procedure, so that out-of-sample results are secretly in-sample. The package offers a fit_window switch that keeps the two designs genuinely separate, and it computes the common-factor loadings from the training window only. This is the kind of plumbing detail that separates trustworthy work from work that just looks trustworthy.

    The negative results are reported. Many packages, and most blog posts about them, would quietly omit the fact that a random walk out-forecasts the model. This one leads with it and then reasons about it. That is how a field accumulates reliable knowledge rather than just encouraging headlines.

    What should a careful reader watch for? The half-life estimate of about nine years is, by the package’s own account, probably conservatively slow — a controlled study of the disaggregation step suggests the true figure may be closer to seven or eight. The cubic nonlinearity is a minor refinement on this data (its coefficient sits near its prior). The Student-t degrees of freedom and the stochastic-volatility scale are weakly identified when both are present, a known tension the documentation flags but does not resolve. And the headline value-coupling result, while striking, is measured on standardized levels that share a cost-price component by construction; the package defends this with a “wedge” argument — subtracting the shared component and testing the residual — but a sceptical reader should follow that argument itself rather than take it on trust.

    None of these caveats undermine the project. They are the project. A statistical framework that cannot articulate its own soft spots is not a framework you should believe.

    Why it is worth your time

    You do not need to be a Marxian economist, or any kind of economist, to get something out of this package. If you work with time series that exhibit slow, noisy reversion toward a moving target — and a great deal of the physical and social world does — the modelling ideas here are directly portable: the nonlinear OU drift, the stochastic volatility, the hierarchical pooling across groups, the careful separation of structural estimation from forecasting, and the three-layer diagnosis of the low-reversion trap.

    And if you are interested in the classical question of whether prices gravitate toward values, this is about as good a statistical treatment as you will find: modern machinery, honest reporting, and a willingness to let the data argue back against the theory that motivated the exercise in the first place.

    The repository, the full mathematical specification, the validation blocks, and a frank discussion of every methodological decision live at github.com/IsadoreNabi/bayesianOU, with the wiki carrying the complete technical detail. Read the methodology notes before you quote a number; that is what they are there for.