Tag: datos de panel

Marx, Adam Smith, and the Law of Large Numbers
A new research uses probability theory — and sixty years of U.S. economic data — to test one of the most consequential (and most overlooked) assumptions in political economy.

The Assumption Hiding in Plain Sight

If you’ve ever read a Marxist analysis of how profits equalize across industries, you’ve probably encountered something called the average rate of profit. The idea is straightforward: competition between capitalists drives different rates of profit in different sectors toward a common, system-wide average. This is one of the pillars of Marx’s theory of value in Capital, Volume III.

But there’s a quieter assumption underneath this one — so quiet that most discussions never mention it explicitly. To arrive at a uniform profit rate, Marx first assumes a uniform rate of surplus value across all productive sectors. In plainer terms: he assumes that the degree to which workers are exploited — the ratio of unpaid labor to paid labor — is roughly the same everywhere, whether you work in steel manufacturing, food processing, or textiles.

Adam Smith proposed this idea before Marx. Smith argued that if one job were obviously more exploitative (in the sense of yielding far more unpaid surplus per dollar of wages paid), workers and capital would flow toward or away from it until the differences vanished. Marx adopted this observation and, as scholar Jonathan Cogliano notes, elevated it to “the status of a central economic law” within his framework.

Yet the assumption has been challenged from multiple directions — Marxist and non-Marxist alike. Is it actually justified? Or is it a convenient simplification that distorts our understanding of how capitalism works?

José Mauricio Gómez Julián, of the Universidad Latina de Costa Rica, decided to approach the question from an unexpected angle: probability theory. His paper, published in Ciencia Económica (2022), asks whether the mathematical law that would need to hold for this assumption to be valid actually does hold — and then checks the answer against six decades of real-world data from the United States.

The Mathematical Backbone: The Law of Large Numbers

If you’ve taken any statistics course, you’ve likely met the Law of Large Numbers (LLN). It tells us that as you observe more and more instances of something random — coin flips, dice rolls, stock returns — the average of those observations settles down toward the true expected value.

There are two versions:
- The Weak Law (WLLN): With enough observations, the sample average is probably close to the expected value.
- The Strong Law (SLLN): With enough observations, the sample average is almost certainly equal to the expected value — a much stronger guarantee.
Gómez Julián’s insight is this: if you think of each productive sector of the economy as a random variable representing that sector’s rate of surplus value, then the LLN tells you what happens to the average across sectors as the number of sectors grows large. In mathematical language:
- Strong Law: The probability that the average surplus-value rate across sectors equals the global expected value, in the limit, is exactly 1.
- Weak Law: The probability that the average deviates from the global expected value by more than any tiny amount shrinks to zero.
If either version holds, you get the result Marx needs: across a sufficiently large number of sectors, the rates of surplus value converge to a common value — uniformity, or at least a powerful tendency toward it.

The Catch: Independence and Identical Distributions

Here’s where things get interesting — and where the classical LLN hits a wall.

The textbook version of the LLN requires two conditions:
1. Independence: The random variables (sectoral surplus-value rates) must be statistically independent of each other.
2. Identical distribution: Each variable must follow the same probability distribution.
Neither condition holds for the real economy. And Gómez Julián is admirably upfront about this. Sectors are deeply intertwined — the steel industry depends on mining, manufacturing depends on steel, services depend on consumer spending powered by manufacturing wages. The idea that one sector’s surplus-value rate has no relationship to another’s is economically unrealistic. Furthermore, different industries have different cost structures, different labor intensities, and different technologies. There is no reason their surplus-value rates should follow the same statistical distribution.

So does this kill the argument? Not at all. In fact, it’s the most intellectually interesting part of the paper.

Non-Classical Varieties: When the Rules Relax

Over the past several decades, mathematicians and econometricians have developed non-classical versions of the LLN that weaken or entirely drop the independence and identical-distribution requirements. Gómez Julián surveys several of these:
- Li, Rao, and Wang (1995) showed the LLN holds for random variables arranged on a lattice structure under certain conditions — a structure that, as it happens, economic data naturally exhibits.
- Adler and Rosalsky (1987) proved the law for weighted sums of independent, identically distributed random variables belonging to a normalized sum, generalizing the classical case.
- Chen and Sung (2016) extended those results further: the variables no longer need to be identically distributed. They only need to be “stochastically dominated” by a single random variable, with certain weighting conditions.
- Sung (2011) showed that the strong law can hold even when variables are dependent on each other, provided their probability moments (roughly, their averages and variability) satisfy certain finiteness conditions.
The crucial point: these results collectively tell us that the LLN’s convergence conclusion can survive even when the classical assumptions are substantially violated — which is exactly the situation with sectoral surplus-value rates.

Gómez Julián argues that the economic dynamics described by Smith — workers and capital moving between sectors in response to unequal advantages — are precisely the kind of compensatory dependence mechanism that these non-classical versions accommodate. The variables aren’t independent, but their dependence is structured in a way that still drives convergence.

What the Data Actually Shows

The theoretical argument is compelling, but Gómez Julián doesn’t stop there. He turns to sixty years of U.S. data (1960–2020), sourced from the Bureau of Economic Analysis (BEA), to see what the empirical evidence says.

He calculates sectoral surplus-value rates using macroeconomic data on gross operating surplus (representing surplus labor time) and employee compensation (representing necessary labor time), following a standard operationalization of Marx’s categories. After carefully determining which sectors qualify as “productive” in the Marxist sense — a nontrivial task, since the service sector includes activities with very different relationships to surplus-value production — he arrives at 36 productive sectors.

Here’s what the statistical analysis found:

Finding 1: No Identical Distributions

A probability distribution fitting exercise (using the Bayesian Information Criterion) revealed that the 36 sectors’ surplus-value rates follow a patchwork of different distributions — Log-Normal, Cauchy, Uniform, Weibull, and Logistic — with none following a normal distribution. The identical-distribution requirement of the classical LLN is not met.

Finding 2: No Statistical Independence

A Pearson correlation analysis across all 630 possible sector pairs yielded a mean correlation of about 0.08 and a median of about 0.14. While these may look small, a deeper cut reveals that roughly 40% of sector pairs have correlations of 0.3 or above — a level that’s practically meaningful. The sectors are not independent. This makes intuitive sense: industries are connected through supply chains, labor markets, and shared macroeconomic conditions.

Finding 3: Differences Tend Toward Zero

This is the key finding. When Gómez Julián computed the differences between each sector’s surplus-value rate and the global average (both the mean and the median), he found that these differences exhibit a strong tendency toward reciprocal nullification — positive differences roughly cancel out negative ones. The sum of all differences relative to the global mean was essentially zero (on the order of 10⁻¹⁴). The mean of differences relative to the global median was 0.0012 — vanishingly small.

Distributional fitting on these differences revealed they follow a Cauchy distribution (when measured against the global mean) or a uniform distribution (against the global median), with the medians of these distributions sitting very close to zero.

In plain language: sectors deviate from the average in different directions, and those deviations largely cancel each other out.

Why This Matters

Gómez Julián’s paper makes two types of contributions that are worth distinguishing:

For Marxist political economy: If the uniformity assumption holds — even approximately, even as a tendency rather than an iron law — then a large body of research on the long-run behavior of the average rate of profit, both within countries and across the global economy, is on sounder footing than critics have suggested. Researchers studying the tendency of the rate of profit to fall (or not) can continue to work without needing to explicitly model sector-by-sector differences in exploitation rates, at least for aggregate, long-run analyses.

For probability theory and economics: The paper demonstrates a productive intersection between a specific question in political economy and the deep mathematics of convergence theorems. It shows that the non-classical LLN theorems aren’t just abstract curiosities — they have direct relevance to understanding real economic phenomena. The structured dependence between economic sectors isn’t a bug that invalidates the mathematical framework; it’s a feature that the right version of the framework already accounts for.

A Few Honest Caveats — And Why They No Longer Apply

The original 2022 paper was unusually transparent about its limitations, and that transparency is one of its strengths. Rather than forcing the data into inappropriate statistical procedures, it openly acknowledged where the available inferential tools broke down.

At the time, three important caveats remained.

First, formal hypothesis testing had to be abandoned.

The reason was purely statistical rather than economic. Classical inferential procedures—Student’s t tests, Wilcoxon tests, and most conventional non-parametric alternatives—are built on assumptions that the data simply did not satisfy. Sectoral surplus-value rates are neither independent nor identically distributed. They are linked through supply chains, technological change, capital mobility, and macroeconomic shocks. Even bootstrap procedures could not fully solve the problem because ordinary resampling may weaken dependence between resamples while leaving the internal dependence structure fundamentally unchanged. Consequently, the 2022 paper relied primarily on descriptive statistics together with probability-theoretic arguments instead of formal significance testing.

Second, the classification of productive sectors inevitably involved theoretical judgment.

Although the paper carefully justified the inclusion and exclusion of economic activities using Marxian categories and modern national accounting, reasonable scholars could still debate where certain services belong within the circuit of capital.

Third, the empirical evidence came exclusively from the United States.

The descriptive regularities were remarkably strong, but demonstrating that the same convergence mechanism operates under different institutional settings naturally remained an empirical question.

Those were genuine limitations in 2022.

Today, however, the first—and arguably the most important—of them has largely been overcome.

A much more comprehensive methodological paper (Gómez Julián, 2026; SSRN 5172185) develops an entirely new inferential framework specifically designed for exactly the type of data that made the original analysis difficult: dependent, heterogeneous, and unbalanced observations. Instead of trying to force classical statistical tests to work outside the assumptions under which they were derived, the newer paper constructs hypothesis testing from the ground up for this class of problems.

The key innovation is recognizing that the convergence of sectoral surplus-value rates is fundamentally a law-of-large-numbers problem under dependence, not an independent-samples problem. The framework therefore combines three complementary asymptotic structures—triangular arrays (TAC), correlation-weighted sums (WSC), and mixingale processes (MPC)—which respectively model hierarchical dependence, contemporaneous intersectoral dependence, and temporal dependence. Rather than treating these as competing approaches, the paper proves conditions under which they become metrically equivalent and therefore support the same inferential conclusions.

The inferential consequences are substantial.

Instead of abandoning significance testing because dependence invalidates classical procedures, the new framework explicitly extends the Neyman-Pearson paradigm to dependent observations, derives dependence-aware confidence regions, establishes rigorous Type I error control under strong-mixing assumptions, and integrates Bayesian and frequentist inference into a single coherent architecture. Robust procedures—including fixed-b heteroskedasticity-and-autocorrelation-robust inference, block bootstrap techniques that preserve dependence, adaptive conformal inference, composite and Whittle likelihoods, and hierarchical Bayesian estimation—serve as mutually reinforcing validation mechanisms rather than isolated alternatives.

In other words, what had been acknowledged as a methodological limitation in the 2022 paper became the central research question of the later work.

Rather than concluding that inference was impossible under dependence, the subsequent research asks a more fundamental question: what should hypothesis testing look like when dependence is the normal state of the data rather than an exception? The result is a unified inferential framework specifically intended for datasets that violate the assumptions of classical statistics—precisely the situation encountered with sectoral surplus-value rates.

The other caveats remain, although they are considerably less problematic than before. The classification of productive sectors continues to depend on theoretical interpretation, because that issue belongs to political economy rather than statistics. Likewise, expanding the empirical analysis to additional countries remains a desirable avenue for future research. Yet the principal statistical objection—that no valid inferential procedure existed for dependent sectoral data—has now been directly addressed through a purpose-built mathematical framework.

Looking back, the 2022 paper can therefore be read as identifying an important statistical obstacle, while the later work attempts to remove it. Together, the two papers form a coherent research program: first demonstrating that the convergence hypothesis is theoretically plausible and descriptively supported, and then developing the inferential machinery required to test that hypothesis rigorously without relying on unrealistic assumptions of independence or identical distributions.

Gómez Julián, J.M. (2022). Sobre la validez del supuesto de uniformidad en las tasas de plusvalía sectorial desde la teoría de las probabilidades. Ciencia Económica, 11(17). DOI: 10.22201/fe.24484962e.2022.11.17.2

Gómez Julián, J.M. (2026). Hypothesis Testing for Dependent Variables with Unbalanced Data: A Unified Framework: Theory, Robustness, and Software. SSRN Electronic Journal. DOI: 10.2139/ssrn.5172185.
Compártelo:
X
Facebook
Print
LinkedIn
Reddit
Tumblr
Pinterest
Telegram
WhatsApp
Email
Threads
Mastodon
Nextdoor
X
Bluesky
Like this:
Like Loading…
15 July, 2026
valueprhr: When Do Market Prices Reflect the Labor That Produced Them? A Modern R Toolkit for an Old Question
You can also find this library at CRAN and download it directly from R and RStudio.

LISTEN TO THIS POST AS A PODCAST:

An introduction to an R package that brings Bayesian inference, panel data econometrics, and rigorous validation to one of political economy’s most enduring empirical debates.

Why This Package Exists

Here is a question that has occupied economists for over two centuries: when you pay for something, does the price you pay bear any systematic relationship to the labor required to make it?

Adam Smith thought so. David Ricardo refined the idea. Karl Marx built an entire theory of exploitation on it. And since the mid-twentieth century, empirical researchers have been trying to measure the strength of this correspondence with real-world data.

The challenge has always been methodological. The datasets are panel data — prices observed across many economic sectors over many time periods — and they demand techniques that respect both the cross-sectional structure (different industries behave differently) and the temporal dimension (relationships can shift over time). A simple scatterplot of values against prices, however illustrative, will not settle the question.

valueprhr is an R package built to close this methodological gap. It provides a complete, reproducible pipeline: from raw price matrices to model estimation, from Bayesian inference to out-of-sample validation, from structural break detection to side-by-side model comparison. It was designed for political economy, but as we will see, its toolkit applies to any panel data problem where you need to assess the correspondence between two variables across entities and time.

The Core Idea (in Plain Language)

In the classical and Marxian tradition, the value of a commodity is determined by the total labor time — direct and indirect — required to produce it. If a table requires 10 hours of socially necessary labor and a chair requires 5, the table’s value is twice the chair’s.

This gives rise to what economists call direct prices (denoted p^d): prices that are strictly proportional to the labor embodied in each commodity. They represent what prices would be if they perfectly mirrored labor content.

But capitalism does not work that way. Capital flows between sectors seeking the highest return, and competition tends to equalize the rate of profit across industries. The prices that emerge from this process are called prices of production (denoted p^π). They redistribute surplus value: sectors with higher organic composition of capital (more machinery relative to labor) tend to have prices of production above their direct prices, and vice versa.

The central empirical question is: despite this redistribution, how closely do direct prices and prices of production correspond?

The standard test is a log-linear regression:

ln(p^π_it) = α + β · ln(p^d_it) + u_it

where i indexes sectors and t indexes time periods.

Three hypotheses are at stake:
- β ≈ 1: a one-percent increase in direct prices is associated with roughly a one-percent increase in production prices (proportionality).
- R² ≈ 1: direct prices explain the vast majority of the variation in production prices.
- Stability: the relationship holds consistently across time periods.
If all three hold, the labor theory of value has strong empirical support. valueprhr gives you the tools to test each one rigorously.

What’s Inside the Package

valueprhr organizes its functionality into six modules. Here is what each does and why it matters.

1. Data Preparation

Real-world data rarely arrives in the format econometric methods require. The package accepts two data frames in wide format (rows = years, columns = sectors) — one for direct prices, one for production prices — and converts them into the long-format panel structure that econometric models expect.
library(valueprhr) # Wide format: Year | Agriculture | Manufacturing | Mining | ... direct <- read.csv("direct_prices.csv") production <- read.csv("production_prices.csv") # Convert to long panel: Year, Sector, direct, production, log_direct, log_production panel <- prepare_panel_data(direct, production, log_transform = TRUE) head(panel) #> Year Sector direct production log_direct log_production #> 1 1960 Agriculture 45.2 48.1 3.81 3.87 #> 2 1961 Agriculture 46.0 49.0 3.83 3.89 #> ...
The function prepare_log_matrices() does the same job but returns matrix format, which is what the multivariate methods (PLS, CCA) need.

2. Panel Data Models

This is where the core econometrics happens. The package implements two complementary specifications:

Two-Way Fixed Effects (FE) controls for both sector-specific and time-specific unobserved heterogeneity:

Y_it = α_i + γ_t + β · X_it + ε_it

In plain terms: every sector has its own baseline (some sectors are systematically more expensive), every year has its own macroeconomic conditions (inflation, crises), and the model isolates the within variation to estimate the core relationship.
fe <- fit_twoway_fe(panel, robust_se = TRUE, cluster_type = "group") print(fe) #> Two-Way Fixed Effects Model #> ============================ #> Observations: 1200 | Sectors: 20 | Years: 60 #> R-squared: 0.9876 | Adjusted R-squared: 0.9870 #> #> log_direct coefficient: #> Estimate = 0.9754, SE = 0.0123, t = 79.30, p = 0.0000
The cluster_type = "group" option computes cluster-robust standard errors at the sector level, which accounts for serial correlation within each sector’s time series.

Mundlak Correlated Random Effects (CRE) takes a different route. Instead of dummy variables for every sector, it decomposes the predictor into a within-sector component (how X_it deviates from sector i‘s average) and a between-sector component (the sector average itself):

Y_it = α + β_W · (X_it − X̄_i) + β_B · X̄_i + u_i + ε_it

In data science language: this is a way to control for group-level confounders without the computational cost of N dummy variables. If β_W = β_B, the within and between effects are the same, and a simpler Random Effects model suffices. If they differ, the relationship between values and prices operates differently within a sector over time than across sectors.
# Add Mundlak terms panel_cre <- create_mundlak_data(panel, x_var = "log_direct") # Fit the model cre <- fit_mundlak_cre(panel_cre, include_time_fe = TRUE) print(cre) #> Mundlak Correlated Random Effects Model #> ========================================= #> Within-sector effect (beta_W): 0.9680 #> Between-sector effect (beta_B): 0.9912 #> #> Mundlak test H0: beta_W = beta_B #> F-stat = 2.14, p-value = 0.1438 #> -> Fail to reject H0: RE/CRE specification is consistent
The function test_mundlak_specification() formalizes this check. A low p-value means you should stick with Fixed Effects; a high p-value means the simpler model is adequate.

The package also includes a Panel Granger Causality test (the Dumitrescu-Hurlin procedure), which tests whether past values of direct prices help predict current production prices — and vice versa.
panel_granger_test(panel, lags = c(1, 2)) #> direction lag W_stat Z_stat p_value significant #> 1 direct -> production 1 8.432 3.126 0.0018 TRUE #> 2 direct -> production 2 6.215 2.441 0.0146 TRUE #> 3 production -> direct 1 5.890 2.103 0.0354 TRUE #> 4 production -> direct 2 4.012 1.332 0.1828 FALSE
3. Bayesian Models

Classical (frequentist) estimation gives you a single point estimate for β. Bayesian methods give you a full probability distribution over possible values, incorporating your prior beliefs and updating them with the data.

In econometric language: instead of β̂ = 0.975 ± 0.012, you get a posterior distribution showing that β lies between 0.95 and 1.00 with 95% probability.

The package offers two Bayesian approaches:

Sector-by-Sector Bayesian GLM fits an independent Bayesian linear model for each sector, using weakly informative priors (the rstanarm package handles the MCMC sampling via Stan). Each sector gets its own slope and intercept, along with Leave-One-Out Cross-Validation (LOO-CV) scores.
bayes <- fit_bayesian_glm_sectors( direct, production, chains = 4, iter = 4000 ) print(bayes$summary_table) #> Sector beta_mean beta_sd beta_lower beta_upper elpd looic n_obs #> 1 Agriculture 0.982 0.025 0.933 1.031 -42.3 84.6 60 #> 2 Manufacturing 0.971 0.031 0.910 1.031 -38.7 77.4 60 #> 3 Mining 0.958 0.042 0.876 1.041 -45.1 90.2 60 #> ...
In data science language: LOO-CV is a principled way to assess out-of-sample predictive performance without holding out data. The LOOIC (LOO Information Criterion) is the Bayesian analogue of AIC — lower is better.

Bayesian Hierarchical Model goes further by pooling information across sectors. Instead of treating each sector in isolation, it assumes that sector-specific slopes are drawn from a common population distribution:

β_i ~ N(μ_β, σ_β²)

Sectors with less data “borrow strength” from the population mean. This is especially valuable when some sectors have short time series.
hier <- fit_bayesian_hierarchical(panel, include_time = TRUE) print(hier) #> Bayesian Hierarchical Model #> ============================ #> Observations: 1200 | Sectors: 20 #> #> LOO-CV: #> ELPD = -312.45 #> LOOIC = 624.90 #> #> Population-level effects: #> parameter mean sd 2.5% 97.5% #> 1 (Intercept) 0.1423 0.0892 -0.032 0.317 #> 2 log_direct 0.9734 0.0145 0.945 1.002 #> 3 Time_scaled 0.0031 0.0018 -0.0004 0.007
4. Multivariate Analysis

When the number of sectors (N) is large relative to the number of time periods (T), standard regression becomes unstable. This is the “small T, large N” problem common in panel data. The package offers three multivariate techniques to handle it:

Partial Least Squares (PLS) extracts latent components that explain covariance between direct prices and production prices. It handles multicollinearity gracefully and is widely used in chemometrics, genomics, and now in value-price analysis.
matrices <- prepare_log_matrices(direct, production) pls <- fit_pls_multivariate( matrices$X_clean, matrices$Y_clean, max_components = 8 ) print(pls) #> Partial Least Squares (PLS) Regression #> ======================================= #> Optimal components: 3 #> #> R-squared by component: #> n_components R2_train R2_cv #> 1 1 0.942 0.938 #> 2 2 0.971 0.965 #> 3 3 0.984 0.980
Canonical Correlation Analysis (CCA) finds linear combinations of direct prices and production prices that are maximally correlated. In econometric language: CCA extracts the “shared economic signal” — the common factor driving both sets of prices.
cca <- run_sparse_cca(matrices$X_clean, matrices$Y_clean, n_components = 3) print(cca) #> Canonical Correlation Analysis #> =============================== #> Components: 3 #> #> Canonical correlations: #> CC1: r = 0.9987 (Var X: 92.3%, Var Y: 91.8%) #> CC2: r = 0.9841 (Var X: 5.1%, Var Y: 5.4%) #> CC3: r = 0.9523 (Var X: 1.8%, Var Y: 1.9%)
The first canonical correlation above 0.99 indicates an extremely tight structural link between the two price systems.

Panel VAR captures dynamic feedback: do lagged values of direct prices predict current production prices, and vice versa?
pvar <- fit_panel_var(panel, lags = 2, transformation = "fd")
5. Cross-Validation

Standard k-fold cross-validation violates temporal ordering. If you train on 1960–1990 and test on 1985–1990, future information leaks into the training set. The package implements two time-aware approaches:

Rolling Window CV trains on t₀ … t_W, tests on t_W+1 … t_W+H, then rolls the window forward.
cv <- rolling_window_cv( panel, window_sizes = c(20, 30), step_size = 2, test_horizon = 3 ) print(cv$summary)
Leave-One-Sector-Out (LOSO) trains on all sectors except one and predicts the held-out sector. This tests cross-sectional generalization: does the value-price relationship estimated from other sectors hold for agriculture? For mining? For finance?
loso <- leave_one_sector_out(panel) print(loso$summary) #> metric mean sd #> 1 RMSE 0.04521 0.01832 #> 2 MAE 0.03587 0.01456 #> 3 R_squared 0.96120 0.02340
An average R² above 0.96 in LOSO-CV means the relationship generalizes robustly across sectors.

6. Structural Break Tests

Has the value-price relationship been stable over time? Or did it shift at some point — due to globalization, a methodological change in data construction, a technological revolution, or a regime shift in profit rate equalization?

The package aggregates the panel to a time series and applies a battery of tests:
breaks <- test_structural_breaks(panel, break_date = 1990) print(breaks) #> Structural Break Tests #> ======================== #> Time-series observations: 60 #> #> Chow Test: #> Break date: 1990 #> F-stat = 1.8420, p = 0.1687 #> #> supF / Bai-Perron Test: #> supF = 5.2130, p = 0.0842 #> Breaks detected: 0
A non-significant result is actually good news here: it means the value-price correspondence has been structurally stable across the entire sample period.

The Full Pipeline in One Command

If you want to run everything at once — data preparation, FE and CRE models, cross-validation, structural break tests, and model comparison — the package offers a single entry point:
results <- run_full_analysis( direct, production, run_bayesian = FALSE, # Set TRUE if you have rstanarm installed run_cv = TRUE, run_breaks = TRUE, verbose = TRUE ) # Access everything print(results$comparison) print(results$cv_summary) cat(format_break_results(results$breaks))
You can then export the comparison table and CV results to CSV:
export_results_csv( results$comparison, results$cv_summary, output_dir = "results/" )
Beyond Political Economy: General Panel Data Applications

Although valueprhr was built for the specific question of value-price correspondence, its methods are general-purpose panel data tools. Any research problem involving the relationship between two variables observed across entities and time can benefit from the package:
- Health economics: Does out-of-pocket spending track underlying treatment costs across regions over time?
- Environmental economics: Do carbon prices reflect the embodied emissions of goods across industries?
- Education: Do standardized test scores correspond to instructional expenditure across school districts over decades?
- Finance: Do book values predict market valuations across sectors?
- Any two-variable panel regression where you need fixed effects, Mundlak decomposition, robust standard errors, time-aware cross-validation, or structural break detection.
The key requirement is that your data has a panel structure (entities × time) and that the distributional assumptions of the models are reasonable for your context. The methods — two-way FE, Mundlak CRE, Bayesian hierarchical models, PLS, CCA, rolling-window CV, structural break tests — are econometric staples that transcend any particular application domain.

Installation and Dependencies

The package requires R ≥ 4.1.0. Core functionality depends only on base R and the Metrics package. Extended features (Bayesian models, panel data infrastructure, structural break tests) are handled through soft dependencies that are loaded on demand:
# Install from GitHub install.packages("devtools") devtools::install_github("isadorenabi/valueprhr") # Optional: install all suggested packages at once suggested <- c( "rstanarm", "loo", "plm", "lme4", "pls", "vars", "panelvar", "strucchange", "lmtest", "sandwich", "dplyr", "tidyr", "tibble" ) install.packages(suggested[!sapply(suggested, requireNamespace, quietly = TRUE)])
Note for Bayesian models: rstanarm requires a working C++ toolchain — Rtools on Windows, Xcode Command Line Tools on macOS, or build-essential on Linux.

What Makes This Package Methodologically Different

Three features distinguish valueprhr from a hand-rolled analysis:
1. Time-aware validation. Most applied work reports in-sample R² as evidence of fit. valueprhr pairs every model with rolling-window and leave-one-sector-out cross-validation, giving you out-of-sample performance that is honest about temporal dependence and cross-sectional generalization.
2. The Mundlak decomposition. By splitting effects into within-sector and between-sector components, the package lets you test whether the value-price relationship operates at the sector level (structural), at the temporal level (cyclical), or both. This is a nuance that most empirical studies in this literature overlook.
3. Bayesian hierarchical pooling. Sectors with short time series are a common headache. The hierarchical model lets small sectors borrow statistical strength from the population, producing more stable estimates than independent sector-by-sector regressions.
A Note on the Underlying Data

The wiki documentation mentions that market price indices used in this framework are constructed by temporally disaggregating the aggregate Consumer Price Index using the Input-Output matrix as a structural indicator, relying on closed-form Bayesian solutions from the BayesianDisaggregation library. This is a methodological detail worth understanding: the sectoral prices are not raw market quotes but statistically consistent decompositions of the macroeconomic aggregate. This ensures that the estimated sectoral price movements add up to the observed CPI, a property that many ad hoc sectoral price datasets lack.

Citation

If you use valueprhr in your research:
@software{gomezjulian2025valueprhr, author = {Gómez Julián, José Mauricio}, title = {valueprhr: Value-Price Analysis with Bayesian and Panel Data Methods}, year = {2025}, url = {https://github.com/isadorenabi/valueprhr}, note = {R package version 0.1.0} }
Author: José Mauricio Gómez Julián — ORCID — isadore.nabi@pm.me

License: MIT

Repository: github.com/IsadoreNabi/valueprhr

The labor theory of value is either one of the most important ideas in the history of economics or one of the most contested. Either way, it deserves better tools than a spreadsheet and a prayer. valueprhr brings the full machinery of modern econometrics to the question — and lets the data speak for itself.
Compártelo:
X
Facebook
Print
LinkedIn
Reddit
Tumblr
Pinterest
Telegram
WhatsApp
Email
Threads
Mastodon
Nextdoor
X
Bluesky
Like this:
Like Loading…
15 July, 2026

Extracting Signals from Noise: How SignalY Tackles Three Hard Problems in Panel Data Analysis

You can also find this library at CRAN and download it directly from R and RStudio.

LISTEN TO THIS POST AS A PODCAST:

The Problem Every Quantitative Researcher Knows

Imagine you are staring at a spreadsheet with 50 columns and a few hundred rows of macroeconomic indicators. Somewhere inside this matrix, a handful of variables carry the signal you care about. The rest is noise — or worse, confounding variation that masquerades as signal. You need to answer three questions: Which variables actually matter? What is the latent structure driving them? And how persistent are the components you extract?

If you have worked with panel data for any length of time, you know that these questions are rarely addressed by a single tool. You run a LASSO in one environment, a PCA in another, an ADF test in a third. Each method lives in a different package with different input formats, different assumptions, and different output structures. Stitching the results together is left to you.

SignalY is an R package that was built to solve exactly this fragmentation problem. Developed by José Mauricio Gómez Julián and released under the MIT license, it provides a unified framework for signal extraction from panel data through multivariate time series analysis. Its design rests on three analytical pillars — column selection, series decomposition, and persistence analysis — that can be used independently or chained together through a single orchestrating function.

This post walks through the problem SignalY solves, the methods it implements, and why the combination matters for applied econometrics and data science.

Pillar 1: Which Variables Matter?

The first challenge in any high-dimensional analysis is selection. When you have dozens of potential predictors, you need a principled way to determine which ones carry structural information and which ones are along for the ride.

The Horseshoe Prior

SignalY approaches this problem through Bayesian sparse regression with the Horseshoe prior. The Horseshoe, introduced by Carvalho, Polson, and Scott (2010) and refined for practical variable selection by Piironen and Vehtari (2017), is a global-local shrinkage prior with a distinctive property: it is aggressive around zero (shrinking noise variables strongly toward zero) while maintaining heavy tails (allowing true signals to escape shrinkage). This dual behavior makes it particularly well-suited for sparse problems where you expect only a few variables to matter, but you do not know which ones.

The math behind this is elegant. Each coefficient βⱼ is given a prior with two layers of shrinkage:

A local parameter λⱼ that controls how much each individual coefficient is shrunk.
A global parameter τ that governs overall sparsity.

The result is a shrinkage profile where most coefficients collapse toward zero — the global pull — while a small number of coefficients stand apart, barely affected — the local escape. This is what gives the Horseshoe its name: the prior density looks like the shape of a horseshoe, with a sharp spike at zero and long, flat arms extending outward.

SignalY’s fit_horseshoe() function estimates this model and provides built-in shrinkage profile diagnostics, so you can visually inspect which variables survived shrinkage and by how much.

Four Ways to Select Variables

Fitting a model is one thing; converting the posterior into a concrete variable selection is another. SignalY offers four distinct selection strategies, each with different strengths:

Projection predictive selection (select_by_projection()): This is the most theoretically robust approach. It works by projecting the full posterior onto candidate submodels and selecting the smallest submodel whose predictive distribution is close enough to the full model. The reference is Piironen and Vehtari (2017), and the implementation respects the posterior geometry rather than relying on ad-hoc thresholds.
Credible interval exclusion (select_by_credible_interval()): Selects variables whose posterior credible intervals do not include zero. Intuitive and easy to interpret, though it can be conservative in high-dimensional settings.
Shrinkage-based selection (select_by_shrinkage()): Uses the kappa (shrinkage fraction) parameters to identify variables that escaped shrinkage. This is useful when you want to understand the degree of shrinkage, not just the binary in-or-out question.
Magnitude-based screening (select_by_magnitude()): A straightforward effect-size filter. Useful as a first pass or when you need to combine Bayesian inference with a frequentist-style screening step.

The fact that SignalY provides all four in a coherent pipeline — not as separate, unrelated functions — is the key design decision. You can run all four and cross-validate the results, or choose the one that best matches your inferential philosophy.

Beyond Regression: Factor Discovery

Sometimes the question is not “which of my 50 variables matter?” but rather “what are the few latent factors driving all 50?” SignalY addresses this through two complementary methods:

PCA with block bootstrap (pca_bootstrap()): Standard principal component analysis, but with block bootstrap confidence intervals that account for temporal dependence in time series data. It also includes entropy-based topology analysis, which measures the informational content of each component.
Dynamic Factor Models (estimate_dfm()): Implements the Bai and Ng (2002) information criteria for automatic determination of the number of static factors, then fits a VAR (Vector Autoregression) on the factor dynamics. This captures not just what the latent factors are, but how they evolve over time.

Pillar 2: What Is the Underlying Structure?

Once you know which variables matter (or have constructed a composite signal), the next question is decomposition: what are the trend, cycle, and residual components of your series?

This is where signal processing meets econometrics, and SignalY implements three methodologically distinct approaches, each with its own strengths.

Wavelet Decomposition

filter_wavelet() implements the Maximal Overlap Discrete Wavelet Transform (MODWT) using Daubechies wavelets, following the framework of Percival and Walden (2000).

Unlike a Fourier transform, which decomposes a signal into infinite sinusoids (losing all time information), a wavelet decomposes a signal into localized, finite-length oscillations at different scales. The MODWT variant is particularly useful for time series because it does not decimate the data (no downsampling), meaning the output length matches the input length at every scale.

In practice, the wavelet decomposition separates a series into:

Detail coefficients (D1, D2, D3, …): capturing oscillations at progressively coarser time scales — high-frequency noise in D1-D2, business-cycle frequencies in D3-D4, and longer cycles in higher levels.
Smooth coefficients (S): the low-frequency approximation that captures the trend.

SignalY includes multi-resolution variance analysis, which tells you how much of the total variance is explained at each scale. This is invaluable for understanding whether your series is dominated by high-frequency noise, medium-term fluctuations, or long-run trends.

Empirical Mode Decomposition

filter_emd() implements Empirical Mode Decomposition (Huang et al., 1998), a fundamentally different approach. Where wavelets impose a predetermined basis (Daubechies, Haar, Symmlet, etc.), EMD is fully data-adaptive. It works by iteratively sifting the signal — identifying local extrema, fitting envelopes, and subtracting the mean — until it extracts Intrinsic Mode Functions (IMFs) that satisfy specific oscillatory conditions.

The key advantage of EMD is that it makes no assumptions about stationarity or linearity. The IMFs are defined by the data itself, not by a mathematical basis. This makes EMD particularly powerful for:

Non-stationary signals whose frequency content changes over time.
Non-linear oscillations that cannot be captured by fixed-basis decompositions.
Signals where the “natural” decomposition is not known a priori.

The trade-off is that EMD can be sensitive to end effects and mode mixing, though SignalY’s implementation includes standard mitigations.

HP-GC Bayesian Filter

filter_hpgc() implements the Grant and Chan (2017) unobserved-components Hodrick-Prescott filter, estimated via MCMC. This is a significant upgrade over the traditional HP filter, which requires you to manually set the smoothing parameter λ (the famous λ = 1600 for quarterly data, or λ = 6.25 for annual data, or any of the other arbitrary rules of thumb floating around the literature).

The HP-GC approach formulates the decomposition as a Bayesian unobserved-components model:

A trend component whose second differences are penalized (this is the smoothness prior, equivalent to the HP penalty).
A cycle component modeled as an AR(2) process.
The smoothing parameter λ is estimated from the data via MCMC, not fixed by the user.

This removes one of the most criticized aspects of the classical HP filter — its sensitivity to the arbitrary choice of λ — while preserving its interpretability. The output includes the estimated trend, cycle, and residual, each with full posterior distributions.

filter_all(): Compare All Three

A particularly useful design choice is the filter_all() function, which runs all three decomposition methods on the same series and returns the results in a comparable format. This is not just a convenience function; it is an epistemological statement. No single decomposition method is universally correct. By running all three and comparing, you can identify components that are robust across methods (strong signal) versus components that depend on the specific decomposition assumptions (potentially method artifact).

Pillar 3: How Persistent Is the Signal?

The third question — what is the persistence regime of your series or its components — is critical for downstream modeling. If your extracted trend is a random walk, that has very different implications than if it is a stationary AR process. If your cycle is near-unit-root, standard mean-reversion models will fail.

A Comprehensive Unit Root Battery

test_unit_root() runs four classical tests with complementary null hypotheses:

Test	Null Hypothesis	Key Feature
Augmented Dickey-Fuller (ADF)	Unit root exists	Most widely used; sensitive to lag selection
Phillips-Perron (PP)	Unit root exists	Non-parametric correction for serial correlation
KPSS	Series is stationary	Reversed null; useful as cross-check against ADF
Elliott-Rothenberg-Stock (ERS)	Unit root exists	Point optimal test with higher power near unity

The critical insight is that no single test is definitive. The ADF and PP tests can fail to reject a false unit root (low power near unity). The KPSS test has the opposite null hypothesis, so it can detect stationarity that ADF misses. By running all four and synthesizing the results, SignalY provides a more robust classification than any individual test.

The automated synthesis follows a standard decision logic:

If ADF/PP/ERS reject unit root and KPSS fails to reject stationarity → stationary.
If ADF/PP/ERS fail to reject and KPSS rejects → unit root.
Mixed results → borderline / near-unit-root, flagged for careful interpretation.

This automated synthesis is not a black box; the individual test statistics and p-values are all available for inspection. But the synthesis gives you a quick, defensible classification without manually cross-referencing four separate test outputs.

The Orchestrator: One Call, Full Pipeline

The signal_analysis() function is the centerpiece of SignalY’s design philosophy. A single call can run the complete analysis pipeline:

			
result <- signal_analysis(
  data = data,
  y_formula = Y ~ X1 + X2 + X3,
  methods = c("wavelet", "emd", "pca", "dfm", "unitroot"),
  verbose = TRUE
)

		

This executes:

Column selection (PCA, DFM, optionally Horseshoe).
Series decomposition (Wavelet, EMD).
Persistence analysis (Unit Root Battery).

…and returns a unified result object with print(), summary(), and plot() methods. The plot() method generates interactive plotly dashboards with filter trends, coefficient profiles, PCA loadings, and DFM factor panels.

The formula interface (Y ~ X1 + X2 + X3) makes it feel like a standard R regression call, while the methods argument lets you mix and match analytical layers as needed.

How Well Does It Work?

The Wiki includes recovery benchmarks on synthetic data with known ground truth. These are worth highlighting because they address the most important question: does this actually work?

Task	Method	Recovery Metric
Factor structure (3 latent factors)	PCA / DFM	r > 0.95, exact factor count
Sparse variable selection (5 of 50)	Horseshoe	F1 > 0.85, Precision > 0.90
Logarithmic trend recovery	EMD	r > 0.95 with true trend
Multi-scale cycle extraction	Wavelet (D3+D4)	r > 0.70 with true cycle
Stochastic trend + AR(2) cycle	HP-GC Bayesian	Trend r > 0.90, cycle r > 0.50
Stationarity classification	Unit Root Battery	4/4 correct on synthetic data

A few things stand out:

The Horseshoe achieves over 90% precision in a 5-of-50 sparse selection problem. This means that when it says a variable matters, it is almost always right. The F1 score above 0.85 indicates a good balance between precision and recall.
Factor recovery is near-perfect (r > 0.95), and the DFM correctly identifies the exact number of latent factors.
Wavelet cycle extraction at r > 0.70 and HP-GC cycle extraction at r > 0.50 reflect the inherent difficulty of extracting cyclical components from noisy data. These are realistic numbers, not inflated claims.
Unit root classification achieves 100% accuracy on synthetic data with clear-cut cases. Real-world data is messier, but this validates the synthesis logic.

Who Should Use SignalY?

SignalY is built for three overlapping communities:

Economists and econometricians working with panel or multivariate time series data who need to move from raw data to structural inference — identifying relevant variables, extracting latent factors, decomposing signals, and characterizing persistence — without stitching together five different packages.

Quantitative researchers in finance, macro, or political economy who face high-dimensional predictor sets and need principled Bayesian variable selection rather than stepwise regression or arbitrary LASSO tuning.

Data scientists working on signal processing problems where the signals are non-stationary, non-linear, or embedded in high-dimensional panels, and where the standard Python signal processing toolkit does not provide the statistical rigor needed for publication-quality inference.

Getting Started

Installation is straightforward:

			
# From GitHub
remotes::install_github("IsadoreNabi/SignalY")
library(SignalY)
# Minimal workflow
data <- data.frame(Y = as.vector(Y), X)
result <- signal_analysis(data = data, y_formula = "Y",
                          methods = c("pca", "wavelet", "unitroot"))
plot(result)

		

The package is MIT-licensed, actively maintained (current version 1.1.2), and designed to work with standard R data frames.

The Bigger Picture

What makes SignalY interesting is not any single method — the Horseshoe prior, MODWT, EMD, and unit root tests all exist in other packages. The value is in the integration. By placing Bayesian sparse regression, spectral decomposition, and persistence analysis inside a single coherent framework with a unified interface, SignalY enables workflows that are difficult to replicate otherwise:

Run a Horseshoe regression to select variables, then decompose the fitted signal with wavelets, then test the stationarity of the extracted components — all without changing packages, data formats, or mental models.
Compare wavelet, EMD, and HP-GC decompositions of the same series to identify robust components versus method-dependent artifacts.
Use the DFM to discover latent factors, then test each factor’s persistence regime to inform your downstream modeling choices.

In applied econometrics, the quality of your inference depends on the coherence of your pipeline. SignalY makes that coherence a feature rather than a chore.

SignalY is developed by José Mauricio Gómez Julián. The source code, documentation, and wiki are available at github.com/IsadoreNabi/SignalY under the MIT License.

15 July, 2026

Tag: datos de panel

Marx, Adam Smith, and the Law of Large Numbers

The Assumption Hiding in Plain Sight

The Mathematical Backbone: The Law of Large Numbers

The Catch: Independence and Identical Distributions

Non-Classical Varieties: When the Rules Relax

What the Data Actually Shows

Finding 1: No Identical Distributions

Finding 2: No Statistical Independence

Finding 3: Differences Tend Toward Zero

Why This Matters

A Few Honest Caveats — And Why They No Longer Apply

Compártelo:

Like this:

valueprhr: When Do Market Prices Reflect the Labor That Produced Them? A Modern R Toolkit for an Old Question

LISTEN TO THIS POST AS A PODCAST:

Why This Package Exists

The Core Idea (in Plain Language)

What’s Inside the Package

1. Data Preparation

2. Panel Data Models

3. Bayesian Models

4. Multivariate Analysis

5. Cross-Validation

6. Structural Break Tests

The Full Pipeline in One Command

Beyond Political Economy: General Panel Data Applications

Installation and Dependencies

What Makes This Package Methodologically Different

A Note on the Underlying Data

Citation

Compártelo:

Like this:

Extracting Signals from Noise: How SignalY Tackles Three Hard Problems in Panel Data Analysis

LISTEN TO THIS POST AS A PODCAST:

The Problem Every Quantitative Researcher Knows

Pillar 1: Which Variables Matter?

The Horseshoe Prior

Four Ways to Select Variables

Beyond Regression: Factor Discovery

Pillar 2: What Is the Underlying Structure?

Wavelet Decomposition

Empirical Mode Decomposition

HP-GC Bayesian Filter

filter_all(): Compare All Three

Pillar 3: How Persistent Is the Signal?

A Comprehensive Unit Root Battery

The Orchestrator: One Call, Full Pipeline

How Well Does It Work?

Who Should Use SignalY?

Getting Started

The Bigger Picture

Compártelo:

Like this: