You can also find this library at CRAN and download it directly from R and RStudio.
LISTEN TO THIS POST AS A PODCAST:
The Problem Every Quantitative Researcher Knows
Imagine you are staring at a spreadsheet with 50 columns and a few hundred rows of macroeconomic indicators. Somewhere inside this matrix, a handful of variables carry the signal you care about. The rest is noise — or worse, confounding variation that masquerades as signal. You need to answer three questions: Which variables actually matter? What is the latent structure driving them? And how persistent are the components you extract?
If you have worked with panel data for any length of time, you know that these questions are rarely addressed by a single tool. You run a LASSO in one environment, a PCA in another, an ADF test in a third. Each method lives in a different package with different input formats, different assumptions, and different output structures. Stitching the results together is left to you.
SignalY is an R package that was built to solve exactly this fragmentation problem. Developed by José Mauricio Gómez Julián and released under the MIT license, it provides a unified framework for signal extraction from panel data through multivariate time series analysis. Its design rests on three analytical pillars — column selection, series decomposition, and persistence analysis — that can be used independently or chained together through a single orchestrating function.
This post walks through the problem SignalY solves, the methods it implements, and why the combination matters for applied econometrics and data science.
Pillar 1: Which Variables Matter?
The first challenge in any high-dimensional analysis is selection. When you have dozens of potential predictors, you need a principled way to determine which ones carry structural information and which ones are along for the ride.
The Horseshoe Prior
SignalY approaches this problem through Bayesian sparse regression with the Horseshoe prior. The Horseshoe, introduced by Carvalho, Polson, and Scott (2010) and refined for practical variable selection by Piironen and Vehtari (2017), is a global-local shrinkage prior with a distinctive property: it is aggressive around zero (shrinking noise variables strongly toward zero) while maintaining heavy tails (allowing true signals to escape shrinkage). This dual behavior makes it particularly well-suited for sparse problems where you expect only a few variables to matter, but you do not know which ones.
The math behind this is elegant. Each coefficient βⱼ is given a prior with two layers of shrinkage:
- A local parameter λⱼ that controls how much each individual coefficient is shrunk.
- A global parameter τ that governs overall sparsity.
The result is a shrinkage profile where most coefficients collapse toward zero — the global pull — while a small number of coefficients stand apart, barely affected — the local escape. This is what gives the Horseshoe its name: the prior density looks like the shape of a horseshoe, with a sharp spike at zero and long, flat arms extending outward.
SignalY’s fit_horseshoe() function estimates this model and provides built-in shrinkage profile diagnostics, so you can visually inspect which variables survived shrinkage and by how much.
Four Ways to Select Variables
Fitting a model is one thing; converting the posterior into a concrete variable selection is another. SignalY offers four distinct selection strategies, each with different strengths:
- Projection predictive selection (
select_by_projection()): This is the most theoretically robust approach. It works by projecting the full posterior onto candidate submodels and selecting the smallest submodel whose predictive distribution is close enough to the full model. The reference is Piironen and Vehtari (2017), and the implementation respects the posterior geometry rather than relying on ad-hoc thresholds. - Credible interval exclusion (
select_by_credible_interval()): Selects variables whose posterior credible intervals do not include zero. Intuitive and easy to interpret, though it can be conservative in high-dimensional settings. - Shrinkage-based selection (
select_by_shrinkage()): Uses the kappa (shrinkage fraction) parameters to identify variables that escaped shrinkage. This is useful when you want to understand the degree of shrinkage, not just the binary in-or-out question. - Magnitude-based screening (
select_by_magnitude()): A straightforward effect-size filter. Useful as a first pass or when you need to combine Bayesian inference with a frequentist-style screening step.
The fact that SignalY provides all four in a coherent pipeline — not as separate, unrelated functions — is the key design decision. You can run all four and cross-validate the results, or choose the one that best matches your inferential philosophy.
Beyond Regression: Factor Discovery
Sometimes the question is not “which of my 50 variables matter?” but rather “what are the few latent factors driving all 50?” SignalY addresses this through two complementary methods:
- PCA with block bootstrap (
pca_bootstrap()): Standard principal component analysis, but with block bootstrap confidence intervals that account for temporal dependence in time series data. It also includes entropy-based topology analysis, which measures the informational content of each component. - Dynamic Factor Models (
estimate_dfm()): Implements the Bai and Ng (2002) information criteria for automatic determination of the number of static factors, then fits a VAR (Vector Autoregression) on the factor dynamics. This captures not just what the latent factors are, but how they evolve over time.
Pillar 2: What Is the Underlying Structure?
Once you know which variables matter (or have constructed a composite signal), the next question is decomposition: what are the trend, cycle, and residual components of your series?
This is where signal processing meets econometrics, and SignalY implements three methodologically distinct approaches, each with its own strengths.
Wavelet Decomposition
filter_wavelet() implements the Maximal Overlap Discrete Wavelet Transform (MODWT) using Daubechies wavelets, following the framework of Percival and Walden (2000).
Unlike a Fourier transform, which decomposes a signal into infinite sinusoids (losing all time information), a wavelet decomposes a signal into localized, finite-length oscillations at different scales. The MODWT variant is particularly useful for time series because it does not decimate the data (no downsampling), meaning the output length matches the input length at every scale.
In practice, the wavelet decomposition separates a series into:
- Detail coefficients (D1, D2, D3, …): capturing oscillations at progressively coarser time scales — high-frequency noise in D1-D2, business-cycle frequencies in D3-D4, and longer cycles in higher levels.
- Smooth coefficients (S): the low-frequency approximation that captures the trend.
SignalY includes multi-resolution variance analysis, which tells you how much of the total variance is explained at each scale. This is invaluable for understanding whether your series is dominated by high-frequency noise, medium-term fluctuations, or long-run trends.
Empirical Mode Decomposition
filter_emd() implements Empirical Mode Decomposition (Huang et al., 1998), a fundamentally different approach. Where wavelets impose a predetermined basis (Daubechies, Haar, Symmlet, etc.), EMD is fully data-adaptive. It works by iteratively sifting the signal — identifying local extrema, fitting envelopes, and subtracting the mean — until it extracts Intrinsic Mode Functions (IMFs) that satisfy specific oscillatory conditions.
The key advantage of EMD is that it makes no assumptions about stationarity or linearity. The IMFs are defined by the data itself, not by a mathematical basis. This makes EMD particularly powerful for:
- Non-stationary signals whose frequency content changes over time.
- Non-linear oscillations that cannot be captured by fixed-basis decompositions.
- Signals where the “natural” decomposition is not known a priori.
The trade-off is that EMD can be sensitive to end effects and mode mixing, though SignalY’s implementation includes standard mitigations.
HP-GC Bayesian Filter
filter_hpgc() implements the Grant and Chan (2017) unobserved-components Hodrick-Prescott filter, estimated via MCMC. This is a significant upgrade over the traditional HP filter, which requires you to manually set the smoothing parameter λ (the famous λ = 1600 for quarterly data, or λ = 6.25 for annual data, or any of the other arbitrary rules of thumb floating around the literature).
The HP-GC approach formulates the decomposition as a Bayesian unobserved-components model:
- A trend component whose second differences are penalized (this is the smoothness prior, equivalent to the HP penalty).
- A cycle component modeled as an AR(2) process.
- The smoothing parameter λ is estimated from the data via MCMC, not fixed by the user.
This removes one of the most criticized aspects of the classical HP filter — its sensitivity to the arbitrary choice of λ — while preserving its interpretability. The output includes the estimated trend, cycle, and residual, each with full posterior distributions.
filter_all(): Compare All Three
A particularly useful design choice is the filter_all() function, which runs all three decomposition methods on the same series and returns the results in a comparable format. This is not just a convenience function; it is an epistemological statement. No single decomposition method is universally correct. By running all three and comparing, you can identify components that are robust across methods (strong signal) versus components that depend on the specific decomposition assumptions (potentially method artifact).
Pillar 3: How Persistent Is the Signal?
The third question — what is the persistence regime of your series or its components — is critical for downstream modeling. If your extracted trend is a random walk, that has very different implications than if it is a stationary AR process. If your cycle is near-unit-root, standard mean-reversion models will fail.
A Comprehensive Unit Root Battery
test_unit_root() runs four classical tests with complementary null hypotheses:
| Test | Null Hypothesis | Key Feature |
|---|---|---|
| Augmented Dickey-Fuller (ADF) | Unit root exists | Most widely used; sensitive to lag selection |
| Phillips-Perron (PP) | Unit root exists | Non-parametric correction for serial correlation |
| KPSS | Series is stationary | Reversed null; useful as cross-check against ADF |
| Elliott-Rothenberg-Stock (ERS) | Unit root exists | Point optimal test with higher power near unity |
The critical insight is that no single test is definitive. The ADF and PP tests can fail to reject a false unit root (low power near unity). The KPSS test has the opposite null hypothesis, so it can detect stationarity that ADF misses. By running all four and synthesizing the results, SignalY provides a more robust classification than any individual test.
The automated synthesis follows a standard decision logic:
- If ADF/PP/ERS reject unit root and KPSS fails to reject stationarity → stationary.
- If ADF/PP/ERS fail to reject and KPSS rejects → unit root.
- Mixed results → borderline / near-unit-root, flagged for careful interpretation.
This automated synthesis is not a black box; the individual test statistics and p-values are all available for inspection. But the synthesis gives you a quick, defensible classification without manually cross-referencing four separate test outputs.
The Orchestrator: One Call, Full Pipeline
The signal_analysis() function is the centerpiece of SignalY’s design philosophy. A single call can run the complete analysis pipeline:
result <- signal_analysis( data = data, y_formula = Y ~ X1 + X2 + X3, methods = c("wavelet", "emd", "pca", "dfm", "unitroot"), verbose = TRUE)
This executes:
- Column selection (PCA, DFM, optionally Horseshoe).
- Series decomposition (Wavelet, EMD).
- Persistence analysis (Unit Root Battery).
…and returns a unified result object with print(), summary(), and plot() methods. The plot() method generates interactive plotly dashboards with filter trends, coefficient profiles, PCA loadings, and DFM factor panels.
The formula interface (Y ~ X1 + X2 + X3) makes it feel like a standard R regression call, while the methods argument lets you mix and match analytical layers as needed.
How Well Does It Work?
The Wiki includes recovery benchmarks on synthetic data with known ground truth. These are worth highlighting because they address the most important question: does this actually work?
| Task | Method | Recovery Metric |
|---|---|---|
| Factor structure (3 latent factors) | PCA / DFM | r > 0.95, exact factor count |
| Sparse variable selection (5 of 50) | Horseshoe | F1 > 0.85, Precision > 0.90 |
| Logarithmic trend recovery | EMD | r > 0.95 with true trend |
| Multi-scale cycle extraction | Wavelet (D3+D4) | r > 0.70 with true cycle |
| Stochastic trend + AR(2) cycle | HP-GC Bayesian | Trend r > 0.90, cycle r > 0.50 |
| Stationarity classification | Unit Root Battery | 4/4 correct on synthetic data |
A few things stand out:
- The Horseshoe achieves over 90% precision in a 5-of-50 sparse selection problem. This means that when it says a variable matters, it is almost always right. The F1 score above 0.85 indicates a good balance between precision and recall.
- Factor recovery is near-perfect (r > 0.95), and the DFM correctly identifies the exact number of latent factors.
- Wavelet cycle extraction at r > 0.70 and HP-GC cycle extraction at r > 0.50 reflect the inherent difficulty of extracting cyclical components from noisy data. These are realistic numbers, not inflated claims.
- Unit root classification achieves 100% accuracy on synthetic data with clear-cut cases. Real-world data is messier, but this validates the synthesis logic.
Who Should Use SignalY?
SignalY is built for three overlapping communities:
Economists and econometricians working with panel or multivariate time series data who need to move from raw data to structural inference — identifying relevant variables, extracting latent factors, decomposing signals, and characterizing persistence — without stitching together five different packages.
Quantitative researchers in finance, macro, or political economy who face high-dimensional predictor sets and need principled Bayesian variable selection rather than stepwise regression or arbitrary LASSO tuning.
Data scientists working on signal processing problems where the signals are non-stationary, non-linear, or embedded in high-dimensional panels, and where the standard Python signal processing toolkit does not provide the statistical rigor needed for publication-quality inference.
Getting Started
Installation is straightforward:
# From GitHubremotes::install_github("IsadoreNabi/SignalY")library(SignalY)# Minimal workflowdata <- data.frame(Y = as.vector(Y), X)result <- signal_analysis(data = data, y_formula = "Y", methods = c("pca", "wavelet", "unitroot"))plot(result)
The package is MIT-licensed, actively maintained (current version 1.1.2), and designed to work with standard R data frames.
The Bigger Picture
What makes SignalY interesting is not any single method — the Horseshoe prior, MODWT, EMD, and unit root tests all exist in other packages. The value is in the integration. By placing Bayesian sparse regression, spectral decomposition, and persistence analysis inside a single coherent framework with a unified interface, SignalY enables workflows that are difficult to replicate otherwise:
- Run a Horseshoe regression to select variables, then decompose the fitted signal with wavelets, then test the stationarity of the extracted components — all without changing packages, data formats, or mental models.
- Compare wavelet, EMD, and HP-GC decompositions of the same series to identify robust components versus method-dependent artifacts.
- Use the DFM to discover latent factors, then test each factor’s persistence regime to inform your downstream modeling choices.
In applied econometrics, the quality of your inference depends on the coherence of your pipeline. SignalY makes that coherence a feature rather than a chore.
SignalY is developed by José Mauricio Gómez Julián. The source code, documentation, and wiki are available at github.com/IsadoreNabi/SignalY under the MIT License.
