Count every institution that owns Apple and call it a buy signal, and you're mixing 800 passive funds that hold it because it's in an index with 50 activists who actually have conviction. The 800 are noise. The 50 are signal. The raw 13F data doesn't tell you which is which, so I built a system that does.
That system is RACS, a Regime-Conditioned Activist Conviction Score. It builds a behavioral profile of every manager from 120.2M real SEC 13F filings, isolates the genuine high-conviction activists, and scores each stock on how hard that group is buying it, discounted when the trade is already crowded and dialed up or down by whether the current macro regime favors it.
$ andria run phase3
...
12/12 real bugs fixed and regression-tested
125 passed <- up from 9 tests actually running
leakage_audit_passed checks_run=6
backtest_completed overall_sharpe=-0.5937
evaluation_gate_rejected reasons=[...]
pipeline_phase3_complete gate_passed=False
Real log lines from this session: 12 real bugs fixed, a test suite actually running end to end, and a real backtest carried all the way through an evaluation gate that reports its outcome honestly.
Real SEC 13F filings ingested
2004 to 2026, verified end to end
Real, reproducible bugs fixed
from a dead test suite to a signal engine that could not run
Tests actually running in CI
up from 9, 106 existed but pytest never collected them
Evaluation gate outcome
honestly, on 9 real trades, not cherry-picked
Real EDGAR rows vs. the original claim
exceeds the documented "116M+", sourced 2004 to 2026
CUSIP resolution via OpenFIGI
up from 3.6% after a live-verified batch-size fix
Real HDBSCAN silhouette-optimal clusters
not the 4 archetypes the original narrative describes
DuckDB memory ceiling, after the crash
a full-dataset first attempt took down the whole VM
Andria Systems turns SEC 13F filings into a regime-conditioned equity signal: 120.2M real filings in, a behavioral profile per manager, HDBSCAN archetypes, an HMM macro regime, and a leakage-audited backtest behind a publish-or-reject gate. It runs end to end on real SEC data, with 125 tests in CI.
The backtest had never run end to end
Every algorithm, clustering, HMM regime detection, RACS scoring, the leakage audit, PBO/DSR/Monte Carlo, was real and individually tested. But no code path in the CLI or the orchestrator ever ran them together, so the dashboard's numbers were pre-computed placeholders rather than the output of a live run against real filings.
9.7GB of shared RAM, no room for error
A first attempt at ingesting all 53 real SEC bulk files in one DuckDB pass, even with a memory_limit set, pushed the whole machine over the edge and crashed the WSL VM outright, wiping /tmp and disconnecting the session mid-run. Every subsequent stage had to be re-architected around that constraint, not just re-run.
Real government data is not the fixture you tested against
SEC changed its own bulk-file packaging after 2023: a single downloaded zip now bundles late amendments spanning a decade of report quarters. The pipeline's own code comment named the correct format ("edgar has source_quarter e.g. 2021Q1") right above a line that produced a different one, true only because nothing had run it against real files before.
Every DataFrame crossing a module boundary is validated against a schema contract; a mismatch raises DataContractError immediately instead of propagating a silently-wrong table downstream.
andria/ingestion/edgar.py
Hive-partitioned Parquet, zstd, 100K row groups
120,181,830 real rows, 89 clean quarters (2004 Q1 to 2026 Q1), 15,180 managers, 163,619 CUSIPs. All 53 real bulk files SEC has ever published in this format.
andria/ingestion/fred.py
Single Parquet, provenance-tracked
VIX, 10Y-2Y spread, HY credit spread, Fed funds, NFCI, pulled live from FRED's public fredgraph.csv endpoint. No API key needed, despite the README documenting one as required.
andria/ingestion/ofr.py
CSV/XLSX with fallback to cached Parquet
OFR's own endpoint returned HTTP 403 (bot-protected CDN) live. Fell back to the documented FRED NFCIRISK proxy, exactly the designed behavior, not a workaround.
andria/data/market_loader.py (yfinance)
Per-ticker cached Parquet, adjusted close only
Design principle stated in the module docstring: never silently falls back to synthetic pricing. Unmapped CUSIPs are excluded, not fabricated.
Ingestion feeds two independent unsupervised models, behavioral archetypes and macro regimes, which combine into the RACS signal. Nothing that signal produces reaches a reported statistic without first clearing the leakage audit.
$ andria run ingest && andria run phase1 && andria run phase2 && andria run phase3
Ingestion
EDGAR, FRED, and OFR pulled and validated against schema contracts at every boundary; a bad row raises DataContractError immediately rather than propagating.
Manager DNA
6-stage DuckDB pipeline builds a 14-feature behavioral profile per manager per quarter, staged temp tables dropped as soon as consumed to bound peak memory.
Archetype clustering
HDBSCAN sweep over 5 min_cluster_size candidates, best silhouette wins; cosine similarity to hand-authored prototypes assigns stable archetype labels.
Regime detection
4-state Gaussian HMM, full covariance, fit on 5 standardized macro features resampled to quarter-end; same cosine-similarity labeling scheme as clustering.
RACS engine
5-stage DuckDB SQL pipeline scores each CUSIP-quarter by activist consensus, discounted for crowding and adjusted by the current regime's posterior probability.
Backtest + audit
45-day filing lag, NYSE-calendar-aware date snapping, T+1 fills, square-root market impact, and a non-bypassable 6-check leakage audit before any result is trusted.
Statistical validation
Walk-forward folds, Probability of Backtest Overfitting (CSCV), Deflated Sharpe Ratio, 3-way Monte Carlo, and an EvaluationGate that can only publish or reject.
HDBSCAN sweeps min_cluster_size over 5 candidates and keeps the best silhouette score, avoiding a hand-picked cluster count. Cosine similarity to hand-authored prototype vectors then assigns each cluster a stable, human-readable archetype name, the same technique reused for HMM regime labeling below.
7,124 (56.2%)
BlackRock Fund Advisors correctly lands here
5,325 (42%)
single-quarter filers and shell entities, excluded by design
218 (1.7%)
the only archetype that feeds the RACS signal
The original design narrative describes 4 archetypes: Index Huggers, Conviction Activists, Macro Tourists, and Nimble Traders. On this real run, the density structure of the actual manager population only supports 2, plus a large noise bucket. Only Conviction Activists ever reaches the RACS signal engine either way.
Regime-Conditioned Activist Conviction Score. Only the Conviction Activists archetype feeds this signal; the other clusters are computed for interpretability and structurally excluded from it.
managers WHERE archetype = 'Conviction Activists'A tiny result set, 218 real managers of 12,667 clustered, feeds everything downstream.
weight = position_value / manager_total_aumQuarterly portfolio weights per activist manager, equity positions only, positive value.
racs_raw = consensus_weight * ln(buyers + 1.1)Requires 2 or more distinct activist buyers per CUSIP-quarter. One activist alone is not a signal.
crowding_penalty = holders / total_managersA cheap, quarter-relative crowding proxy that discounts positions "everyone" already owns.
racs = raw * (1 - crowding) * (1 +/- weight * regime_prob)Sign flips to a discount under Rate_Shock/Recession_Fear, a boost under Goldilocks/Recovery, scaled by the HMM's own posterior confidence.
AlphaFactoryEngine, WalkForwardValidator, CapacityAnalyzer, SignalDecayAnalyzer, the PBO/Deflated-Sharpe implementations, MonteCarloTester, EvaluationGate, every one of these was real, working, individually tested code. Nothing in the CLI or the orchestrator ever called them together. That's the actual, root-cause reason the dashboard's backtest and validation pages showed pre-computed placeholders instead of a live run.
1$ grep -rn "backtest" andria/cli/main.py2(no matches)3 4# AlphaFactoryEngine, WalkForwardValidator, CapacityAnalyzer,5# SignalDecayAnalyzer, ProbabilityOfBacktestOverfitting,6# DeflatedSharpeRatio, MonteCarloTester, EvaluationGate --7# all real, all tested in isolation, zero orchestration8# wiring them together. This is the actual reason9# backtest.json and validation.json held pre-computed placeholders.A bandit-suppression comment landed inside a SQL string literal instead of before it, meaning RACSEngine.compute() had never successfully run against real data even once. Six headline bugs like this, all invisible to the existing synthetic-fixture test suite, only surfaced by actually touching real data end to end.
1$ andria run phase22...3racs_stage detail='computing raw RACS scores' stage=3/54Traceback (most recent call last):5 File "andria/signals/racs.py", line 111, in compute6 conn.execute(f""" # nosec B6087_duckdb.ParserException: Parser Error: syntax error at or near "#"8 9LINE 1: # nosec B60810 ^11 12# the bandit-suppression comment was placed INSIDE the SQL string13# literal, not before it. every real invocation of the signal14# engine crashed on this before this fix.1$ andria run phase32...3leakage_audit_passed checks_run=64backtest_completed overall_sharpe=-0.59375pbo_insufficient_data n=9 partitions=166dsr_insufficient_data n=97bootstrap_complete observed_sharpe=-0.594 p_value=0.8248evaluation_gate_rejected reasons=[9 'Provenance 0.82 below 0.90 threshold',10 'PBO score 1.00 exceeds 0.40 limit'11]12pipeline_phase3_complete gate_passed=FalseThe 1,341-line validation suite was never actually running
tests/validation/phase4_validation_suite.py held 106 tests covering the leakage audit, PBO, DSR, walk-forward, and drift detection, but pytest only auto-discovers files named test_*.py. This one wasn't. CI's "pytest tests/" had been silently running 9 tests, not 115, since the file was created.
Renamed to test_phase4_validation_suite.py. All 106 tests ran immediately and passed cleanly, the suite itself was solid, it just never executed.
The RACS signal engine could not run at all
A bandit-suppression comment, conn.execute(f""" # nosec B608, was placed after the opening triple-quote instead of before it, landing the literal text "# nosec B608" inside the SQL string itself. DuckDB received it as SQL and threw a ParserException on the very first real call to RACSEngine.compute().
Moved the comment outside the string (it was redundant anyway, CI already globally skips B608). Wrote a regression test that runs the engine end to end against a synthetic fixture, which would have caught this on day one.
Every regime label silently came back "Unknown"
RACS builds its own quarter key to join against regime data: the code's own comment says "edgar has source_quarter (e.g. 2021Q1)", then the line directly below built "2021_Q1", with an underscore EDGAR never uses. The join never matched, so every signal's regime_label defaulted to "Unknown", quietly breaking the entire regime-conditioning half of the signal's name.
Removed the underscore to match EDGAR's real format. A test asserting the real regime label appears in the output (not "Unknown") now guards this.
source_quarter was read from the wrong field
SEC's current bulk packaging groups filings by when they were received, not what they report on: one downloaded batch, "01sep2024-30nov2024", turned out to contain 9,144 genuine Q3-2024 filings and a long tail of late amendments reporting on quarters back to 2014, all of which the pipeline was labeling "2024Q3" because it read the quarter from the folder name instead of each filing's own REPORTCALENDARORQUARTER field.
Derived source_quarter from the real per-filing report date instead of the batch directory. Verified against a fixture reproducing the exact real-world case (a primary filing plus an old mislabeled amendment) before re-running ingestion on all 53 real files.
A documented data-quality gate was never wired in
ingest.min_valid_date = "2004-01-01" existed in config and in the README ("enforces a minimum valid date") but was never referenced anywhere in the ingestion code. Running against real data surfaced the consequence directly: filings dated "1900" and "1987", real typos in SEC's own raw data, were flowing straight through.
Added the WHERE filter the config always implied. Re-ingesting the real 120M-row dataset with it in place dropped exactly 364 malformed rows and left 89 clean, correctly-dated quarters.
OpenFIGI's real batch limit is 10, not 100
The CUSIP to ticker resolver (rebuilt in this pass to replace an 11-entry hardcoded list) assumed OpenFIGI's documented-elsewhere "100 identifiers per request." A real batch of 27 CUSIPs came back HTTP 413 "Request may only contain 10 mapping jobs", silently resolving 0 of 27 and collapsing real pricing coverage to 3.6%.
Verified the real limit directly against the live API, dropped the batch size to 10. CUSIP resolution recovered to 82.1% real coverage on the first re-run.
The full statistical-validation stack ran against the real backtest, not a stand-in. Where a method's own minimum sample size was not met, it returned an honest NaN or an explicit insufficient-data flag instead of a fabricated number.
Is the observed Sharpe a sampling artifact?
Does entry timing matter, or just general drift?
Is regime-conditional dispersion real or random?
significance threshold, alpha = 0.05, marked on each bar
| Regime | n | Return | Sharpe |
|---|---|---|---|
| Goldilocks Best-performing regime, on 3 trades | 3 | +8.6% | +0.88 |
| Recovery Majority of surviving trades landed here | 6 | −19.8% | −1.03 |
| Rate_Shock Signals generated, none survived to a priced trade | 0 | n/a | n/a |
| Recession_Fear Signals generated, none survived to a priced trade | 0 | n/a | n/a |
every fold requires 10+ trades, total is 9
below CSCV's minimum sample requirement
below the method's minimum sample requirement
8 of 9 trades survive regression prep, needs 10+
120.2M real filings in, all 12 fixes applied, real market pricing via OpenFIGI and yfinance: 144 RACS signals, 9 backtested trades, and an evaluation gate that ran the same rigorous checks it always would have and correctly rejected the run.
Real backtested trades
top-decile RACS signals with resolvable real pricing
Annualized Sharpe, real
bootstrap p=0.824, not statistically significant
Leakage audit, real
0 errors, 0 warnings across 86 priced signals
Signal IC half-life, real
Spearman IC vs. forward return, computed on real prices
What was chosen, why, what it cost, and what happened. Specific to this project, not a repeat of the portfolio's global decision log.
The pipeline is Polars-first everywhere else, but RACS is a 5-stage aggregation with multi-table joins and window functions where a subtle bug, like double-counting an activist's position across quarters, is easy to introduce.
ReasoningDuckDB's SQL makes a GROUP BY and its join keys directly readable and auditable in a way a chained functional Polars pipeline is not. For this specific bug class, SQL's auditability was judged more valuable than Polars' type safety.
The whole computation runs in one DuckDB connection, with activist_weights dropped immediately after raw_racs is derived from it to bound peak memory.
Every number above came from a real run, on real SEC filings, this session.