Not every layer deserves the same bits.
A genetic algorithm searches 3.1×10^10 layer-wise quantization configs for a real LLM, then checks its own cheap proxy against real GPU hardware instead of assuming it works: r=0.940.
Repeating 4/8/16-bit pattern, discovered by the search, applied via real bitsandbytes kernels.
Candidate configuration space
22 layers, 3 bit-width choices each
Proxy-to-hardware correlation
predicted vs. measured loss, 5-point DoE
Test suite, real CUDA re-run
up from 39, all GPU-gated tests included
Weight-storage reduction
evolved genome vs. FP16, 2-3x lower latency
Sensitivity measurements
22 layers x 4 bit-widths, the proxy's only training data
New NSGA-II tests added
dominance, multi-front sort, crowding, end-to-end
Dataset split integrity
disjoint calibration/val/test, overlap raises an error
Checkpoint resume guarantee
a drifted config cannot silently resume
Uniform quantization is a bet every layer loses the same way.
22 layers, 3 valid bit-widths each, and a proxy cheap enough to search fast but not automatically trustworthy: three problems this project had to actually solve, not just state.
A search space too large to hand-search
22 independently-quantizable transformer layers, 3 valid bit-width choices per layer (4/8/16-bit: the set the real backend actually supports), gives 3^22 ≈ 3.1×10^10 candidate architectures. Small enough to state, far too large for grid search.
Layers are not uniformly compressible
The project's own sensitivity profile shows a 23x spread at 2-bit: ~0.019 nats of Delta-loss on layer 11 (tolerant) versus ~0.45 nats on layer 2 (load-bearing). Uniform quantization cannot exploit this structure; a per-layer search can.
A cheap proxy makes the search fast, and unverified by default
Scoring every genome with a real forward pass would make the search itself GPU-bound. An additive proxy solves that, but its ranking of candidates matching real hardware is then an assumption running through the whole codebase, not a fact, until someone actually checks it.
The proxy is only as honest as the data it was measured from.
A dataset protocol with SHA-256 fingerprinting and explicit overlap rejection, an opt-in-only synthetic-data escape hatch, and the two artifacts that make the whole project checkable: the sensitivity profile the proxy is built from, and the calibration report that checks it.
44,040,192 quantizable (Linear-module) parameters per layer, verified as constant across all 22 layers directly from the sensitivity profile JSON.
Tokenized into one long token stream, then chunked into fixed-length samples at random offsets.
Calibration (train), validation (seed 43), test (seed 44). Each split SHA-256 fingerprinted over (sample_id, token_bytes); overlap across splits raises DatasetLoadError. Default: 4 samples x 128 tokens per split, sized for fast iteration, not statistical power.
Generates random token tensors instead of downloading WikiText when explicitly enabled. Off by default, and a dedicated test enforces that silent fallback to fake data cannot happen by accident.
Produced once by scripts/profile_sensitivity.py: fake-quantizes one layer at a time, measures Delta-loss vs. the FP16 baseline (3.3008 nats) on the calibration split, restores the original weights. The proxy's entire "training data."
uniform_fp16, uniform_int8, uniform_int4, alternating_4_8, alternating_8_16, each with both the proxy's predicted quality-loss delta and the actually-measured loss, perplexity, latency, peak memory, and throughput on an RTX 4070.
A hard boundary between cheap search and real execution.
A swappable QuantizationBackend Protocol is what let the real GPU backend (Phase 4) replace numerical simulation without touching the search engine at all.
$ python -m empas.search conf/config.yaml algorithm=nsga2 search_space=mixed_precision
WikiText-2 Corpus
Tokenized into one long stream, chunked into fixed-length samples at random offsets.
Dataset Protocol
3 disjoint, SHA-256 fingerprinted splits. Overlap between them raises an error at load time.
Sensitivity Profiler
One-time pass: 22 layers x 4 bit-widths, fake-quantize one layer, measure Delta-loss, restore weights.
Search Space
22 genes, choices {4,8,16}, min_avg_bitwidth 2.5 floor. 3^22 ≈ 3.1×10^10 candidate genomes.
GA Engine, real NSGA-II
Crowded tournament select, uniform crossover, per-gene mutation, mu+lambda combine, sort, trim, every generation.
Pareto Archetypes
A 50-generation run exports max_accuracy / balanced / max_compression genomes to deployment artifacts.
Validate and Serve
Validator + BitsAndBytesBackend apply the genome on real GPU hardware; FastAPI serving loads balanced.json.
Two tracks, and a predicted number can never become a measured one.
A cheap additive proxy drives the search; a separate Validator applies the winners on real GPU hardware. The split is enforced structurally, not just by convention.
1# src/core/proxy_evaluator.py :: ProxyEvaluator2# no model forward pass required for any of the three3 4predicted_loss = baseline_loss + sum(5 sensitivity[layer_i][chosen_bits_i]6 for layer_i in range(22)7)8 9predicted_vram_mb = sum(10 layer_params_i * bits_i / 811 for layer_i in range(22)12) / 1e6 + 1024 # fixed context overhead constant13 14predicted_latency = (sum(bits) / max_possible_bits) * 10015# a unitless bandwidth score, not a millisecond estimateA 23x spread across layers of the same model.
88 measurements: 22 layers x 4 candidate bit-widths, each a fake-quantize-and-restore pass against the calibration split. This is the proxy's entire training data, and it shows exactly why per-layer search beats a fixed heuristic.
Same model, same bit-width, two layers: one nearly free to compress aggressively, one that destroys quality if you do. A uniform quantization policy cannot see this difference; the search does.
Per-layer quantization sensitivity, all 88 measurements
Log-scale Delta-loss vs. FP16, 22 layers x 4 bit-widths. Sensitivity is low and even at 4/8-bit almost everywhere, then sharply higher and unevenly distributed at 2-bit: the direct, data-backed reason 2-bit was excluded from the search space.
Branded NSGA-II, then actually rewritten to be NSGA-II.
An audit found every config file and the README naming the algorithm NSGA-II while crowding distance existed nowhere in the codebase. The fix pass closed that gap with a genuine rewrite, not a rename.
- get_pareto_front() was a naive O(N^2) single-front extraction: one non-dominated set, not a ranked partition into fronts.
- Crowding distance did not exist anywhere in the codebase, confirmed by grepping the entire source tree.
- Selection was a plain tournament on raw pairwise dominance with a coin-flip tiebreak.
- Next generation: keep up to pop_size/2 shuffled elites from the single front, fill the rest via tournament + crossover + mutation.
- Every config file and the README named this NSGA-II. It was an elitist, Pareto-aware GA, but not NSGA-II by Deb et al.'s (2002) definition.
- fast_nondominated_sort(): the real algorithm, partitions the population into all ranked fronts using domination counts and dominated sets.
- crowding_distance(): per-front diversity; boundary individuals get infinite distance, interior ones accumulate normalized neighbor gaps.
- crowded_comparison_key(): lower front rank wins; ties go to the individual with larger crowding distance (less crowded, more diverse).
- ga.py::step(): breed offspring via crowded tournament + crossover + mutation, combine parents+offspring (mu+lambda), re-sort the combined 2N population, fill front-by-front, trim the last front by crowding distance.
- 17 new tests (tests/test_phase3.py): dominance semantics, multi-front sorting, crowding boundary/tie behavior, both operators, end-to-end GAEngine behavior.
One observable behavioral consequence: generation stats are now computed over the combined 2N parent+offspring population, not just the N parents, so the reported population average is noticeably less monotonic generation-to-generation. True NSGA-II optimizes for front quality and spread, not a smoothly decreasing average, and that shows up directly in the data rather than being taken on faith.
Convergence is fast. What happens after is the real story.
50 generations, pop_size 20, re-run with the corrected algorithm. Best-seen loss settles by generation 20 to 30; the Pareto front size then oscillates around pop_size rather than growing unbounded, real NSGA-II environmental selection at work, not a bug.
1
Best (min) loss
3.3157
Combined-pop avg loss
3.3459
Best (min) VRAM (MB)
1822.0
Pareto front size
5
10
Best (min) loss
3.3057
Combined-pop avg loss
3.3262
Best (min) VRAM (MB)
1591.0
Pareto front size
12
20
Best (min) loss
3.3041
Combined-pop avg loss
3.3266
Best (min) VRAM (MB)
1591.0
Pareto front size
20
30
Best (min) loss
3.3030
Combined-pop avg loss
3.3206
Best (min) VRAM (MB)
1570.0
Pareto front size
21
40
Best (min) loss
3.3023
Combined-pop avg loss
3.3160
Best (min) VRAM (MB)
1570.0
Pareto front size
19
50
Best (min) loss
3.3022
Combined-pop avg loss
3.3206
Best (min) VRAM (MB)
1549.0
Pareto front size
21
50-generation convergence, real NSGA-II
Best-seen loss converges fast (gen 1 to 20), then flattens. The Pareto front size (right panel) now oscillates around pop_size=20 rather than growing unboundedly: real NSGA-II environmental selection actively managing diversity, not a bug.
Three archetypes, one Pareto front, all deployable.
max_accuracy, balanced, and max_compression are exported directly from generation-50's real rank-0 Pareto front. Every gene in every archetype now falls in {4,8,16}: the real backend can load all three.
Generation-50 population: loss vs. predicted VRAM
The full population scattered against the real 20-genome rank-0 Pareto front (via fast_nondominated_sort), with the three regenerated archetypes marked: max_accuracy, balanced, max_compression.
Archetypes vs. FP16 baseline
Predicted loss, VRAM, and average bit-width for the three regenerated Pareto archetypes. The loss spread is now much tighter (3.30 to 3.35) than the pre-fix artifacts (3.30 to 4.55): crowding-distance selection keeps the front denser near the low-loss end.
Predicted loss
3.3008
Predicted VRAM
2872 MB
Avg bit-width
16.00
Predicted loss
3.3022
Predicted VRAM
2074 MB
Avg bit-width
9.09
Predicted loss
3.3136
Predicted VRAM
1696 MB
Avg bit-width
5.82
Predicted loss
3.3476
Predicted VRAM
1549 MB
Avg bit-width
4.55
The proxy, finally checked against real hardware.
5 designs, each a fresh model load plus real bitsandbytes quantization plus real validation forward passes on an RTX 4070: the first time this repository's central bet has ever been measured instead of assumed.
Loss-axis correlation
the number that validates the whole search strategy
Memory-axis correlation
expected sign: more predicted loss means fewer bits means less memory
Latency-axis correlation
weak, reflecting single-run timing noise, not a proxy failure
uniform_fp16
Proxy predicted Delta-loss
0.0000
Measured loss
2.5714
Measured latency
486.1 ms
Measured peak memory
2149 MB
uniform_int8
Proxy predicted Delta-loss
0.0030
Measured loss
2.5856
Measured latency
52.8 ms
Measured peak memory
1230 MB
uniform_int4
Proxy predicted Delta-loss
0.1087
Measured loss
2.6364
Measured latency
79.2 ms
Measured peak memory
793 MB
alternating_4_8
Proxy predicted Delta-loss
0.0373
Measured loss
2.6164
Measured latency
76.4 ms
Measured peak memory
1437 MB
alternating_8_16
Proxy predicted Delta-loss
0.0014
Measured loss
2.5774
Measured latency
28.7 ms
Measured peak memory
2117 MB
Proxy calibration: predicted vs. measured, r = 0.940
The single most important new plot this project has: 5 real-hardware measurements checking the proxy against reality instead of just running the proxy. The trend line is fitted, not assumed.
Five gaps, found by auditing, closed on real CUDA hardware.
An audit-then-fix process: inspect the repository as it stood, verify every claim, flag real gaps, then go back in and close all of them in the same environment used for verification.
1$ pytest tests/ -v2...339 passed in 12.4s4 5$ grep -rn "crowding_distance" src/6(no matches)7 8# the evolutionary engine is branded "NSGA-II" in every9# config file and the README. grepping the entire source10# tree finds no crowding-distance implementation anywhere.1$ pytest tests/ -v2...356 passed in 18.9s (+17 new: tests/test_phase3.py)4 5$ python scripts/benchmark.py --doe6uniform_fp16 predicted_dloss=0.0000 measured_loss=2.57147uniform_int4 predicted_dloss=0.1087 measured_loss=2.63648pearson_r(loss) = 0.9409calibration_report_saved -> data/calibration/tinyllama_calibration.jsonsrc/hardware/profiler.py was dead scaffolding
An empty file the README described as doing real GPU device-introspection work (get_hardware_identity, current_memory_snapshot); benchmark/harness.py had its own duplicate device-introspection logic instead.
Implemented profiler.py for real. benchmark/harness.py::get_hardware_identity() now delegates to it as the single source of truth, matching what the README's project-structure table always claimed it did.
The evolutionary engine had zero dedicated tests
The algorithm doing the project's actual core work, search plus Pareto selection, was the one major component with no unit tests at all, confirmed by direct inspection of tests/.
Added tests/test_phase3.py: 17 tests covering dominance semantics, multi-front sorting (including a hand-constructed 3-front example and an "everyone non-dominated" edge case), crowding-distance boundary/tie behavior, both operators, and end-to-end GAEngine behavior.
An evolutionary engine branded "NSGA-II" that was not
get_pareto_front() was a naive O(N^2) single-front extraction, not a ranked multi-front partition; crowding distance did not exist anywhere in the codebase, despite every config file and the README naming the algorithm NSGA-II.
Implemented genuine fast_nondominated_sort and crowding_distance in src/engine/pareto.py, and rewrote src/engine/ga.py::step() around crowded tournament selection and mu+lambda environmental selection: standard NSGA-II, not an approximation of it.
Stale deployment artifacts held bit-widths the real backend rejects
deployment/artifacts/*.json (max_accuracy/balanced/max_compression) were exported from an earlier 4-choice [2,4,8,16] search space, but the real BitsAndBytesBackend only supports {4,8,16}-bit: the 2-bit genes in those artifacts were not loadable by the current backend.
Re-ran the 50-generation search with the corrected NSGA-II algorithm and regenerated all three artifact JSONs from it. Every gene now falls in {4,8,16}, verified programmatically against BitsAndBytesBackend.supported_precisions.
The calibration pipeline was built, unit-tested, and never run
16 passing unit tests existed for the Phase 6 proxy-calibration machinery (the ProxyMetrics/MeasuredMetrics separation, the correlation function, the design-of-experiments protocol), but scripts/benchmark.py was still log-line stubs: none of it had ever touched real measured data.
Completed scripts/benchmark.py to actually execute the 5-design DoE on a real RTX 4070, and saved the repository's first real calibration report, data/calibration/tinyllama_calibration.json: r=0.940 between predicted and measured loss.
A genuinely positive result, reported with the same rigor as the gap next to it.
Measured on real GPU hardware, bitsandbytes backend, TinyLlama-1.1B, RTX 4070 Laptop GPU: the evolved genome against the FP16 baseline it was evolved to beat.
Real-hardware calibration
first empirical check this repo has ever run against reality
Real GPU loss delta
~1.5% relative, evolved genome vs. FP16, real bitsandbytes hardware
Real GPU latency
evolved genome vs. FP16 baseline, single-run measurement
Weight-storage reduction
avg_bits ~ 9.09 vs. FP16's 16.0
FP16 baseline (all layers, 16-bit)
Mean loss
2.5714
Perplexity
13.08
Latency
~150-315 ms
Evolved genome, [4,8,16]-repeating mix
Mean loss
2.6097
Perplexity
13.60
Latency
~60-155 ms
From benchmark_vs_baseline.py, independent of the search re-run (hardcoded genome, numerical fake-quantization, so VRAM is identical across all three rows: this table shows the quality trade-off, not a memory reduction).
Six calls from this project.
What was chosen, why, what it cost, and what happened. Specific to this project, not a repeat of the portfolio's global decision log.
Evaluating every genome in every generation with a real forward pass would make the search itself GPU-bound, orders of magnitude slower, for no benefit during exploration.
ReasoningThe 88-measurement sensitivity profile is computed once and then reused for the entire 50-generation search: each fitness evaluation becomes a handful of dict lookups and arithmetic, not a GPU forward pass.
Now empirically justified, not just assumed: real-hardware calibration shows r=0.940 loss correlation across the 5-point DoE.
What this search doesn't prove.
- ·Predicted VRAM sits ~30-35% high versus measured peak memory at this sequence length and batch size (2872MB predicted vs. 2149MB measured for uniform_fp16), most plausibly the proxy's fixed 1024MB context-overhead constant being tuned for a larger usage profile than this project's small validation protocol.
- ·Dataset splits are deliberately small (4 samples x 128 tokens per protocol split): fast enough to profile 22 layers x 4 bit-widths and run a 50-generation search interactively, not large enough to claim statistically robust perplexity estimates.
- ·Real-GPU loss and perplexity numbers are reproducible; latency figures are single runs without a synchronized, warmed-up protocol (the FP16 calibration design's own cold-start 486ms is the clearest example) and should be read as indicative only.
- ·The FastAPI serving path is true at the mechanism level, it does load an exported archetype and apply it via the real backend, but it was never benchmarked as its own comparison table: not a production-hardened deployment claim.
- ·The 5-point calibration DoE is the first empirical check this repository has ever run against real hardware, not a large, statistically powered study.
- ·Phases 7 to 14 of the original 14-phase plan (for example, a systematically measured Pareto frontier across many baselines) were never started, stated plainly in the README rather than silently declared done.
An honest gap is often just execution away from closed.
- Shipping the tooling for validation is not the same as shipping the validation, and the gap is closeable: completing an already-correct calibration pipeline and running it for real took a few hours of GPU time, not a redesign.
- A real execution backend changes what "valid" means, and that ripples backward into earlier artifacts, not just code: the 2-bit removal from the search space was a one-line diff, but its full consequence, stale exported archetypes with ungenerateable genes, was not actually resolved until this fix pass regenerated them.
- An algorithm's own name is a claim: grepping the entire source tree for crowding_distance and finding nothing is what separates "branded NSGA-II" from NSGA-II, and a correlation number can be good news too, reported with the same rigor as the VRAM overestimate it sits next to.
A search that got faster, a proxy that got checked, and both numbers reported as measured.