Self-Supervised Complex-Wavelet Learning and Statistically Governed Transfer
Scientific Report 3: The V0–V7 Pretraining Ladder, the Transfer Gap, and Phase A

Study ID R3. Incoming state: the deterministic bottleneck of Report 2. Changed factors: self-supervised data/task scale (V0–V7), then the transfer protocol itself (Phase A). Evidence tiers: E1-TRAIN pretext metrics; E3-VAL transfer comparisons — early ones exploratory, Phase A under the new bootstrap governance. Outgoing protocol (frozen): decoder_last_block; fresh/reused learning rates $10^{-4}/5\times10^{-6}$; validation once per completed epoch; unweighted residual_skill_v0; guarded best_skill.pt selection with Niño-3.4 correlation (leads 3–14) primary. Timeline: 2026-07-01 through 2026-07-03; test never accessed.

Report 2 located the deterministic bottleneck in the objective/data regime: 335 supervised windows cannot teach a 1.4M-parameter decoder boundary what evolving tropical-Pacific anomalies look like. Masked spatiotemporal coefficient learning is the natural response — reconstruction tasks manufacture supervision from unlabeled sequences, and historical SST reconstructions extend the record by more than a century. Two questions structure the study:

1. Do distinct historical SST reconstructions improve multiscale representation quality, measured by masked-coefficient and physical-space reconstruction diagnostics on fixed holdouts?
2. Does that quality transfer to phase/amplitude forecasting under a controlled supervised fine-tune?

The answers turn out to be “yes, consistently” and “no, not reliably” — and the divergence between them is the report's central scientific object.

Masked autoencoding is the dominant self-supervised recipe for vision and video [1, 2]: hide most of the input, reconstruct it, and fine-tune the encoder. Its transfer success is typically reported for semantic downstream tasks; whether reconstruction quality transfers to dynamical extrapolation — forecasting — is far less established, and our results bear directly on that question. In climate ML, pretraining has helped when the pretext matched the downstream task: CMIP-pretrained CNNs for ENSO indices [3] and climate-simulation-pretrained ResNets for medium-range fields [4] both pretrain on (approximately) the forecast task itself. Our ladder isolates the complementary case: same architecture, same data modality, but a reconstruction pretext.

A caveat governs all multi-product claims: ERSSTv5 [5], HadISST1 [6], and COBE-SST2 [7] are distinct reconstructions built from substantially shared observations (ICOADS-era ship and buoy records). They diversify reconstruction methodology, gridding, and noise structure — not the set of ENSO events. Store counts below are therefore never read as independent sample sizes.

For every transfer comparison: tropical Pacific $30^{\circ}$S–$30^{\circ}$N, $120^{\circ}$E–$280^{\circ}$E on the $24\times48$ grid; 12-month context, 14-month target; observed train 1981–2010; observed validation 2011–2015 (35 windows); 2016–2020 reserved and untouched; cosine-weighted Niño-3.4 box with the leads-3–14 correlation summary; persistence and zero-anomaly baselines. Ensemble size: not applicable. Pretext stores extend earlier than 1981 only (next section).

The pretext corpus is: ERA5 1981–2010 training months (never validation/test); ERSSTv5 1854-01–1980-12 (1,524 months, 1951–1980 climatology); HadISST1 1870-01–1980-12 (1,332 months, same climatology interval); COBE-SST2 1850-01–1980-12 (1,572 months). Every external store is declared pretrain_only, is rejected by supervised loaders, and fails closed on any 2011–2020 overlap. Each store carries source-local per-channel normalization: ERSST's $2^{\circ}$ sampling has a different highpass variance profile from ERA5, and applying ERA5 statistics produced an immediately rejected coefficient-scale diagnostic. Manifests record spans, month counts, coefficient/normalization hashes, sealed-period overlap flags, allowed_use, and test_data_read (Table 1).

Sampling modes are proportional, exact-quota balanced, and weighted (largest-remainder); realized per-store batch counts are recorded in checkpoints (e.g. V6: 732/756/732/780 for ERA5/ERSST/HadISST/COBE). Because stores are normalized separately, pooled normalized losses are explicitly marked not strictly comparable across stores; cross-store discussion prioritizes inverse-normalized physical-space RMSE (X2) — field, equatorial band, and Niño-3.4 — which live in common SST-anomaly units.

The pretext model is the same 177-state coefficient-native Earthformer used for forecasting, run at 12-month geometry. Corruption operators are: element masks, whole-month temporal masks, and space–time block masks, plus two-month middle/future gaps in later rungs. With mask set $\mathcal{M}$ and normalized coefficients $z$, the base objective is hidden-only reconstruction,

with validity-masked cells excluded. Physical diagnostics (X2) inverse-normalize and inverse-DTCWT merged predictions to score reconstruction RMSE for the field, equatorial band, and Niño-3.4. Train/holdout sequence partitions are deterministic; a timestamp audit immediately before array materialization rejects any month after 2010-12. Checkpoints round-trip all 177 states exactly; supervised initialization loads 172 tensors with exactly five declared 12-to-14-month temporal-geometry skips, and residual persistence equivalence after zero-output initialization is verified to be exact.

Each rung changes one factor and exact-loads its predecessor:

V7 keeps the masked objective and adds three physical-space regional terms computed only for hidden target months: predicted values at masked locations are merged with visible targets, inverse-normalized with the store's own statistics, inverse-DTCWT'd, and reduced to cosine-weighted indices. With hidden-month index $N_t$, equatorial index $E_t$, and monthly tendency $\Delta N_t = N_t - N_{t-1}$,

The targets remain self-supervised functions of the unlabeled SST sequence; no forecast target, validation month, or supervised optimization enters. The state schema is unchanged, so V7 checkpoints remain compatible with the supervised initialization surgery.

The ladder improves nearly monotonically on its fixed holdouts. Selected milestones (pooled unless stated): V1, masked MSE $1.0138\to0.7352$, field reconstruction RMSE $0.1310\to0.0958$; V2, hidden MSE $0.7774\to0.7510$ with full/equatorial/Niño-3.4 RMSE $0.1216/0.1680/0.1627\to0.1174/0.1578/0.1544$; V4 (balanced two-store), pooled hidden $0.7553\to0.7375$ with both stores improving individually; V5 (three-store), pooled hidden $0.6822\to0.6686$, with ERA5 and HadISST improving on all seven tracked diagnostics but small ERSST physical/regional regressions — the one non-monotone rung; V6 (four-store), pooled hidden $0.6990\to0.6693$, field $0.1952\to0.1887$, Niño-3.4 reconstruction $0.2648\to0.2444$, with all seven diagnostics improving for every store and the V5 ERSST regression not persisting. V7 then improves everything its auxiliary targets: field $0.1887\to0.1796$, equatorial $0.2278\to0.2040$, Niño-3.4 reconstruction $0.2444\to0.2063$, index MSE $0.0460\to0.0294$, tendency MSE $0.0378\to0.0271$, tendency correlation $0.8016\to0.8524$ — all ten tracked diagnostics, every store, no cross-source regression. These are representation diagnostics, not forecast skill.

Three pretrained initializations receive an identical bounded supervised fine-tune (coefficient-native, persistence-residual, decoder_last_block, unweighted residual_skill_v0, 1,000 CUDA steps, guarded selection; all selected step 335):

The decisive contrast is V6 versus V7 (Fig. 1): the auxiliary tasks improved exactly the pretext quantities that look forecast-relevant (hidden-month index and tendency reconstruction) and still worsened transfer — Niño-3.4 correlation $0.1267\to0.0380$, ACC $0.2876\to0.2768$, RMSE $0.5432\to0.5450$. Cancellation eased marginally under V7 (cosine $-0.851\to-0.837$; opposing energy $96.2\to95.6\%$) while amplitude stayed damped and 2015 worsened. Reconstruction quality, even regionally targeted, is not the transfer bottleneck.

Table 2's deltas are small — $0.002\,{}^{\circ}\mathrm{C}$ of RMSE, a few points of correlation — and the study's early rungs were being compared without uncertainty. This forced the governance reform that the rest of the project runs on:

• Append-only consultation ledger. Every observed-validation family comparison is entered before results are inspected, declaring candidate, fixed reference, primary metric, guards, practical minimum effects, bootstrap settings, decision branches, checkpoint-selection source, and the consultations remaining in the phase (budgeted at two per phase for Phases A–D).
• Paired moving-block bootstrap. For metrics recomputed from the same ordered 35 windows in both runs: blocks of 6 consecutive windows, 10,000 resamples, 90% percentile intervals, seed 20260703, identical resampled indices applied to both runs. Aggregate summaries are never expanded into synthetic samples; a missing per-window artifact is reported as missing.
• Decision language. Resolvable improvement requires the CI to exclude zero favorably and the predeclared minimum effect; an interval containing zero is unresolved, never a win or loss. Mixed primary/guard outcomes follow the predeclared branch.
• Selection separation. Checkpoint selection moves off 2011–2015 for Phases B–D (held-out simulations or a frozen training-tail holdout); 2011–2015 is reserved for predeclared family-level comparisons.
• Case-study boundary. The 2015 event may be reported and rendered, but its numbers are never selectors, family criteria, stop rules, or minimum effects.

A retrospective decision-resolvability audit re-examined the ladder's earlier choices under paired bootstrap where artifacts permitted, distinguishing decisions the data actually resolved from those made on unresolved deltas — the direct motivation for Phase A's predeclared design.

Phase A selects a reusable transfer protocol for Phases B–C — not a forecast model — through two predeclared consultation batches under the §9 machinery. Primary: Niño-3.4 correlation, leads 3–14; guards: pooled RMSE and mean-window field ACC; adoption thresholds: primary CI $>0$ with delta $\ge +0.02$, no resolvable RMSE regression $\ge +0.005\,{}^{\circ}\mathrm{C}$, no resolvable ACC regression $\le -0.01$ (Fig. 2).

Candidates make all 166 optimizable parameters trainable (4 fresh at $10^{-4}$; 162 reused, including 87 encoder parameters, at $10^{-6}$ or $10^{-5}$) against the fixed V7 decoder_last_block reference. Outcomes: lr $10^{-6}$ (selected step 1675) — RMSE $0.5756$, primary delta $+0.219$ $[-0.077,+0.797]$ unresolved, RMSE delta $+0.0306$ $[+0.0174,+0.0430]$ resolvably worse; lr $10^{-5}$ (step 670) — primary $-0.030$ $[-0.388,+0.708]$ unresolved, RMSE $+0.0323$ $[+0.0165,+0.0415]$ resolvably worse. Neither meets the rule; the smaller boundary is retained and the conditional V6 replication branch is not triggered.

Report 2's repeated step-335 selections (the first epoch boundary) suggested the fine-tune saturates in one epoch. A2 tests every-84-step validation against epoch cadence on V6. The finer cadence does find a pre-335 within-run peak (selected step 84: RMSE $0.5458$, ACC $+0.2897$, Niño-3.4 $+0.2330$), refining saturation as partly an observation-cadence artifact — but at the family level the primary is unresolved ($+0.106$ $[-0.141,+0.218]$) and the ACC guard resolvably regresses ($-0.0260$ $[-0.0483,-0.0156]$). Epoch cadence is retained.

A training-only audit reconstructs physical maps from isolated coefficient groups (zeroing all others; isolated shares do not allocate cross-group DTCWT covariance). Normalized isolated Niño-3.4-relevant variance shares for lowpass/highpass-1/2/3 are $92.02/0.00/0.42/7.56\%$, while mean normalized coefficient SSE shares across selected V6/V7 checkpoints are $1.81/76.64/17.70/3.84\%$: the highpasses carry $\mathbf{7.98\%}$ of isolated index-relevant variance and $\mathbf{98.19\%}$ of the loss. This triggered the one predeclared scale-weighted run (V7, weighted coefficient MSE): RMSE improved resolvably ($-0.0037$ $[-0.0053,-0.0004]$) but the primary did not resolve ($+0.023$ $[-0.080,+0.127]$), so the unweighted loss remains frozen. The audit's significance outlives Phase A: the same loss-allocation imbalance, remeasured on correction temporal structure, becomes the causal lever of Phase G (Report 8).

Pretraining (E1-TRAIN): X1 masked-coefficient and X2 physical/regional reconstruction diagnostics by store, as summarized above; per-store values live in the run summaries and cross-source diagnostic reports. Transfers (E3-VAL): Table 2 carries D1–D6; Phase A adds paired deltas and CIs (Fig. 2). X4 provenance: V6 checkpoint SHA-256 5a81ab7a…, V7 16b4a801…; every transfer audit records 172 loaded / 5 skipped tensors, the 53/223,621-parameter trainable boundary, 113/1,193,780 frozen, exact residual-persistence equivalence, and test_data_read: false.

Why does reconstruction fail to transfer while its diagnostics improve? Four mechanisms are consistent with the record. (i) Objective mismatch: masked reconstruction learns a conditional interpolant of hidden months given visible ones; forecasting requires a conditional extrapolant 14 months forward. The interpolant's sufficient statistics need not contain the extrapolant's. (ii) Fine-scale gradient dominance: with per-channel normalization, $98\%$ of pretext gradient serves bands carrying $8\%$ of index-relevant variance (A3) — representation capacity is spent where the downstream task needs it least. (iii) Normalization and source heterogeneity: store-local statistics preserve each product's variance profile, but the supervised task lives in ERA5's; cross-store averaging can pull shared features toward reconstruction-generic solutions. (iv) Short fine-tuning: 335 windows, one effective epoch of adaptation — but A1/A2 show that opening the backbone or refining cadence does not release the constraint. V7 sharpens the puzzle: a correct local task (hidden-month index and tendency) still fails, because reconstructing a hidden month among visible neighbors is far easier than predicting it from the past alone. The decisive next intervention is supervised forecast-task pretraining at simulation scale — same architecture, same residual target, abundant trajectories — which Report 4 executes.

Shared observational ancestry across SST products; distinct per-store normalization limiting pooled-loss comparability; unequal effective event counts across stores; a single observed validation slice (35 overlapping windows) consulted repeatedly; adaptive rung-to-rung choices in the ladder (V0–V6 predate the ledger's bootstrap discipline); no random-seed replication of pretraining runs; and bootstrap governance covering transfer comparisons only — pretext metric uncertainty is not quantified. The 2016–2020 split is untouched throughout.

The frozen Phase A protocol (Table 3) — decoder_last_block, $10^{-4}/5\times10^{-6}$, epoch cadence, unweighted residual_skill_v0, guarded best-skill selection — is this study's durable product, along with a negative lesson now measured three ways: pretext reconstruction quality, even with regional phase/amplitude auxiliaries, does not become observed forecast skill in this regime. Report 4 inherits the coefficient-native checkpoint bridge, the persistent scorecard, the held-out-selection principle, and the explicit hypothesis that training on the actual 12-to-14-month forecast task across CMIP6 models will transfer where masked reconstruction did not.