Normalize, Clean, Resample, and Train (NCRT): An End-to-End Pipeline Study for Solar Energetic Particle Forecasting

Resource information

Format

zip

Original repository

22841609

DOI

10.5281/zenodo.22841609

Credit basis

Creators in Zenodo record metadata

Access

Open files available from the original Zenodo deposit

Last checked

Nov 5, 2026

License

mit-license

Last record update

Nov 5, 2026

About this resource

Abstract Solar energetic particle (SEP) events are rare, high-impact space weather phenomena that pose serious radiation hazards to astronauts and polar-route aircraft crews, damage satellite electronics, and disrupt satellite communications and navigation systems. Forecasting SEPs allows preventive measures to be taken before an event's onset. This task is difficult because SEP events, defined as periods when the flux of > 10 MeV protons exceeds 10 proton flux units (pfu), are extremely rare and make up less than 1% of the samples in our dataset. We present a stage-by-stage study of a novel preprocessing pipeline for SEP forecasting (normalization -> borderline data cleaning -> class rebalancing -> training) and evaluate it across both classical and state-of-the-art classifiers (SVM, GRU, PatchTST, InceptionTime). To address data scarcity, we compare classical interpolation-based over-sampling (SMOTE, ADASYN) against two generative approaches, TimeGAN and a state-of-the-art diffusion model, for synthesizing minority-class sequences from only 118 real SEP events. Using the True Skill Statistic (TSS), we find that our novel hybrid log/z-score normalization and Tomek Links-based borderline data cleaning yield the largest gains, while classical over-sampling methods consistently outperform both generative approaches in this data-scarce regime. Our best configuration named NCRT-GRU achieves a TSS of 0.671 ± 0.00 on a test set that mimics an operational setting and outperforms recently published SEP forecasters. How to Use the Notebooks Run the notebooks in numeric order — each stage's output is the input to the next: 0_Dependancies.ipynb — installs required packages. Run once. 1_DataPreparation.ipynb — builds the multivariate time series dataset from the raw per-channel CSVs in dataset/ , and produces the chronological 70/10/20 train/validation/test split used by every downstream experiment. 2_ImpactsOfNormalization.ipynb / 2.1_InitialClassification.ipynb — evaluate the four normalization schemes (Hybrid, Z-score, Min-Max, Log) and select classifier hyperparameters. 3_BorderlineCleaning.ipynb — applies Tomek Links, ENN, and NearMiss-3 to the best-normalized training set and compares results. 4_RandomUnderSampling.ipynb — evaluates RUS alone, at several retained-NSEP levels, on the cleaned training set. 5_RUS&OS-SMOTE.ipynb , 6_RUS&OS-TimeGAN.ipynb / 7_RUS&OS-TimeGAN-Results.ipynb , 8_RUS&OS-Adasyn.ipynb , 9_RUS&OS-Diffusion.ipynb / 10_RUS&OS-Diffusion-Results.ipynb — combine RUS with each of the four over-sampling methods (classical: SMOTE, ADASYN; generative: TimeGAN, diffusion) at every shared configuration, and report classification results. 11 – 19 ( *_Figures_*.ipynb ) — regenerate every figure in the paper from the saved experiment results. Run 11_Figures_Style.ipynb first in the same kernel session (it defines the shared color palette and result-loading helpers the later figure notebooks depend on), then run any of 12 – 19 as needed. Each notebook documents its own expected inputs/outputs in its first markdown cell. Repository Structure ├── dataset/ # Raw per-channel OMNI/GOES time series (gzipped CSVs) │ ├── F.csv.gz, Np.csv.gz, P4.csv.gz, P5.csv.gz, P6.csv.gz, │ │ Tp.csv.gz, V.csv.gz, Vx.csv.gz, Xl.csv.gz, Xs.csv.gz │ ├── masked-ts-diffusion/ # Causal-attention DDPM implementation used for │ ├── model.py # generative over-sampling │ ├── api.py │ └── __init__.py │ ├── timegan.py # TimeGAN implementation used for generative over-samp