Dataset & feature reduction
Objective (i) result: how many of the original telemetry features the wrapper kept versus the two reference baselines.
–
Total samples
–
Wrapper-selected features
–
Feature-space reduction
Selected features (kept by the wrapper for training)
Feature-subset stability
Jaccard similarity of the selected subset across cross-validation folds, plus a paired t-test flagging any fold whose accuracy differs significantly from the rest.
–
Mean Jaccard similarity across folds
Per-fold significance (p < 0.05 = fold behaves differently)
| Fold | t-stat | p-value | Flag |
|---|
Classifier comparison (Objective ii)
Every classifier from Section 3.5, trained on the wrapper-selected feature subset and evaluated on the untouched 20% test split.
| Model | Accuracy | Precision | Recall | F1 (macro) | ROC-AUC |
|---|
Wrapper vs. baselines (Objective iii)
The best model re-trained on three different feature sets — the wrapper subset, the full feature set, and (for reference) the filter's feature count — compared on quality and computational cost.
| Feature set | # features | F1 (macro) | Train time (s) | Inference / sample (ms) | Peak memory (KB) |
|---|
Confusion matrix — best model
Per-class classification breakdown across the five threat categories, on the held-out test set.