Sparse supervised boosting

allboost() is the componentwise L2 boosting routine that fits the BAE encoder, usable on its own, with no autoencoder and no AnnData. It is pure NumPy and needs only the core install.

import numpy as np
from structboost import allboost

# targets should be standardized, predictors need not be
targets_std = (targets - targets.mean(axis=0)) / targets.std(axis=0)

betamat = allboost(
    sourcemat,        # (n_samples, n_features)
    targets_std,      # (n_samples, n_targets)
    stepno=20,
    nu=0.1,
    csf=0.9,
    independent=True,
)
# betamat: (n_targets, n_features) -- note the orientation

A natural single-cell use is regressing cluster indicator columns on genes, which gives a sparse marker signature per cluster directly.

How it relates to the BAE

It is the encoder fitter. Every iteration of fit() calls it with sourcemat = [X | nuisance] and targetmat = z*, and the config maps one-to-one:

BAEConfig

allboost

boosting_stepno

stepno

boosting_nu

nu

boosting_csf

csf

boosting_independent

independent

mandatory_genes

mandatory_features

prior_mode="anchored"

beta_init

So everything in How the BAE works about sparsity applies here too: stepno caps how many distinct features can enter.

The selection criterion

Each step picks the feature maximizing penalized variance reduction, (xⱼ'r)² / (‖xⱼ‖² + penaltyⱼ). This is scale-invariant in the predictor columns and does not systematically favour low-norm features, which is why predictor standardization is recommended but not required. The algorithm uses actual column norms.

That the selection is unbiased rests on the penalty, not on the criterion alone. Boosting selects without bias when its base-learners are comparable in degrees of freedom [1], and the initial penalty penaltyⱼ = ‖xⱼ‖²(1/nu − 1) makes the effective ridge degrees of freedom equal to nu for every feature, whatever its column norm. csf then moves features off that common footing on purpose, that is the diversity mechanism, not a bias.

csf adapts the per-feature learning rate after each selection: nu_j ← 1 - (1 - nu_j)^csf. Below 1 it promotes diversity. Above 1 it reinforces already-selected features. Adapting per-feature penalties across boosting steps is the mechanism Binder & Schumacher [2] use to fold external biological knowledge into the fit: pathway membership in their case, mandatory markers or a chosen csf here.

Forcing features in

betamat = allboost(
    sourcemat,
    targets_std,
    stepno=20,
    mandatory_features=np.array([0, 5, 12], dtype=np.intp),
)

Mandatory features enter an unpenalized adjustment block via a joint OLS pre-step, the mandatory-covariate mechanism of Binder & Schumacher [3], so they escape competitive selection. As with the BAE, this does not guarantee a non-zero coefficient. If the contribution is estimated as zero, the coefficient is zero.

Warning

Boolean masks and float index arrays are rejected outright. A mask silently reinterpreted as the indices 0/1 would select the wrong features. Use np.flatnonzero(mask).

Inspecting the path

betamat, hist = allboost(sourcemat, targets, stepno=200, return_history=True)

hist.selection   # (n_targets, stepno) which feature was chosen each step
hist.beta_path   # (n_targets, stepno, n_features) -- can be memory-intensive

from structboost import plot_boosting_coefficient_paths

plot_boosting_coefficient_paths(hist.beta_path, gene_names=feature_names)

Reusing the covariance cache

Repeated fits on the same sourcemat (cross-validation, parameter sweeps) can share the predictor Gram matrix:

betamat, covcache = allboost(sourcemat, targets, stepno=20, return_covcache=True)
betamat2 = allboost(sourcemat, other_targets, stepno=20, covcache=covcache)

Danger

Reuse a cache only with the same sourcemat. The Gram matrix depends on the rows, so a cache from different rows produces silently wrong updates. This is why stability selection builds a fresh cache per subsample.

compute_covariance_cache() builds one eagerly. It is O(p²) in memory.

References