User guide

Everything the BAE can do, grouped by the question it answers.

Concepts

Read these once. They explain choices that otherwise look arbitrary.

• How the BAE works: the boosting/SGD alternation, why the encoder is reset to zero every iteration, and where sparsity actually comes from.

• The standardization contract: why the input must be z-scored, and the second job that contract quietly does.

• What a fit writes into your AnnData: every key fit writes into your AnnData, and which two metrics are routinely confused with each other.

• Reading reconstruction quality: why a reconstruction MSE means nothing on its own, and how to decide between “more dimensions” and “more iterations”.

Capabilities

Fitting a model

The core loop: an AnnData in, a sparse encoder and a latent space out. Fitting a model covers the minimal path, the config fields worth touching (and the several that benchmarks say to leave alone), early stopping, and warm starts from an existing embedding or a PCA.

→ BAE, BAEConfig

Getting a gene list you can trust

Exploratory

Stability selection is under active development. It works and is documented with what is known about it, but it carries no formal error control and its API and defaults are more likely to change than the core fit’s. See Exploratory features.

A single fit reports one gene list, and that list is not reproducible: the reconstruction loss converges long before the encoder support does. Two stability modes quantify different sources of that instability, and mandatory_genes forces known markers into the model specification.

Getting a gene list you can trust also covers the part most easily misread, mandatory_genes forces genes into the specification, not into the fitted support, and neither stability mode gives you usable error control.

→ BAE.stability_selection, StabilitySelectionResult, BAE.apply_encoder

Handling batches and unwanted covariates

Name the covariate once with batch_key, and batch_integration_mode decides which of two mechanisms act on it: giving it to the decoder, regressing it out inside the boosting fit so it cannot confound gene selection, or both, which is the default.

transform stays gene-only and needs no batch labels. That is what makes the encoder deployable on data with no covariate annotation.

→ Batches and unwanted covariates

Carrying a model to a new dataset

Exploratory

Weight transfer is under active development, and the latent-scale issue described on that page is a live limitation rather than a settled design. See Exploratory features.

The transferable product of a fit is its encoder matrix: k sparse gene programs. from_reference() aligns such a matrix to a new gene panel by gene name, freezes it, and adds dimensions for the structure the prior cannot explain.

→ Transferring a model to a new dataset, read_encoder_weights(), write_encoder_weights()

Working out what a dimension means

Ranked gene lists per dimension, storage for functional annotations, an interactive HTML explorer, and diagnostic plots.

→ Interpreting latent dimensions, extract_gene_rankings(), export_interactive_html()

Saving a fitted model

One .pt checkpoint holding the encoder, decoder, config, covariate encodings and diagnostics, and deliberately not holding the optimizer state or any cell-level training data.

→ Saving and loading a model, BAE.save, BAE.load

Testing a method against known truth

A negative-binomial simulator with planted gene programs, a cell-type hierarchy, and optional batch, ambient-RNA and dropout effects, so you can score marker recovery against ground truth.

→ Simulating data with known truth, sim_scrnaseq_anndata()

Sparse supervised boosting on its own

allboost without the autoencoder: a sparse coefficient matrix mapping features to targets. Regressing cluster indicators on genes gives per-cluster marker signatures directly.

→ Sparse supervised boosting, allboost()

Exploratory features

The whole package is pre-1.0, but these four are under active development and less settled than the rest. Each works and each is documented with what is known about it; what distinguishes them is that their behaviour, defaults and APIs are more likely to change, and that results from them warrant more scepticism than the core fit does.

Feature	Why it is still exploratory
Stability selection	No formal error control: expected_false_positives is deliberately NaN, because training iterations are neither independent nor exchangeable. Per-dimension frequencies are only meaningful when dimensions keep their identity, which dim_match_quality reports and does not guarantee. See Getting a gene list you can trust.
Disentanglement	On by default since 0.5.0, and both methods are provisional. Decorrelation is an extra constraint that real gene programs do not satisfy, so it costs biological structure — measured at marker-recovery F1 0.98 to 0.88 on simulated data. disentanglement_alpha softens it; "none" turns it off. See Fitting a model.
Starting from an existing representation	init_pca and init_obsm are applied once, on the first iteration, and silently override latent_dim when the supplied representation is a different width. The benefit also depends on the decoder having enough steps in that first iteration to follow the warm start. See Fitting a model.
Starting from a prior encoder matrix	Transfer works, but the two latent blocks land on incomparable scales — a 232x gap in per-dimension standard deviation was measured — so every Euclidean consumer must be handed obsm["X_bae_scaled"]. See Transferring a model to a new dataset.

Caveats worth reading before you trust a result

Each is explained where it belongs, and each has bitten someone:

Caveat	Where
mandatory_genes does not guarantee a non-zero weight	Getting a gene list you can trust
Stability selection carries no formal error control	Getting a gene list you can trust
novel_variance_share on training data is not evidence of novel biology	Transferring a model to a new dataset
Installing a stability-selected encoder on a frozen transfer can silently zero the prior	Transferring a model to a new dataset
After a transfer, hand X_bae_scaled, not X_bae, to any Euclidean tool	Transferring a model to a new dataset
A mismatched gene panel warns. It does not raise	What a fit writes into your AnnData
A bare reconstruction MSE is uninterpretable	Reading reconstruction quality