Scgpt

scGPT: Foundation Model for Single-Cell Biology

Overview

scGPT is a generative pretrained transformer trained on 33 million human single cells. It learns a universal representation of gene expression that transfers across tissues, diseases, and experimental batches.

Core tasks:

• Cell type annotation — classify cell types by fine-tuning on labeled data
• Cell embedding — zero-shot or fine-tuned cell representations for clustering/UMAP
• Perturbation prediction — predict gene expression changes after knockouts/overexpression
• Multi-batch integration — harmonize data across studies and technologies
• Reference mapping — map query cells onto a large reference atlas
• Gene regulatory network (GRN) — infer gene-gene relationships from learned embeddings
• Multi-omics — integrate RNA + protein (CITE-seq) or RNA + ATAC

Input: AnnData objects with raw or normalized count matrices (adata.X, genes × cells).
Output: Cell embeddings (stored in adata.obsm['X_scGPT']), cell type predictions, perturbation forecasts.

Installation

pip install scgpt "flash-attn<1.0.5"

Requirements: Python ≥ 3.7.12, PyTorch ≥ 1.13, CUDA 11.7 recommended.

Key dependencies installed automatically: scanpy, scvi-tools, numba, umap-learn, leidenalg.

Pretrained Checkpoints

Download from the scGPT Google Drive. Each checkpoint contains vocab.json, best_model.pt, and args.json.

Model	Training data	Best for
scGPT_human (whole-human)	33M normal human cells	General-purpose default
scGPT_CP (continual-pretrained)	Refined on Tabula Sapiens	Zero-shot embedding, cell typing
scGPT_brain	13.2M human brain cells	Brain tissue analysis
scGPT_blood	10.3M hematopoietic cells	Blood / immune studies
scGPT_heart	1.8M cardiac cells	Heart tissue
scGPT_lung	2.1M lung cells	Lung tissue
scGPT_kidney	814K kidney cells	Kidney tissue
scGPT_panCancer	5.7M cancer cells	Cancer type classification

Core Workflows

1. Load pretrained model

import json
import torch
from pathlib import Path
from scgpt.tokenizer.gene_tokenizer import GeneVocab
from scgpt.model import TransformerModel

model_dir = Path("save/scGPT_human")
vocab = GeneVocab.from_file(model_dir / "vocab.json")
config = json.load(open(model_dir / "args.json"))

model = TransformerModel(
    ntokens=len(vocab),
    embsize=config["embsize"],
    nhead=config["nheads"],
    d_hid=config["d_hid"],
    nlayers=config["nlayers"],
    vocab=vocab,
)
model.load_state_dict(torch.load(model_dir / "best_model.pt"))
model.eval()

2. Preprocess AnnData

import scanpy as sc
from scgpt.preprocess import Preprocessor

adata = sc.read_h5ad("data.h5ad")

preprocessor = Preprocessor(
    n_top_genes=1200,   # highly variable genes to keep
    binning=51,         # expression quantization bins
    normalize_total=10000,
    log1p=True,
)
preprocessor(adata, batch_key=None)  # pass batch_key for multi-batch

3. Extract cell embeddings (zero-shot)

from scgpt.tasks.cell_emb import get_batch_cell_embeddings

embeddings = get_batch_cell_embeddings(
    adata,
    cell_embedding_mode="cls",  # "cls", "avg-pool", or "w-pool"
    model=model,
    vocab=vocab,
    max_length=1200,
    batch_size=64,
)
adata.obsm["X_scGPT"] = embeddings

# Downstream: UMAP, clustering
sc.pp.neighbors(adata, use_rep="X_scGPT")
sc.tl.umap(adata)
sc.pl.umap(adata, color="cell_type")

4. Cell type annotation (fine-tuning)

import torch.nn as nn

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
criterion = nn.CrossEntropyLoss()

for epoch in range(10):
    model.train()
    for batch in train_loader:
        gene_ids = batch["gene_ids"].to(device)
        values   = batch["values"].to(device)
        labels   = batch["labels"].to(device)

        output = model(gene_ids, values, CLS=True)
        loss = criterion(output["cls_output"], labels)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

5. Multi-batch integration

import scanpy as sc

adata = sc.concat([adata1, adata2], label="batch")
preprocessor(adata, batch_key="batch")

# Encode with batch labels
cell_embeddings = model.encode_batch(
    all_gene_ids,
    all_values.float(),
    batch_labels=torch.from_numpy(batch_ids).long(),
    batch_size=64,
    return_np=True,
)
adata.obsm["X_scGPT"] = cell_embeddings

6. Perturbation prediction

from scgpt.model.generation_model import TransformerGenerator

generator = TransformerGenerator(...)  # load with pretrained weights

perturbations = [["GENE1"], ["GENE1", "GENE2"]]
predictions = generator.pred_perturb(batch_data, pert_list=perturbations)
# predictions: (n_cells, n_genes) predicted expression matrix

7. Gene regulatory network

from scgpt.tasks.grn import GeneEmbedding

# Extract gene embeddings from the model
gene_emb = GeneEmbedding(gene_embeddings_dict)

# Find genes similar to a query gene
similar = gene_emb.get_similar_genes("TP53", subset=hvg_list)

# Build network graph
network = gene_emb.generate_network(threshold=0.5)

Key Parameters

Parameter	Default	Description
n_top_genes	1200	Highly variable genes to select
binning	51	Expression quantization bins
normalize_total	10000	Depth normalization target
max_length	1200	Max genes per cell (sequence length)
batch_size	64	Inference batch size
cell_embedding_mode	"cls"	Embedding pooling: "cls", "avg-pool", "w-pool"
embsize	128–512	Model embedding dimension (from config)
mask_ratio	0.4	MLM masking ratio during training

Gene Vocabulary

scGPT uses a vocabulary of 60,697 human gene symbols (HGNC standardized). Special tokens: <pad>, <cls>, <eoc>. The vocabulary file (vocab.json) maps gene names to integer IDs and is bundled with each checkpoint.

If your data uses Ensembl IDs or non-standard gene names, convert to HGNC symbols first.

Scripts

• scripts/embed.py — extract cell embeddings from an h5ad file; see scripts/embed.py --help

Resources

• GitHub: https://github.com/bowang-lab/scGPT
• Paper: Cui et al., Nature Methods 2024 — https://doi.org/10.1038/s41592-024-02201-0
• Tutorials: tutorials/ directory in the repository (annotation, perturbation, integration, GRN, multi-omics)

References

• references/api-reference.md — full API reference: TransformerModel, Preprocessor, GeneVocab, task functions, multi-omics, FAISS reference mapping