alterlab-transformers

Transformers

Overview

Hugging Face Transformers loads pretrained checkpoints from the Hub for NLP, vision, audio, and multimodal tasks, runs inference through pipeline() or the Auto* classes, and fine-tunes them with Trainer. This skill targets transformers v5 (≥ 5.0; current 5.17.0 as of 2026-09). v5 is PyTorch-only and changes several v4 idioms, so check "v5 changes that break v4 code" below before reusing older snippets from papers, blogs, or model cards.

When to Use This Skill

Use this skill when the user wants to:

Run quick inference with a Hub checkpoint via pipeline() — text classification, NER, zero-shot classification, text generation with a chat model, speech recognition, image classification/detection, or image-text-to-text with a VLM.
Load a model plus tokenizer/processor with explicit dtype, device_map, attention backend, or 4/8-bit quantization.
Control decoding in model.generate() (greedy, sampling, beam search, streaming, chat templates).
Fine-tune an encoder or decoder on a custom labelled dataset with Trainer, optionally with LoRA via peft.
Port v4-era code (torch_dtype=, load_in_8bit=, removed pipelines, TF/Flax models) to v5.

Does NOT Trigger

Scenario	Use Instead
Classical ML on tabular features (random forests, preprocessing pipelines, CV grid search)	`alterlab-scikit-learn`
Structuring a custom PyTorch architecture's training loop, multi-GPU strategy, and checkpointing with a LightningModule	`alterlab-pytorch-lightning`
Topic models, embeddings, or text classifiers as a social-science measurement design (validity, BERTopic, dictionaries)	`alterlab-text-as-data`
Protein language models (ESM3 / ESM C embeddings, protein design)	`alterlab-esm`
Zero-shot time-series forecasting with a pretrained foundation model	`alterlab-timesfm`

Installation

uv pip install "transformers>=5" torch accelerate datasets

Vision: add pillow and torchvision (the default image-processor backend; timm only for timm-backed models).
Audio: add librosa soundfile.
LoRA / quantization: peft, bitsandbytes (CUDA). Metrics: evaluate or plain scikit-learn metrics.

Authentication

Gated or private models need a Hub token. Either log in once:

from huggingface_hub import login
login()  # or run `hf auth login` in a shell (huggingface-cli is deprecated)

or export the variable the Hub client reads:

export HF_TOKEN="your_token_here"

Tokens: https://huggingface.co/settings/tokens. Pass token= (not the removed use_auth_token=) when you need it explicitly.

Quick Start

from transformers import pipeline

# Text generation with a small open chat model (CPU-friendly; add device_map="auto" on GPU)
generator = pipeline("text-generation", model="Qwen/Qwen3-0.6B")
messages = [{"role": "user", "content": "Explain p-hacking in two sentences."}]
out = generator(
    messages,
    max_new_tokens=128,
    tokenizer_encode_kwargs={"enable_thinking": False},  # Qwen3 chat-template switch; omit for other models
)
print(out[0]["generated_text"][-1]["content"])

# Text classification — pin the checkpoint; task defaults change between releases
classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
classifier(["This movie was excellent!", "Terrible pacing."])

# Zero-shot classification
zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
zero_shot("The grant covers two postdoc salaries.", candidate_labels=["funding", "teaching", "ethics"])

Use max_new_tokens (tokens to generate) rather than max_length (prompt + output, and in the text-generation pipeline it also feeds tokenizer truncation). Summarization, translation, and question answering no longer have dedicated pipelines in v5 — prompt a chat model as above.

v5 Changes That Break v4 Code

v4 idiom	v5 replacement
`torch_dtype=torch.float16`	`dtype=torch.float16` (`torch_dtype` only warns). Default is now `dtype="auto"` — the checkpoint's dtype, often bf16 — not float32
`TFAutoModel…` / `FlaxAutoModel…`, `framework="tf"`, `return_tensors="tf"`	Removed; PyTorch only (`return_tensors` accepts `"pt"`, `"np"`, `"mlx"`)
`load_in_8bit=True` / `load_in_4bit=True`	`quantization_config=BitsAndBytesConfig(...)`
`pipeline("summarization" / "translation_xx_to_yy" / "text2text-generation" / "question-answering")`	Removed — `pipeline("text-generation")` with a chat model and an instruction prompt
`pipeline("image-to-text" / "visual-question-answering")`	`pipeline("image-text-to-text")` with a VLM, e.g. `Qwen/Qwen3-VL-2B-Instruct`
`AutoModelForVision2Seq`, `AutoModelWithLMHead`	`AutoModelForImageTextToText`; `AutoModelForCausalLM` / `AutoModelForMaskedLM` / `AutoModelForSeq2SeqLM`
`apply_chat_template(..., tokenize=True)` returned a tensor of ids	Returns a `BatchEncoding` (`input_ids`, `attention_mask`) → `model.generate(**inputs)`
Slow vs fast tokenizers, `use_fast=`	One tokenizer per model on the 🤗 tokenizers backend; `AutoTokenizer` ignores `use_fast`
`batch_decode`, `encode_plus`, `additional_special_tokens`	`decode` handles batches, `tokenizer(...)`, `extra_special_tokens` (old names kept for backward compatibility)
`penalty_alpha` (contrastive search), `force_words_ids` / `constraints`, group beam search, DoLa	Moved to Hub `custom_generate` repos — e.g. `custom_generate="transformers-community/contrastive-search", trust_remote_code=True`
`TrainingArguments(logging_dir=..., warmup_ratio=...)`	Both removed: set the `TENSORBOARD_LOGGING_DIR` env var; `warmup_steps=0.1` (a float < 1 is a ratio)
`Trainer(tokenizer=...)`	`Trainer(processing_class=...)`; `report_to` now defaults to `"none"`
`save_pretrained(safe_serialization=False)`, `use_auth_token=`	Always safetensors; `token=`
`TRANSFORMERS_CACHE`, `low_cpu_mem_usage=True`	`HF_HOME` / `HF_HUB_CACHE`; low-memory loading is always on (flag ignored)

Source: the official v5 migration guide, cross-checked against the 5.17.0 release.

Core Capabilities

1. Pipelines for Quick Inference

One call covers tokenization, the forward pass, and post-processing for text classification, NER, zero-shot, fill-mask, text generation, ASR, audio classification, image classification/segmentation, object detection, depth estimation, and image-text-to-text. Use for prototyping and batch inference without custom preprocessing. See references/pipelines.md.

2. Model Loading and Management

from_pretrained with dtype, device_map, attn_implementation (SDPA by default), and quantization_config; saving, Hub upload, and ONNX export options. See references/models.md.

3. Text Generation

generate() with greedy, sampling (temperature/top-k/top-p), and beam search; chat templates, streaming, and static caches. See references/generation.md.

4. Training and Fine-Tuning

Trainer + TrainingArguments with mixed precision, gradient accumulation/checkpointing, callbacks, hyperparameter search, and LoRA via peft. See references/training.md.

5. Tokenization

Padding, truncation, special tokens, offsets, and chat templates. See references/tokenizers.md.

Common Patterns

Pattern 1: Simple Inference

pipe = pipeline("task-name", model="org/model-id")
output = pipe(input_data)

Pattern 2: Explicit Model + Tokenizer (chat model)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "State the central limit theorem in one sentence."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    enable_thinking=False,  # Qwen3-specific template variable
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=128)
new_tokens = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Pattern 3: Fine-Tuning a Classifier

from datasets import load_dataset
from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
                          DataCollatorWithPadding, Trainer, TrainingArguments)

model_id = "google-bert/bert-base-uncased"  # or answerdotai/ModernBERT-base
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=5)

ds = load_dataset("Yelp/yelp_review_full")
ds = ds.map(lambda b: tokenizer(b["text"], truncation=True), batched=True)

args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=16,
    eval_strategy="epoch",
    warmup_steps=0.1,          # ratio of total steps
    report_to="tensorboard",   # default is "none" in v5
)
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=ds["train"],
    eval_dataset=ds["test"],
    processing_class=tokenizer,
    data_collator=DataCollatorWithPadding(tokenizer),
)
trainer.train()

Practical Notes

Pin model IDs (and ideally revision=) in research code: pipeline defaults and Hub repos change, which silently changes results.
Report the exact checkpoint, transformers version, decoding parameters, and seed (transformers.set_seed) when generated text or fine-tuned metrics appear in a paper.
Check the model card's license and gating terms before redistributing weights or outputs.

Reference Documentation

references/pipelines.md — supported v5 tasks, parameters, batching, and removed pipelines
references/models.md — loading, dtype/device/attention/quantization, saving, export
references/generation.md — decoding strategies, chat templates, streaming, caches
references/training.md — Trainer workflow, TrainingArguments, PEFT, tuning
references/tokenizers.md — tokenization, special tokens, chat templates

Part of the AlterLab Academic Skills suite.