NLP and transformers with Hugging Face

What you will do: use pretrained transformer models from Hugging Face to classify sentiment, tag text with labels you choose, turn sentences into vectors for semantic search, and fine-tune a small model on your own labelled data.

Prerequisites: basic Python, *Neural networks explained*, and a free Colab or Kaggle notebook (see *Free notebooks: Google Colab and Kaggle*). Both already have the libraries. On your own machine: pip install transformers datasets torch.

The first run of each model downloads it from the Hugging Face Hub, so you need internet and a little patience on slow connections. Models are usually from tens of MB to a few GB.

Before transformers, models read text one word at a time and tended to forget the start of long sentences. A transformer uses attention: when it processes a word, it looks at all other words in the sentence at once and decides which ones matter. In "The bank near the river was muddy", attention lets the word *bank* look at *river* and understand it is not a money bank.

Three flavours matter in practice:

- Encoder models (BERT, DistilBERT): read a whole text and produce a good representation. Best for classification, search and extracting information. - Decoder models (GPT-style, and today's chatbots): generate text one token at a time. - Encoder-decoder models (T5, BART): turn one text into another, such as translation and summarisation.

Hugging Face is the main library and website for sharing these models: the Hub has hundreds of thousands of models and datasets, and the transformers library lets you use them in a few lines. Many models are free to download; always check each model's licence before using it in a product.

Models cannot read letters. A tokenizer splits text into pieces called tokens (often whole words or word parts) and maps each to an integer ID.

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")

text = "Brew & Bean's cold brew is unbelievably good!"
print(tok.tokenize(text))
print(tok(text)["input_ids"])

Shows the word pieces (rare words are split, e.g. into 'un', '##bel', ...) and their IDs. The special tokens [CLS] and [SEP] are added at the start and end of the ID list.

Each model has its own tokenizer, so always load the tokenizer and model from the same name. The context length (maximum tokens) differs by model; long texts are cut with truncation=True.

A pipeline hides tokenising, running the model and decoding the result. Naming the model explicitly means your results do not change if library defaults change.

from transformers import pipeline

clf = pipeline(
    "sentiment-analysis",
    model="distilbert/distilbert-base-uncased-finetuned-sst-2-english",
)

reviews = [
    "The cappuccino was perfect and the staff were lovely.",
    "Waited 40 minutes and my order was still wrong.",
    "It was okay, nothing special.",
]
for r, out in zip(reviews, clf(reviews)):
    print(out["label"], round(out["score"], 3), "|", r)

This model only knows POSITIVE and NEGATIVE, so the neutral review will be forced into one of them. A confident score does not mean a correct answer.

Sometimes you have no labelled data. A zero-shot model can sort text into labels you invent on the spot. It works by asking itself, for each label, "does this text logically entail this label?". It is slower than a trained classifier but needs no training.

from transformers import pipeline

zs = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")

text = "My UPI payment went through but I never got an order confirmation."
labels = ["payment issue", "food quality", "delivery delay", "app bug"]

result = zs(text, candidate_labels=labels)
for label, score in zip(result["labels"], result["scores"]):
    print(f"{label:15s} {score:.2f}")

Labels are printed from most to least likely. This model is large (over a GB), so the first download can take a while.

An embedding is a list of numbers (a vector) that captures the meaning of a sentence. Sentences with similar meaning have vectors that point in similar directions, measured by cosine similarity. This is the base of semantic search and of retrieval for AI assistants (RAG).

We use sentence-transformers/all-MiniLM-L6-v2, a small, popular embedding model. We average the token vectors (mean pooling), ignoring padding.

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

name = "sentence-transformers/all-MiniLM-L6-v2"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModel.from_pretrained(name)
model.eval()

def embed(sentences):
    enc = tok(sentences, padding=True, truncation=True, return_tensors="pt")
    with torch.no_grad():
        out = model(**enc).last_hidden_state          # (batch, tokens, 384)
    mask = enc["attention_mask"].unsqueeze(-1)         # ignore padding tokens
    pooled = (out * mask).sum(dim=1) / mask.sum(dim=1)
    return F.normalize(pooled, p=2, dim=1)             # unit length vectors

docs = [
    "How do I reset my password?",
    "Steps to recover a forgotten login",
    "What time does the shop open?",
    "Opening hours on Sunday",
]
query = ["I can't remember my password"]

d = embed(docs)
q = embed(query)
scores = (q @ d.T)[0]                                  # cosine similarity
for score, doc in sorted(zip(scores.tolist(), docs), reverse=True):
    print(f"{score:.2f}  {doc}")

The two password sentences rank at the top even though they share almost no words with the query. That is the advantage over keyword search.

When a generic model is not accurate enough on your kind of text, fine-tune a small pretrained model: continue training it on your labelled examples. We use IMDB movie reviews (positive or negative), DistilBERT, and only 2,000 training examples so it finishes in a few minutes on a free GPU. Switch on the GPU in Colab or Kaggle first. On CPU this will be very slow.

import numpy as np
from datasets import load_dataset
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    DataCollatorWithPadding,
    TrainingArguments,
    Trainer,
)

name = "distilbert/distilbert-base-uncased"
tok = AutoTokenizer.from_pretrained(name)

ds = load_dataset("stanfordnlp/imdb")
train = ds["train"].shuffle(seed=42).select(range(2000))
test = ds["test"].shuffle(seed=42).select(range(500))

def tokenize(batch):
    return tok(batch["text"], truncation=True, max_length=256)

train = train.map(tokenize, batched=True)
test = test.map(tokenize, batched=True)

model = AutoModelForSequenceClassification.from_pretrained(name, num_labels=2)

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    return {"accuracy": float((preds == labels).mean())}

args = TrainingArguments(
    output_dir="imdb-model",
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    learning_rate=2e-5,
    report_to="none",
)

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train,
    eval_dataset=test,
    data_collator=DataCollatorWithPadding(tok),   # pads each batch to its longest text
    compute_metrics=compute_metrics,
)

trainer.train()
print(trainer.evaluate())

Prints the training loss and then a dictionary containing eval_accuracy. After even one epoch on 2,000 examples it is typically well above 80 percent; exact numbers vary. A warning that some weights were newly initialised is normal: it is the new classification layer.

Then save and reuse your model:

trainer.save_model("imdb-model/final")
tok.save_pretrained("imdb-model/final")

my_clf = pipeline("sentiment-analysis", model="imdb-model/final", tokenizer="imdb-model/final")
print(my_clf("A slow start, but the last hour was brilliant."))

Add from transformers import pipeline if you started a new session. The labels will appear as LABEL_0 (negative) and LABEL_1 (positive) unless you set id2label when loading the model.

In practice, try the cheap options first and measure on a labelled test set. If a prompt to a large language model already gets the accuracy you need, you may not need to train anything. Using an LLM through an API is covered in the AI Engineering track.

I need to [describe task, e.g. sort customer emails into 5 categories]. I have [number] labelled examples, mostly in [language(s)], and a free Colab GPU. Compare: a zero-shot pipeline, embeddings with logistic regression, fine-tuning DistilBERT, and prompting an LLM. Recommend one to try first, how to measure it on a held-out test set, and what could go wrong.

What to expect: A good answer asks about class balance and language, and suggests a baseline first. Verify any model names it gives on huggingface.co before using them.