Project: predict customer churn

Goal: build a model that predicts which customers of a subscription business are likely to leave (churn), so the company can contact them before they go. You will take the project from a raw CSV file to a saved model that scores new customers.

The finished result: a notebook or script that loads churn.csv, compares a baseline, logistic regression and gradient boosting using cross-validation, chooses a threshold based on business cost, explains what drives churn, and saves a model file you can load later.

Prerequisites: the earlier lessons in this module: *Classification*, *Train, validation, test and overfitting*, *Metrics*, *Feature engineering* and *Decision trees, random forests and gradient boosting*. Use Google Colab or Kaggle.

Imagine a broadband and OTT subscription company in India with 4,000 customers in this sample. Winning a new customer costs far more than keeping an existing one, so the retention team wants a short list of at-risk customers each month. They can call or offer a discount to a limited number of people.

Translate that into machine learning:

- Task: binary classification. Label: churn (1 = left, 0 = stayed). - Output: a churn probability per customer, so the team can sort by risk. - Metric: accuracy is a poor guide, so we use ROC AUC to compare models and precision/recall to choose how many customers to contact.

The standard public dataset for this problem is the Telco customer churn data, available on Kaggle. To keep this project fully runnable without logging in or downloading anything, the script below creates a realistic synthetic dataset with the same style of columns. The patterns are built in on purpose: month-to-month contracts, short tenure, many support calls and cash on delivery payments raise churn risk. The rest of the project works unchanged if you later swap in a real CSV with similar columns.

# make_data.py  (run once: it creates churn.csv)
import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 4000

contract = rng.choice(["Month-to-month", "One year", "Two year"], size=n, p=[0.55, 0.25, 0.20])
tenure = rng.integers(1, 73, size=n)                       # months with the company
plan = rng.choice(["Basic", "Standard", "Premium"], size=n, p=[0.4, 0.4, 0.2])
base = {"Basic": 399, "Standard": 599, "Premium": 899}     # monthly price in rupees
monthly_charges = np.array([base[p] for p in plan]) + rng.normal(0, 40, size=n).round(0)
payment = rng.choice(["UPI", "Card", "Net banking", "Cash on delivery"], size=n, p=[0.45, 0.25, 0.2, 0.1])
support_calls = rng.poisson(1.5, size=n)
paperless = rng.choice(["Yes", "No"], size=n, p=[0.6, 0.4])
total_charges = (tenure * monthly_charges * rng.uniform(0.95, 1.05, size=n)).round(0)

# Churn is more likely for month-to-month, new, expensive customers with many support calls.
logit = (
    -0.6
    + 1.4 * (contract == "Month-to-month")
    - 1.2 * (contract == "Two year")
    - 0.04 * tenure
    + 0.002 * (monthly_charges - 600)
    + 0.35 * support_calls
    + 0.4 * (payment == "Cash on delivery")
)
churn = rng.random(n) < 1 / (1 + np.exp(-logit))

df = pd.DataFrame({
    "customer_id": [f"C{i:05d}" for i in range(1, n + 1)],
    "contract": contract,
    "tenure_months": tenure,
    "plan": plan,
    "monthly_charges": monthly_charges,
    "total_charges": total_charges,
    "payment_method": payment,
    "support_calls": support_calls,
    "paperless_billing": paperless,
    "churn": np.where(churn, "Yes", "No"),
})
# Real data is messy: hide some total_charges values
df.loc[rng.choice(n, size=25, replace=False), "total_charges"] = np.nan

df.to_csv("churn.csv", index=False)
print(df.shape)
print(df["churn"].value_counts(normalize=True).round(3))

Run once. It writes churn.csv to the current folder. In Colab, paste it into a cell and run it.

Always explore first. Check size, types, missing values and class balance, and look at how the target varies with a few columns.

import pandas as pd

pd.set_option("display.width", 120)
pd.set_option("display.max_columns", 20)

df = pd.read_csv("churn.csv")
print(df.head(3).T)
print(df.isna().sum())
print(df.groupby("contract")["churn"].apply(lambda s: (s == "Yes").mean()).round(2))
print(df.groupby("support_calls")["churn"].apply(lambda s: (s == "Yes").mean()).round(2))

Run in a new cell or notebook; it reads the file created in stage 1.

Three findings. First, 33% of customers churned: not rare, but imbalanced enough that a do-nothing model gets 67% accuracy. Second, contract type is a strong signal: 47% of month-to-month customers churned against 7% of two-year customers. Third, churn rises with support calls (the last rows have few customers, so the dip at 7 calls is noise). total_charges has 25 missing values that we must handle.

We convert the label to 0/1, drop customer_id (an ID carries no pattern and lets a flexible model memorise), and keep 20% of the customers as a final test set. The test set is not touched until the end. The baseline predicts the most common class for everyone.

from sklearn.dummy import DummyClassifier
from sklearn.model_selection import train_test_split

df["churn"] = (df["churn"] == "Yes").astype(int)     # 1 = left, 0 = stayed

X = df.drop(columns=["customer_id", "churn"])        # an ID carries no signal
y = df["churn"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print("Baseline accuracy:", round(baseline.score(X_test, y_test), 3))

Continues in the same notebook (uses df from stage 2).

Any model that cannot beat 0.67 accuracy, or an ROC AUC of 0.5, is no better than guessing.

We prepare the data differently for each model. Logistic regression needs missing numbers filled and numbers scaled. Gradient boosting (HistGradientBoostingClassifier) accepts missing values and does not care about scale, so it only needs the text columns one-hot encoded. In both cases the preprocessing sits inside a pipeline so that cross-validation never leaks.

We use stratified 5-fold cross-validation on the training set, scoring with ROC AUC and F1.

from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_validate
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_cols = ["tenure_months", "monthly_charges", "total_charges", "support_calls"]
categorical_cols = ["contract", "plan", "payment_method", "paperless_billing"]

# Logistic regression needs imputed, scaled numbers and one-hot categories.
linear_prep = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), numeric_cols),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical_cols),
])
logreg = Pipeline([("prep", linear_prep), ("clf", LogisticRegression(max_iter=1000))])

# Gradient boosting handles missing numbers itself; it only needs the categories encoded.
tree_prep = ColumnTransformer(
    [("cat", OneHotEncoder(handle_unknown="ignore"), categorical_cols)],
    remainder="passthrough",
)
boosting = Pipeline([("prep", tree_prep), ("clf", HistGradientBoostingClassifier(random_state=42))])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
for name, model in [("Logistic regression", logreg), ("Gradient boosting", boosting)]:
    result = cross_validate(model, X_train, y_train, cv=cv, scoring=["roc_auc", "f1"])
    print(f"{name:20s} ROC AUC {result['test_roc_auc'].mean():.3f}   F1 {result['test_f1'].mean():.3f}")

Runs 5-fold cross-validation for each pipeline on the training data only.

Logistic regression wins here, which may surprise you. The reason is that we built the data from a logistic formula, so a straight-line model is exactly the right shape. On real data with more complicated interactions gradient boosting often wins, which is why you always compare instead of assuming. Gradient boosting with default settings also has to learn that simple pattern from only 3,200 training rows full of random noise, which is harder. Try the exercises to tune it.

We pick logistic regression: it is accurate enough here, fast and easy to explain to the retention team.

Now fit the chosen model on the full training set and evaluate it once on the test set. The model gives a churn probability. The default threshold of 0.5 is not necessarily right: it depends on how many customers the team can contact and the cost of a wasted call or offer versus a lost customer.

from sklearn.metrics import (classification_report, confusion_matrix,
                             precision_score, recall_score, roc_auc_score)

logreg.fit(X_train, y_train)
proba = logreg.predict_proba(X_test)[:, 1]          # probability of churn
print("ROC AUC on test:", round(roc_auc_score(y_test, proba), 3))

for threshold in [0.5, 0.4, 0.3, 0.2]:
    pred = (proba >= threshold).astype(int)
    print(f"threshold {threshold}: precision {precision_score(y_test, pred):.2f}, "
          f"recall {recall_score(y_test, pred):.2f}, flagged {pred.sum()} of {len(pred)}")

pred = (proba >= 0.3).astype(int)
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred, target_names=["stayed", "churned"]))

Prints ROC AUC, a table of thresholds, a confusion matrix and a classification report at threshold 0.3.

The test ROC AUC of 0.79 is close to the cross-validated 0.81, a healthy sign that we have not fooled ourselves. The threshold table shows the trade-off in the language the team understands:

- At 0.5 the team calls 207 customers and reaches 47% of the leavers, but 40% of calls are wasted. - At 0.3 the team calls 394 customers and reaches 77% of the leavers; about half the calls (precision 0.52) go to people who would stay.

If each retention call is cheap and each lost customer is expensive, choose 0.3 or lower. If the team can only make about 200 calls a month, choose 0.5 or the top-200 customers by probability. This is a business decision, and your job is to lay out the options clearly.

A linear model can be read from its coefficients. After scaling, a positive coefficient pushes the prediction towards churn and a negative one towards staying; bigger magnitude means stronger effect. (One-hot columns come in groups, so read each one relative to the others in its group.)

import pandas as pd

names = logreg.named_steps["prep"].get_feature_names_out()
coefs = pd.Series(logreg.named_steps["clf"].coef_[0], index=names).sort_values()
print(coefs.round(2))

Prints each preprocessed feature with its weight, sorted.

The story for the retention team: month-to-month customers, customers with many support calls and those on the Premium plan are at higher risk, while two-year contracts and long tenure protect. Be careful with wording: these are associations in the data, not proof that changing a contract type will stop a customer leaving.

A premium plan raising risk is plausible, as higher bills push people to look at alternatives, and in this dataset it was built that way through the price. In a real project you would check such a finding with the business before acting.

A model in a notebook helps nobody. Save the whole pipeline, preprocessing included, with joblib, and load it wherever you need predictions. Because the pipeline contains the preprocessing, new data only needs the same raw columns.

import joblib

joblib.dump(logreg, "churn_model.joblib")

# Later, in another script or notebook:
loaded = joblib.load("churn_model.joblib")

new_customers = pd.DataFrame([{
    "contract": "Month-to-month", "tenure_months": 3, "plan": "Premium",
    "monthly_charges": 899, "total_charges": 2700, "payment_method": "Cash on delivery",
    "support_calls": 4, "paperless_billing": "Yes",
}, {
    "contract": "Two year", "tenure_months": 48, "plan": "Basic",
    "monthly_charges": 399, "total_charges": 19000, "payment_method": "UPI",
    "support_calls": 0, "paperless_billing": "No",
}])
print(loaded.predict_proba(new_customers)[:, 1].round(2))

The two example customers are very different. The first is high risk; the second is low risk.

The new month-to-month, three-month-old, four-support-calls customer has a 93% churn probability; the two-year customer of 48 months has 1%. Turning this into a web service is covered in *MLOps basics: from notebook to deployed model*.

One caution: pickled or joblib files can run code when loaded, so only load model files you created or trust, and use the same scikit-learn version when loading as when saving.

To repeat this with the public Telco customer churn data from Kaggle, download the CSV, load it with pd.read_csv, and adjust the column lists. Things to watch for: the churn column is "Yes"/"No" as here; TotalCharges may load as text because a few rows contain a blank space, so convert it with pd.to_numeric(df['TotalCharges'], errors='coerce'); and drop the ID column. Check the dataset's page for the exact column names and licence.

I built a customer churn model. Details:
- Data: [rows, period, source]
- Best model: [model] with ROC AUC [value] on a held-out test set
- At threshold [value] it flags [N] of [M] customers, catching [recall]% of churners with [precision]% precision
- Main drivers: [list from coefficients]

Write a one-page summary for a non-technical manager: what the model does, how good it is in plain words, what action the team should take, the risks and limits (including that drivers are associations, not causes), and what to monitor once it is in use.

What to expect: A good summary avoids jargon, states the trade-off in calls and customers, and mentions limits and monitoring.