Regression: predicting numbers

Regression means predicting a number. In this tutorial you will build a model that predicts how many cups Brew & Bean sells in a day from the temperature, whether it is a weekend and whether a promotion is running. You will split the data properly, beat a simple baseline, read what the model learned, measure the error in cups and plot the mistakes.

Prerequisites: the pandas and EDA lessons and a Colab or Kaggle notebook. The data is the same kind of made-up daily sales as in the EDA lesson (without the missing values and the wrong entry), so the cell below recreates it.

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 365
dates = pd.date_range("2025-11-01", periods=n, freq="D")
temp = 24 + 6 * np.sin((dates.day_of_year.to_numpy() - 80) / 365 * 2 * np.pi) + rng.normal(0, 2, n)
weekend = (dates.day_of_week >= 5).astype(int)
promo = rng.binomial(1, 0.2, n)
cups = 90 + 6 * (temp - 24) + 25 * weekend + 18 * promo + rng.normal(0, 10, n)

sales = pd.DataFrame({
    "temp_c": temp.round(1),
    "is_weekend": weekend,
    "promo": promo,
    "cups_sold": cups.round().astype(int),
})
print(sales.head())

This is simulated data. The true rule hides inside it: roughly 6 extra cups per degree, 25 extra on weekends, 18 extra on promo days, plus random noise of about 10 cups. A good model should recover numbers close to these.

X holds the features and y the label. Then we split the rows into a training set (the model learns from it) and a test set (kept hidden, used once at the end to judge the model). Here 80% trains and 20% tests. random_state=42 makes the split repeatable.

from sklearn.model_selection import train_test_split

X = sales[["temp_c", "is_weekend", "promo"]]
y = sales["cups_sold"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print(X_train.shape, X_test.shape)

Prints (292, 3) (73, 3): 292 days to learn from, 73 days held back.

A baseline is the dumbest sensible prediction. If your model cannot beat it, it is useless. For regression, the usual baseline is to always predict the average of the training labels.

from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error

baseline = DummyRegressor(strategy="mean").fit(X_train, y_train)
base_mae = mean_absolute_error(y_test, baseline.predict(X_test))
print("Baseline MAE:", round(base_mae, 1), "cups")

The baseline is off by about 28.6 cups per day on average. That is the number to beat.

Linear regression predicts with a weighted sum: cups = intercept + w1 * temp + w2 * weekend + w3 * promo. Training finds the weights that make the squared errors on the training data as small as possible.

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)

print("intercept:", round(model.intercept_, 1))
for name, weight in zip(X.columns, model.coef_):
    print(f"{name:10s} {weight:6.2f}")

You should see about: temp_c 6.07, is_weekend 23.48, promo 17.60 and an intercept near -55. These are close to the hidden rule (6, 25, 18). In words: each extra degree adds about 6 cups, a weekend about 23, a promotion about 18.

This readability is the great strength of linear regression. You can explain to the shop owner exactly what drives sales. The intercept (-55) is the prediction when every feature is 0, which here means 0 degrees; that is not a real situation, so do not read meaning into it.

from sklearn.metrics import mean_squared_error, r2_score

pred = model.predict(X_test)

mae = mean_absolute_error(y_test, pred)
rmse = np.sqrt(mean_squared_error(y_test, pred))
r2 = r2_score(y_test, pred)

print("MAE :", round(mae, 1))
print("RMSE:", round(rmse, 1))
print("R2  :", round(r2, 3))

About MAE 8.6, RMSE 10.5 and R2 0.905. The model's average error (8.6 cups) is about a third of the baseline's (28.6). The error left over is mostly the random noise built into the data, which no model can predict.

Always report the error in the units of the problem. "We predict within about 9 cups on a typical day" means something to the owner; "R2 is 0.9" does not.

new_day = pd.DataFrame({"temp_c": [30.0], "is_weekend": [1], "promo": [0]})
print(model.predict(new_day).round(0))

A hot Saturday with no promotion gives about 150 cups. The input must have the same columns, in the same order, as the training features. Using a DataFrame with named columns helps catch mistakes.

Residuals are actual minus predicted. If the model is sound, residuals look like random noise centred on 0. A pattern in them (a curve, or errors that grow with the prediction) means the model is missing something.

import matplotlib.pyplot as plt

residuals = y_test - pred

plt.scatter(pred, residuals, alpha=0.7)
plt.axhline(0, color="red")
plt.xlabel("Predicted cups")
plt.ylabel("Actual minus predicted")
plt.title("Residuals on the test set")
plt.show()

You should see a shapeless cloud around the red zero line, with all points within about 30 cups. That means no obvious pattern is left.

from sklearn.ensemble import RandomForestRegressor

forest = RandomForestRegressor(n_estimators=200, random_state=42)
forest.fit(X_train, y_train)
print("Forest MAE:", round(mean_absolute_error(y_test, forest.predict(X_test)), 1))

MAE is about 10.1, slightly worse than the linear model (8.6). This data was generated from a straight-line rule, so the simple model wins. On real data with curved or interacting effects, a forest or boosted trees often win. Random forests are covered in *Decision trees, random forests and gradient boosting*.

Our three features are on different scales (degrees, 0/1, 0/1). Plain linear regression does not mind, because the predictions are the same. But if you want to compare the weights to see which feature matters most, or you use regularised models (Ridge, Lasso) or distance-based ones (k-nearest neighbours), put features on the same scale first. StandardScaler subtracts the mean and divides by the std. Wrap it with the model in a Pipeline so that the scaler learns only from the training data.

from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

pipe = make_pipeline(StandardScaler(), LinearRegression())
pipe.fit(X_train, y_train)
print(pipe.named_steps["linearregression"].coef_.round(1))

Prints about [27.9 10.7 7.1]. Now the weights are comparable: one std of temperature moves sales most, then weekend, then promo. The test error is identical to before.

I trained a [model type] regression model to predict [target] from [features]. On the test set: MAE = [value], RMSE = [value], R2 = [value]. A baseline that predicts the mean has MAE = [value]. The learned weights are: [paste weights].

Explain what these results mean for a non-technical manager, in 5 bullet points. Say whether the model is good enough for [decision], list two things that could make the results misleading, and suggest one next experiment.

What to expect: The explanation should compare against the baseline and use the units of the target. Check that it does not claim the weights prove cause and effect.