Python for data: NumPy and pandas

You will learn the two libraries behind almost all Python data work: NumPy (fast arrays of numbers) and pandas (tables, called DataFrames). By the end you will load a CSV, find missing values, filter rows, join two tables and summarise sales by item and by day.

Prerequisites: basic Python (variables, lists, functions) and a notebook from the previous lesson. Every code block runs in Colab or Kaggle as it is. We use a small Brew & Bean orders file written inside the code, so you need no download.

A NumPy array is like a Python list of numbers, but it does maths on the whole list at once. This is called vectorised maths, and it is much faster and shorter than writing a loop.

import numpy as np

prices = np.array([120, 180, 220, 80, 90])
print(prices.mean(), prices.max())   # 138.0 220
print(prices * 2)                    # [240 360 440 160 180]
print(prices[prices > 100])          # [120 180 220]  (a boolean mask)

m = np.arange(6).reshape(2, 3)       # 2 rows, 3 columns
print(m.shape)                       # (2, 3)
print(m.sum(axis=0))                 # down the columns: [3 5 7]
print(m.sum(axis=1))                 # across the rows: [ 3 12]

prices > 100 creates True/False for each element. Using it inside the brackets keeps only the True ones. This mask idea is the same one pandas uses for filtering.

A DataFrame is a table: rows are records, columns are variables. A Series is one column. Real data usually arrives as a CSV file, read with pd.read_csv("file.csv"). To keep this lesson self-contained we read the CSV text from a string.

import pandas as pd
from io import StringIO

csv = """order_id,date,item_id,qty,city
1,2026-10-01,espresso,2,Bengaluru
2,2026-10-01,cappuccino,1,Bengaluru
3,2026-10-02,cold-brew,3,
4,2026-10-02,croissant,2,Bengaluru
5,2026-10-03,espresso,,Bengaluru
6,2026-10-03,masala-chai,4,Bengaluru
7,2026-10-04,cappuccino,2,Bengaluru
8,2026-10-04,espresso,1,Bengaluru
"""

orders = pd.read_csv(StringIO(csv), parse_dates=["date"])
print(orders.head(3))
orders.info()

parse_dates turns the date column into real dates instead of text. With your own file, use pd.read_csv("orders.csv", parse_dates=["date"]).

head() shows the first rows. info() lists each column, its type and how many values are not missing. You should see 8 rows, and qty and city should each show 7 non-null values. That tells you each has one missing value. (Exact type names such as object or str depend on your pandas version.)

print(orders.describe())     # count, mean, min, max... for number columns
print(orders.isna().sum())  # missing values per column

describe() gives a quick feel for the numbers. isna().sum() counts missing values in each column: here 1 for qty and 1 for city.

You have three choices for missing data: drop the rows, fill them with a sensible value, or leave them and use a method that copes. The right choice depends on why the value is missing. For a missing quantity, the median is a safe fill. For a missing city, a label like "Unknown" keeps the row.

orders["qty"] = orders["qty"].fillna(orders["qty"].median())
orders["city"] = orders["city"].fillna("Unknown")
print(orders.isna().sum().sum())   # 0

The median of the known quantities is 2, so order 5 now has qty 2. The last line prints 0 because no missing values remain.

print(orders["qty"])                      # one column (a Series)
print(orders[["order_id", "item_id"]])    # several columns: note the double brackets
print(orders[orders["qty"] >= 3])         # rows where qty is at least 3
print(orders[(orders["item_id"] == "espresso") & (orders["qty"] > 1)])
print(orders.sort_values("qty", ascending=False).head(3))

When you combine conditions, wrap each in parentheses and use & for and, | for or. Writing and / or here causes an error.

Orders only contain item_id. Prices live in the menu table. A merge joins them on the shared column, like a SQL JOIN.

menu = pd.DataFrame({
    "item_id": ["espresso", "cappuccino", "cold-brew", "filter-coffee",
                "masala-chai", "croissant", "banana-bread", "veg-sandwich"],
    "name": ["Espresso", "Cappuccino", "Cold Brew", "Filter Coffee",
             "Masala Chai", "Butter Croissant", "Banana Bread", "Veg Grilled Sandwich"],
    "category": ["coffee", "coffee", "coffee", "coffee", "tea", "bakery", "bakery", "food"],
    "price": [120, 180, 220, 80, 90, 150, 130, 160],
})

df = orders.merge(menu, on="item_id", how="left")
df["revenue"] = df["qty"] * df["price"]
print(df[["order_id", "name", "qty", "revenue"]])

how="left" keeps every order even if its item is not found in the menu. A new column revenue is created by multiplying two columns, row by row, with no loop.

The qty column shows 2.0 rather than 2 because a column with a missing value was stored as decimals.

groupby is the most useful pandas idea. It splits the table by a column, applies a calculation to each part and combines the answers: split, apply, combine.

by_item = df.groupby("name")["revenue"].sum().sort_values(ascending=False)
print(by_item)

daily = df.groupby("date").agg(orders=("order_id", "count"),
                               revenue=("revenue", "sum"))
print(daily)

print(df["category"].value_counts())

Cold Brew leads with 660, then Espresso 600 and Cappuccino 540. The daily table has 2 orders each day, with revenue 420, 960, 600 and 480. value_counts() counts rows per category: coffee 6, bakery 1, tea 1.

I have a pandas DataFrame called [df] with these columns: [list columns and their meaning]. Example rows:

[paste 3 rows]

Write pandas code to: [describe the task in plain English].

Requirements: use only pandas and NumPy, explain each line in a comment, and show what the result should look like for my example rows.

What to expect: Run the code on your data and check one or two numbers by hand before trusting the whole result.