Security pitfalls in AI-written code

AI writes code that works. It does not reliably write code that is safe. It has learned from millions of examples, including insecure tutorials and quick demos, and it will happily copy their shortcuts. If you ship AI code without checking it, you may ship its security holes too.

This lesson covers the mistakes that appear most often, shows what they look like in the Brew & Bean project, and gives you a checklist and prompts to catch them. It is defensive only: how to protect your own apps.

There are three reasons. First, models optimise for 'it runs', and the insecure version is often shorter. Second, they imitate their training data, which includes old and sloppy code. Third, they only see your prompt, not your threat model: if you did not say that real customers will use the app, the model assumes a demo.

The fix is not to stop using AI. It is to ask for secure code explicitly, review for the usual suspects, and never trust the AI's claim that code is secure.

A secret is anything that gives access: API keys, database passwords, tokens. AI examples often paste a key straight into the code, such as api_key = "sk-...". If that file goes to GitHub, bots can find the key within minutes. This is one of the most common ways people get surprise bills.

Keep secrets in environment variables, load them in code, and keep the file that holds them out of Git.

# .env  (stays on your machine, never committed)
DATABASE_URL=postgresql://user:password@host/dbname
SECRET_KEY=change-me-to-a-long-random-value

# .gitignore  (add this line BEFORE your first commit)
.env

The .env file holds real values on your machine. .gitignore tells Git never to commit it. Hosting services let you set the same variables in their dashboard.

# config.py
import os

DATABASE_URL = os.environ["DATABASE_URL"]   # fails loudly if missing
SECRET_KEY = os.environ["SECRET_KEY"]

os.environ["NAME"] raises an error if the variable is missing, so you notice at start-up instead of running with a blank key.

Everything the browser sends can be changed by the user, using developer tools or a tool like curl. The rule: validate everything on the server. Check types, lengths, ranges and allowed values, and never rely on the website's form checks alone.

The classic example in a shop is the price. If the browser sends the price or total, a user can change it to 1 rupee.

# BAD: the server trusts the price and total sent by the browser
class OrderIn(BaseModel):
    customer_name: str
    items: list[dict]   # {"item_id": "espresso", "qty": 1, "price": 1}
    total: float        # the browser says: 1

This is a very common AI-generated shape. Anyone can send total: 1 for any order.

# GOOD: the browser sends only item ids and quantities.
# The server looks up prices itself and computes the total.
MENU = {"espresso": 120, "cappuccino": 180, "masala-chai": 90}

class OrderItem(BaseModel):
    item_id: str
    qty: int = Field(ge=1, le=20)

class OrderIn(BaseModel):
    customer_name: str = Field(min_length=1, max_length=60)
    items: list[OrderItem] = Field(min_length=1)

def compute_total(items: list[OrderItem]) -> int:
    total = 0
    for it in items:
        if it.item_id not in MENU:
            raise HTTPException(status_code=422, detail="Unknown item")
        total += MENU[it.item_id] * it.qty
    return total

Prices come from the server's own menu (later, from the database). Pydantic's Field limits reject empty names, zero or huge quantities. This is the design used in *Design the order API with AI*.

SQL injection happens when user text is joined into a SQL query. A phone number such as x' OR '1'='1 changes what the query means and can expose every order. Never build SQL with f-strings or +. Use parameters, or an ORM like SQLAlchemy, which does it for you.

# GOOD: the value is passed separately as a parameter
from sqlalchemy import text

@app.get("/orders")
def find_orders(phone: str):
    query = text("SELECT * FROM orders WHERE phone = :phone")
    return db.execute(query, {"phone": phone}).fetchall()

The database receives the query and the value separately, so the value can never be read as SQL.

Cross-site scripting (XSS) means a user's text runs as code in someone else's browser. Imagine a customer writes a note on an order, and the staff screen shows it as HTML. A note containing a script tag would run on the staff member's screen. React escapes text by default, so you are safe unless you opt out with dangerouslySetInnerHTML. The name is a warning. AI sometimes uses it to render text with line breaks or formatting.

// BAD in React: renders customer-supplied text as raw HTML
<div dangerouslySetInnerHTML={{ __html: order.note }} />

// GOOD: React escapes text for you
<div>{order.note}</div>

If you really need to show formatted content, sanitise it with a library built for the job; don't write your own filter.

AI builds the happy path: an endpoint that does the thing. It often forgets to ask who is allowed to do it. In Brew & Bean, PATCH /orders/{id}/status should be for staff only. If anyone can call it, any visitor can mark orders as picked up or cancel them.

Also check ownership: GET /orders/{id} should not let one customer read another's order just by changing the number in the URL. Using hard-to-guess ids (UUIDs) helps but is not a replacement for a check.

If you add logins, use a proven library or your host's auth service, and store passwords only as hashes made by a standard library (bcrypt or argon2). Never write your own password storage or encryption.

CORS controls which websites may call your API from a browser. When a CORS error blocks you in development, AI often suggests allowing everything. That removes a protection. Also turn off debug mode and detailed error pages in production, because stack traces reveal your code and file paths.

# BAD for production: any website may call your API from a browser
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])

# BETTER: only your own site
app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://your-coffee-site.example"],
    allow_methods=["GET", "POST", "PATCH"],
    allow_headers=["Content-Type", "Authorization"],
)

Allow only the origins you actually use. During development you may add http://localhost:5173 (Vite's default) as well.

AI sometimes recommends a package that does not exist, or one with a name very close to a real one. Attackers can publish packages with such names, hoping you install them. This is called slopsquatting. Before installing a package AI suggested, search for it on npm or PyPI, check that it is well known (downloads, age, a real repository), and ask whether you need it at all. Also run npm audit now and then to see known problems in your dependencies.

Act as a security reviewer for a small web app. Review the code below for these issues: hard-coded secrets, missing server-side validation, SQL injection, XSS, missing authentication or authorisation, over-permissive CORS, debug settings, and unsafe dependencies. For each finding give: where it is, why it is a problem, and a fixed version of the code. If you find nothing in a category, say so. Do not invent problems.

Context: [what the app does, who uses it, where it is deployed]

```
[paste code]
```

What to expect: A review by AI is a second pair of eyes, not a guarantee. It misses things and sometimes flags false problems. Check each finding yourself.

A note on privacy: the order form collects phone numbers. In India the Digital Personal Data Protection (DPDP) Act applies to handling personal data. If you run a real service, collect only what you need, say why, and protect it. For a learning project this is good practice; for a real business, get proper advice.

You may be asked 'What is SQL injection and how do you prevent it?' or 'Where do you store API keys?'. Good answers are short and concrete: parameterised queries, environment variables, server-side validation. Saying 'I review AI-generated code for these issues' also shows maturity. The Security track goes deeper on these topics.