Rate limiting

Rate limiting restricts how often a client (identified by API key, user id or IP address) can call an API, for example 100 requests per minute. Requests over the limit are rejected, typically with HTTP 429 Too Many Requests and a Retry-After header.

It protects against abuse (brute-force logins, scraping), keeps one noisy customer from starving others, controls costs of expensive operations, and enforces pricing tiers. It can run in the client, at an API gateway or edge, or inside each service.

1. 1Identify the client for each request (API key, user id, or IP for anonymous traffic) and look up its rule, such as 10 requests per second with bursts up to 20.
2. 2Token bucket: each client has a bucket of capacity C that refills at R tokens per second. Each request takes one token; if the bucket is empty the request is rejected. This allows short bursts up to C while enforcing an average of R.
3. 3Leaky bucket: requests enter a queue that drains at a fixed rate, smoothing output; a full queue rejects new requests.
4. 4Fixed window counter: count requests per calendar window (for example per minute). Simple, but a client can send up to twice the limit around a window boundary (end of one minute plus start of the next).
5. 5Sliding window log stores each request timestamp and counts those in the last 60 seconds: exact but memory-heavy. Sliding window counter estimates with two counters: count = current window + previous window x (fraction of the previous window still inside the sliding range).
6. 6In a distributed system, keep counters in a shared store like Redis and update them atomically (a Lua script or INCR with EXPIRE) so all API servers see the same count.
7. 7Return 429 with Retry-After and rate-limit headers, and decide fail-open (allow traffic if the limiter is down) or fail-closed (reject) per endpoint.

import time

class TokenBucket:
    def __init__(self, capacity, refill_per_sec):
        self.capacity = capacity
        self.rate = refill_per_sec
        self.tokens = capacity
        self.last = time.monotonic()

    def allow(self):
        now = time.monotonic()
        # Refill lazily for the time elapsed, never above capacity
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.rate)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

# One bucket per client key
buckets = {}

def is_allowed(client_id, capacity=5, rate=1.0):
    bucket = buckets.setdefault(client_id, TokenBucket(capacity, rate))
    return bucket.allow()