Design a payment gateway for merchants (Razorpay)

In India, the same payment can often go through several acquiring banks or PSPs, and success rates vary by issuer, card type, method and time of day; a bank's server can degrade for 20 minutes during a sale. A few percentage points of success rate is real revenue for merchants, so routing is a core feature. The router runs in-process on the payment service with config and stats in memory, so a decision takes under 5 ms. Step one: eligibility, from rules (merchant category, method, card network, amount limits, merchant's own acquirer contracts). Step two: scoring, using live success rates per (method, issuer bank, terminal) over sliding windows of 1, 5 and 30 minutes computed by Flink from payment outcome events and pushed to Redis. Blend success rate, latency and cost. Step three: exploration, sending a small share of traffic (a few percent) to non-preferred terminals so the system notices when a degraded terminal recovers, like a multi-armed bandit. Health checks open a circuit on a terminal with error spikes. Be careful to count only failures caused by the bank or network, not user drop-offs or wrong OTPs, otherwise the router punishes healthy terminals. Retry on another terminal is possible only for some failures and only before the customer is charged; never retry when the result is unknown.

Bank and NPCI systems often reply late or not at all. A UPI payment can show failed or pending on our side while the customer's account was debited; a card authorisation can time out after approval. Handle three cases. First, pending/unknown: do not mark failed; poll the bank or PSP status API with backoff and within their rate limits (NPCI limits how often status checks can be made), and keep the checkout showing 'processing' for a while. Second, late success after the order was cancelled or the customer paid again: the payment becomes captured; for merchants with auto-refund settings, a job refunds late successes that were not used, otherwise the merchant is told via webhook. Third, end-of-day truth: every bank and network sends MIS/settlement files. Recon matches each line to a payment by gateway_ref or RRN/ARN and amount, and builds exception queues: debited but failed in our system, success but missing in file, amount mismatch. Most exceptions resolve automatically by moving the payment to the correct state and posting ledger corrections; customers who were debited for a failed payment get auto-refunds within the RBI turnaround time (TAT) framework, which also mandates compensation for delays.

Gateways in India must keep merchant funds in a nodal or escrow account and settle according to RBI's payment aggregator guidelines. The ledger is the source of truth for what each merchant is owed. Every captured payment posts: debit receivable from acquirer, credit merchant balance (gross minus fee), credit fee revenue, credit GST payable. Refunds reverse the merchant part. When an acquirer credits the escrow account, recon moves the money from receivable to escrow cash. Settlement runs per cycle: for each merchant, sum settleable balance (payments captured before the cut-off and already received from acquirers), subtract holds for disputes or risk reserves, and create a payout. Payouts are idempotent by (merchant, cycle) so a re-run never pays twice. Instant settlement is a product on top: pay from the gateway's own working capital before acquirers settle, for a fee, which is credit risk and needs limits. Daily, the escrow bank statement must equal the sum of ledger escrow entries; any difference is a high-severity alert.

A few merchants (a ticketing site during a cricket ticket drop, an e-commerce sale at midnight) can send 50x normal traffic. Protect everyone else. Rate-limit and quota per merchant at the edge with burst allowances agreed in advance; isolate the biggest merchants into dedicated shards or cell pools so a surge does not exhaust shared database connections; use per-bank bulkheads so a slow bank's adapter cannot consume all threads. Pre-scale before known events and run load tests with banks, since bank capacity, not ours, is often the limit; the router can spread load across acquirers to avoid tripping a single bank's limits. Keep checkout static assets on a CDN. Shed non-critical work first under pressure: dashboards, reports and analytics consumers can lag; payment creation and callbacks cannot.

Fixed priority lists are easy to reason about and match contract cost, but react slowly to bank outages. Live scoring lifts success rate but needs reliable stats, careful failure classification and exploration traffic. Most gateways combine rules for eligibility and cost with live scoring.

Auto-capture reduces unclaimed authorisations and is the default for most merchants. Manual capture lets merchants confirm stock first, but uncaptured payments must be auto-refunded after a window, which customers dislike.

Instant settlement improves merchant cash flow but the gateway pays before receiving funds, taking credit and fraud risk. Offer it with limits based on merchant history and refund rates.

Webhooks are timely but can be delayed or lost on the merchant side. Provide both: at-least-once signed webhooks with retries and a status API, and tell merchants to verify signatures and handle duplicates.

During a sale, one issuer or acquirer hits its TPS limit and starts timing out. The router spreads traffic, circuit-breaks the degraded terminal and uses exploration to detect recovery.

One giant merchant overwhelms a shard. Move large merchants to dedicated cells, and partition their payments by payment_id within the cell.

File sizes jump 10x. Run recon in Spark over S3 files partitioned by bank and date, and match with hash joins rather than per-row DB lookups.

The payment stays pending; status pollers query the bank with backoff. End-of-day MIS files settle the final state. Customers debited for failed payments get auto-refunds within TAT.

Deliveries retry with exponential backoff for up to 24 hours; after that the merchant is alerted and can fetch events via API. Other merchants are unaffected because each merchant has its own delivery queue.

Stats go stale; the router detects stale timestamps and falls back to static priority rules with health checks, so payments continue at a slightly lower success rate.