Design UPI payments (PhonePe / Google Pay)

Many candidates design UPI as if the app debits accounts. It does not. A TPAP like PhonePe or Google Pay is a front-end; regulated PSP banks (e.g. Yes Bank, ICICI, Axis, SBI) sponsor it and connect to NPCI's UPI switch. NPCI routes a payment as a debit request to the payer's (remitter) bank and a credit request to the payee's (beneficiary) bank; banks hold the money and settle among themselves through NPCI's net settlement several times a day. The PIN is captured by NPCI's common library inside the app and encrypted so only the issuing bank can verify it; the TPAP and PSP cannot read it. The VPA handle (@ybl, @ibl, @axl, @okicici and so on) maps the user to a PSP bank. This shapes the design: our database is a record of requests and outcomes, not a ledger of balances; our critical job is to send each request exactly once with a unique transaction ID, track its outcome, and present it honestly. It also explains resilience: apps spread users across several PSP banks so one bank's outage does not stop all payments, and the PSP connector must handle each bank's API quirks and capacity limits.

With hundreds of banks of varying quality, a fraction of a percent of transactions end without a clear answer in time: the payer bank debited but the payee bank timed out, or the response was lost. At 300M transactions a day, 0.5% is 1.5M users who see 'pending'. Rules: never treat a timeout as failure and never let the user retry with a new transaction ID while the first is pending, since that is how double debits happen; show clear status and guidance instead. Use the client_txn_id from the app so network retries of the same tap map to one transaction. Resolve pending transactions by callbacks first, then by status checks through the PSP; NPCI restricts how many status checks can be made and how soon, so the resolver uses a delayed queue with spaced attempts rather than tight polling. Some transactions end as 'deemed approved' when the beneficiary bank's credit is assumed done but confirmation is pending; these are settled through reconciliation. Debits on failed transactions are reversed by the remitter bank, and RBI's TAT framework sets deadlines and compensation for delayed reversals, so the app tracks those deadlines and escalates.

Payments are short-lived requests, but history is forever and read constantly (users open history right after paying). Split the two. The hot transaction store is sharded MySQL/Postgres by txn_id or payer user_id, holding the last 30-90 days with strong consistency and unique constraints. Every status change is an event on Kafka, consumed to update a per-user history table in Cassandra or HBase, partitioned by user_id with time-ordered clustering, written once for the payer and once for the payee if they are our users. Reads of history are a single-partition query, which scales horizontally. Very active merchants have huge partitions, so bucket them by month (user_id + yyyymm). Older data moves to cheaper storage (S3 + Parquet) for analytics and statements. Notifications (push, SMS) also hang off the event stream so the payment path stays short. Capacity planning focuses on peaks: month start, festival evenings and events like Diwali, when TPS can triple in minutes; pre-scale and run load tests with PSP banks, since bank capacity is often the true limit.

UPI fraud is mostly social engineering: fake collect requests ('enter PIN to receive money'), screen-sharing apps, fake customer care numbers, and SIM swap. Controls in the app: device binding tied to SIM, re-binding when SIM or device changes with cooling-off limits on amounts; clear warning screens on collect requests ('You are paying, not receiving'); detection of screen-sharing apps during PIN entry; velocity limits per user and per new payee; risk scoring with features like account age, payee's complaint history and unusual time or amount. The risk service runs inline with a tight latency budget (~20 ms) using precomputed features in Redis/Aerospike, and fails open with conservative rules rather than blocking all payments. Merchant VPAs flagged by many complaints can be blocked or shown with warnings. All payment data stays in Indian data centres per RBI rules.

One PSP bank is simpler to integrate and operate, but its outages take down all payments for the app. Several PSP banks cost more integration and reconciliation work but give resilience and negotiating power. Large apps use several handles and spread users.

Waiting synchronously for the PSP response gives a quick answer for most payments but holds connections for seconds. Callback-driven processing frees resources and handles slow banks better; most designs use a short synchronous wait, then switch to callback + status check, showing 'processing' meanwhile.

Transactions need uniqueness and conditional status updates (SQL). History needs massive, cheap, per-user time-ordered reads (Cassandra/HBase). Using one store for both either loses guarantees or costs too much at 100B+ rows a year.

Failing closed blocks legitimate payments for millions when the risk service is slow. Failing open with simple rules (amount caps for new payees) keeps payments flowing while limiting exposure.

Our backend can scale out; banks often cannot. Spread traffic across PSP banks where possible, pre-agree capacity, and shed non-payment load (balance checks, offers) first during peaks.

A busy merchant gets thousands of payments an hour. Bucket the partition key by month or day, and paginate.

When a bank goes down, millions of transactions become pending and naive pollers flood the PSP. A delayed queue with spaced checks, per-bank rate limits and NPCI's limits prevents a retry storm.

Users on that handle cannot pay via that bank; the app shows a clear message and, where the user has another linked handle or account, suggests it. New users are onboarded to healthy PSPs. Pending transactions resolve when the bank recovers.

The transaction ends as failed or deemed; the remitter bank auto-reverses the debit. The app tracks the reversal, shows it to the user and escalates via the dispute system if the TAT deadline passes.

The client_txn_id is unique per payment attempt, so the backend returns the existing transaction instead of sending a second ReqPay.