Design a unique ID generator (like Snowflake)

Database auto-increment is simple and strictly ordered, but one database becomes a bottleneck and single point of failure, and with multiple primaries you need tricks like odd/even offsets that do not scale. A ticket server (Flickr's design: a MySQL table using REPLACE INTO to bump a counter, two servers with different offsets) works but adds a network hop per ID. Range leasing (a service hands out blocks of 10,000 IDs) removes the per-ID hop, but IDs are only ordered within a block. UUIDv4 needs no coordination at all, but is 128 bits, random, and bad for B-tree primary keys: inserts land on random pages, causing page splits, write amplification and low cache hit rates; MySQL InnoDB, which clusters rows by primary key, suffers most. UUIDv7 (RFC 9562, 2024) and ULID put a millisecond timestamp first, fixing index locality, but are still 128 bits. Snowflake gives 64-bit, time-ordered, coordination-free IDs: generation is a few hundred nanoseconds of local CPU. Choose Snowflake when you need 64-bit integers and high throughput; choose UUIDv7 when 128 bits is fine and you want zero setup.

Uniqueness depends entirely on no two live generators sharing a worker ID at the same millisecond. Static config (worker ID in a config file) breaks with autoscaling and Kubernetes pods that come and go. Using the last bits of an IP address can collide across subnets or after IP reuse. The safe approach is a lease in a strongly consistent store such as etcd or ZooKeeper: create /id-workers/17 only if absent, attach a 30 s lease, and renew every 10 s. If the generator loses contact with etcd and its lease might have expired, it must stop issuing IDs before the lease TTL passes (fencing by time), because another pod may have taken worker 17. To reduce reliance on etcd, generators keep their worker ID across restarts on the same node, and use etcd only at startup and for renewal; a short etcd outage does not stop generation as long as the lease has not expired. With 1,024 IDs available and a few hundred pods, there is room; if you need more, take bits from the sequence (e.g. 12 worker bits, 10 sequence bits = 1,024 IDs/ms/worker).

Snowflake trusts the system clock, and clocks misbehave. NTP can step the clock backwards after a large drift, a VM can pause and resume, and leap seconds can repeat a second on some systems. If the clock goes back 10 ms and the generator keeps going, it can reissue timestamps it already used, and with a reset sequence it can create duplicate IDs. Defences: configure chrony/NTP to slew (adjust gradually) rather than step; keep last_ms in memory and never issue an ID with a timestamp lower than last_ms; for small backward jumps, wait; for large ones, either keep using last_ms with an incrementing sequence (borrowing future time) or stop and alert. Persist last_ms periodically so a restart after a clock reset does not reuse time. Use a monotonic clock source for measuring progress plus wall-clock for the initial value. Also note that cross-machine ordering is only as good as clock sync, usually within a few ms, so IDs are k-sorted, not strictly ordered: never use them to decide which of two events in different services came first.

A library has no network hop and no extra service to run, but needs worker ID management in every service and versions in every language. A central gRPC service is easier to govern and change, but adds ~0.5-1 ms and becomes critical infrastructure; batching (100 IDs per call) and client-side buffers reduce the cost. Many companies use the library for core services and the service for the long tail.

Snowflake halves key size (8 vs 16 bytes, which matters across billions of rows and every secondary index) and fits JavaScript only up to 2^53 safely, so APIs should send it as a string. UUIDv7 needs no worker IDs at all and has 74 random bits, so collisions are practically impossible without coordination. If your team cannot safely run worker leases, UUIDv7 is the safer choice.

Time-ordered IDs reveal creation time, and sequential patterns reveal business volume (competitors can estimate orders per day). That is usually acceptable for internal IDs; for public-facing IDs, expose a separate random or encrypted identifier, and never use any ID as an access token.

If one ID service instance serves a big batch job, it can hit 4,096 IDs/ms and spin. Spread callers across instances, use batches, or give hot services their own in-process generators. In practice few systems reach this limit.

Frequent pod churn with slow lease expiry can use up 1,024 IDs. Use shorter leases, reuse IDs per node, or reallocate bits.

Running generators continue while their lease is valid, and can keep generating safely as long as they stop before a possible lease expiry. New pods cannot start generating until etcd returns; keep a warm pool of running generators so scale-out is not blocked.

The generator detects now < last_ms, refuses to issue IDs (or uses logical time), fires an alert and removes itself from the load balancer. Callers retry on another instance. A few failed requests are far better than duplicate payment IDs.