Design an online judge (LeetCode)

Every submission is hostile until proven otherwise: fork bombs, infinite memory allocation, reading /etc or test files, opening sockets to mine crypto or exfiltrate data, or exploiting the kernel. Plain Docker containers share the host kernel and are not a security boundary on their own. Use defence in depth. Isolation: gVisor (a user-space kernel that intercepts syscalls) or Firecracker microVMs (lightweight VMs with ~125 ms boot) give a much smaller attack surface; nsjail or isolate (used by many contest systems) combine Linux namespaces, seccomp-bpf syscall filters and cgroups. Limits via cgroups v2: CPU time, memory (memory.max, then MLE), pids.max to stop fork bombs, file size and output size. No network namespace interfaces at all. Filesystem: read-only base image per language, a small tmpfs as the working directory, only the current test's input mounted; expected outputs and other tests stay outside the sandbox. Run as an unprivileged user with no capabilities. Destroy the sandbox after each submission. Finally, run judge hosts in an isolated network segment with no credentials on the machine, so even a full escape finds little to steal.

Users notice when the same code gets TLE once and AC the next time, and in contests it changes rankings. Sources of noise: other jobs on the same machine competing for CPU cache and memory bandwidth, CPU frequency scaling, and wall-clock delays from I/O. Mitigations: measure CPU time (user + sys from cgroup accounting), not wall time, for the limit, but also enforce a wall-clock limit (2-3× CPU limit) so sleeping programs cannot hang a worker. Pin each sandbox to dedicated physical cores (cpuset) and run fewer concurrent sandboxes than cores, avoiding hyperthread siblings. Use a homogeneous instance type for the contest pool and disable turbo or fix frequency where possible. Language multipliers (Java and Python get more time) or per-language limits keep it fair across languages. Run each test once normally; if the total is within ~10% of the limit, re-run the slow tests and take the minimum, which removes most flaky TLEs for a small extra cost. Publish the judge hardware and compiler versions so users can reason about limits.

Practice traffic is steady; contests concentrate tens of thousands of users into the same 90 minutes, with peaks at the start (easy problem) and the last minutes. One shared queue would let a contest flood practice users, or worse, let practice traffic delay contest verdicts. Use separate queues: contest, practice-submit and practice-run, with workers that always pull from contest first and with a reserved worker pool for contests. Because contests are scheduled, scale before they start: a scheduled job raises the contest pool to the expected size 15 minutes early (machines need minutes to boot and warm caches), then queue-depth autoscaling (KEDA on queue length) handles the rest. Pre-load test data for that contest's problems onto workers before start. Warm sandbox pools per language avoid ~1 s startup per job. Under extreme load, degrade gracefully: 'Run' with custom input is rate-limited first, then practice is paused, and contest submissions keep flowing. Show queue position to users; transparency reduces resubmits that make the queue worse.

Output comparison is not a simple byte compare. Default comparators are token-based (ignore trailing whitespace and line endings) because Windows and Linux line endings otherwise cause angry Wrong Answers. Floating-point answers use an absolute or relative error tolerance such as 1e-6. Problems with many valid answers (print any shortest path) ship a checker program that reads input, expected output and the user's output and decides; checkers run in the sandbox too, with their own limits, since setters make mistakes. Interactive problems need a judge process talking to the user's program over pipes, with both limited. For speed, Submit stops at the first failing test, and tests are ordered with samples and small cases first so most wrong solutions fail in milliseconds. Large inputs (100 MB) stay on local SSD and are passed by file descriptor, not copied. LeetCode-style function problems wrap the user's function in a per-language harness that parses input and serialises output, which must be generated and tested per language.

MicroVMs and gVisor are safer but add overhead (gVisor slows syscall-heavy programs; Firecracker adds boot time, mitigated with warm pools and snapshots). nsjail-style sandboxes are fast and proven in contest systems. For a public site where anyone can submit, pay for the stronger boundary.

Holding the HTTP request open for 5-15 s ties up connections and breaks on retries. Async with WebSocket push (polling as fallback) is more robust and lets the queue absorb spikes.

Redis gives O(log n) updates and rank lookups for 100k users easily. Postgres stays the source of truth so the board can be rebuilt; final standings are frozen from Postgres.

Lambda-style functions scale instantly but have cold starts, inconsistent CPU and per-invocation cost that hurts at contest scale. A dedicated, homogeneous fleet with scheduled scaling gives fairer timing; use spot instances only for practice runs.

3-4k cores needed for a short time. Pre-scale, reserve on-demand capacity for contest pools, and keep practice on spot instances that can be preempted.

100k users refreshing the board. Cache top pages for a few seconds, serve 'around me' with ZREVRANK + ZREVRANGE, and freeze the board in the last 15 minutes like many contests do, which also reduces load.

Hundreds of new workers pulling 2 GB each at contest start can saturate S3 bandwidth for one prefix. Pre-warm caches before the contest and share via a node-local cache.

The job's visibility timeout expires and another worker re-runs it. Judging is idempotent: same code, same tests, same verdict written to the same submission row.

Seccomp violations kill the process with Runtime Error and log a security event; repeated attempts suspend the account. Host-level anomaly detection drains and reimages the machine.

Rebuild the sorted set from contest_scores in Postgres in seconds (100k rows). Users see the board briefly unavailable, never wrong final results.