Part 4 — Running It in Production
Q Server versus Cloud: Running the Infrastructure
Somebody has to own the machines your ERP runs on, and that somebody is you. This chapter gives you a decision framework for managed platforms versus your own servers, a like-for-like cost model that includes the term most comparisons omit, your time, and a complete, executable walkthrough for running the whole thing yourself if you need to.
In this chapter
- What you need to know first
- The five shapes of infrastructure
- A decision framework you can actually apply
- Cost modeling done properly
- The circumstances that justify leaving managed
- Self-hosting, step by step: provisioning and hardening
- Self-hosting: PostgreSQL
- Self-hosting: reverse proxy, TLS, and DNS
- Self-hosting: running the app and deploying without downtime
- Containers from zero
- Database operations you own regardless of host
- Scaling, in the order it actually happens
- High availability and disaster recovery
- Security operations
- On-call for a very small team
- Data residency and where the servers physically are
- Migrating between managed and self-hosted
What you need to know first
A server is a computer that stays on and answers requests from other computers. That is the entire idea. The laptop you write code on could be a server if you left it plugged in, gave it a permanent address, and never closed the lid. Everything else in this chapter is about buying reliability, network capacity, and somebody else's operational labor on top of that basic idea.
A virtual private server, or VPS, is a slice of a much larger physical machine, carved out by software so that it behaves like its own computer with its own operating system, its own disk, and its own network address. You get root access, root is the all-powerful administrator account on a Linux machine, which means total control and total responsibility. Providers of VPS capacity include Hetzner, DigitalOcean, Akamai (formerly Linode), and the plain compute products of the big clouds (AWS EC2, Google Compute Engine, Azure Virtual Machines).
You reach these machines over SSH (Secure Shell), an encrypted protocol that gives you a command prompt on a remote computer. SSH normally listens on port 22. You will use it constantly in the second half of this chapter.
Managed means the vendor runs the machine and you never touch the operating system. You hand over code or a database schema and the vendor handles patching, hardware failure, capacity, and usually backups. Self-hosted or self-managed means you do all of that. Between those poles sit several intermediate shapes, which is most of what this chapter sorts out.
Platforms, containers, and orchestration
Platform as a Service (PaaS) is the most managed shape for application code. You connect a Git repository, the platform builds your app and runs it, and you never think about a machine. Vercel, Railway, Render, and Fly.io are examples. Infrastructure as a Service (IaaS) is the least managed shape: you rent raw virtual machines, disks, and networks and assemble everything above them. AWS EC2 and a Hetzner VPS are both IaaS, differing mostly in price and the size of the surrounding catalog.
A container is a packaged, isolated copy of your application together with every library it needs, running on a shared operating system kernel — the kernel being the core of the operating system that hands out memory, CPU time, and access to hardware. A container is lighter than a virtual machine because it does not carry its own operating system. Docker is the tool most people use to build and run containers.
Orchestration is software that schedules many containers across many machines, restarts them when they die, and routes traffic to them. Kubernetes is the best-known orchestrator, and later in this chapter I argue that you should not use it.
Proxies, TLS, and DNS
A reverse proxy is a program that sits in front of your application, receives every incoming request from the internet, and forwards it to your app. It terminates encryption, applies rate limits, serves static files, and can spread traffic across several copies of your app. Caddy, nginx, and HAProxy are common choices.
TLS (Transport Layer Security, the thing that makes a URL start with https) encrypts traffic between the browser and your server. It requires a certificate, a signed file proving you control the domain. Let's Encrypt issues these free and automatically. The automation happens over ACME (Automatic Certificate Management Environment), a standard protocol in which your server proves domain control, usually by serving a specific file over port 80 or by publishing a DNS record, and receives a certificate in return, with no human involved.
DNS (Domain Name System) is the internet's address book: it translates erp.yourbrand.com into an IP address. You edit DNS wherever your domain's records are hosted — Cloudflare, your registrar, or your platform. A handful of record types matter here:
- An A record points a name at an IPv4 address.
- An AAAA record does the same for an IPv6 address.
- A CNAME record ("canonical name") points one name at another name rather than at an address, so the target's own records are followed. Platforms hand you a CNAME target for subdomains.
- A TXT record holds arbitrary text and is how email authentication and domain-ownership proofs are published.
- A CAA record lists which certificate authorities are allowed to issue certificates for your domain.
- A PTR record is the reverse lookup: it maps an IP address back to a name, and mail providers check it.
Process management and backup vocabulary
A process manager keeps your application running: it starts it on boot, restarts it if it crashes, and captures its output. On Linux the built-in one is systemd, the program with process ID 1 that starts and supervises every other service on the machine. Use it instead of adding a Node-specific supervisor.
Three backup terms matter and people mix them up constantly. A dump is a logical export of your data — a file that can recreate the database. Continuous archiving is copying the database's write-ahead log (WAL, the sequential record of every change) off the machine as it is produced.
Point-in-time recovery (PITR) means restoring a base backup — a full copy of the database's files, taken while it keeps running, and then replaying WAL on top of it up to a chosen moment — 14:07 last Thursday, one minute before somebody ran the bad UPDATE. A dump alone cannot do that. As the PostgreSQL manual puts it, a dump "represents a snapshot of the database at the time pg_dump began running", so the only moment you can return to is the moment the dump started.
Two planning numbers govern every backup decision. Recovery point objective (RPO) is how much data you accept losing, measured in time. Recovery time objective (RTO) is how long you accept being down. Both are business decisions with price tags.
Finally, on-call means somebody responds when the system breaks outside working hours, and paging means waking them up. For a two-person company that is the difference between a system you can live with and one that eats your life.
Infrastructure choices are labor choices. Every dollar you save by running your own servers is bought with hours of your attention, and your attention is the scarcest resource in a small company. Price the hours before you price the servers.
The five shapes of infrastructure
There are five realistic ways to run an ERP for a wholesale apparel brand. They differ in how much of the stack the vendor operates and how much you do. Learn the shapes and the decision gets much easier.
Shape 1: fully managed platform
Your Next.js app runs on Vercel, your PostgreSQL database on Supabase, your sync service on PowerSync Cloud. You never see an operating system. Deployment is git push. Backups, patching, TLS certificates, hardware failure, and capacity are somebody else's job. What you give up is control over runtime details, arbitrary PostgreSQL extensions and system packages, and predictable pricing at very high usage. What you get is your evenings.
Failure modes:
- the vendor has an outage and you can only wait;
- the vendor changes pricing or deprecates a feature you depend on;
- a runaway query generates a usage bill you did not expect.
All real, all survivable. The usage-bill risk is the one to actively manage, and both vendors give you controls. On Supabase, spend caps are on by default on the Pro plan; leave them on until you have a reason to lift them, and note that read replicas, extra copies of the database that answer read-only queries, are billed outside the cap.
On Vercel, new teams start with spend-management notifications switched on at a spend amount of $200 per billing cycle. That amount only triggers warnings by default. If you also want the platform to stop spending, switch on the separate option that pauses the production deployment of every project when the amount is reached. Check both dashboards on day one. (Vendor controls checked July 2026.)
Shape 2: managed database, self-managed app hosting
The database stays on Supabase (or Neon, AWS RDS, or Crunchy Bridge). The app moves to a container host you configure more directly — Fly.io, Railway, Render, or a plain VPS running Docker. This is the most common first step away from full management, usually taken for one of three reasons:
- a long-running process the PaaS does not support well,
- a need for control over the runtime,
- or a compute bill that got silly.
Keeping the database managed is the important part. It is where the irreplaceable data lives, where the hardest operational skill is required, and where mistakes are permanent. Application servers are replaceable. A corrupted database with no recoverable backup ends the company.
Shape 3: containers on a cloud provider
You build Docker images and run them on a managed container service: AWS ECS with Fargate, Google Cloud Run, Azure Container Apps. The provider runs the machines and schedules your containers. You own the image, the configuration, the networking, and the access-policy sprawl.
Two acronyms show up immediately here. A VPC (virtual private cloud) is your own private network inside the provider's data center, carved into subnets with firewall rules called security groups. IAM (identity and access management) is the permission system that decides which machine, person, or service may call which API.
This shape is common in companies with a platform engineer and rare in companies without one, because VPCs, subnets, security groups, load balancers, task definitions, and roles take weeks to learn. (A load balancer is a machine that spreads incoming requests across several copies of your app. A task definition is the file that tells AWS how to run one container: which image, how much memory, which environment variables.)
Google Cloud Run is the gentlest of the three, because it will run a container from a URL with almost no surrounding configuration.
Shape 4: a plain VPS you administer
You rent one or two Linux machines, install PostgreSQL and Node yourself, put a reverse proxy in front, and deploy with a script. This is the cheapest shape in dollars and the most expensive in hours. It teaches you the most, and it is the one this chapter documents completely. Failure modes:
- you forget to patch and get compromised;
- the disk fills and PostgreSQL stops accepting writes;
- the host has a hardware failure at 3am during market week — the two or three weeks each season when buyers travel in and place the orders that make your year;
- your only backup turns out to be unrestorable.
Each is preventable with the procedures below, and each requires you to actually run them.
Shape 5: on-premise hardware at the brand's office
A physical server in a cupboard behind the sample room. For a modern web ERP this is almost always the wrong choice, and the reasons are unglamorous:
- office internet connections have low upload bandwidth and no service guarantee;
- office power is unprotected;
- nobody in the building knows how to replace a failed drive;
- and your sales team needs the system to work from a hotel in Paris during market week.
The narrow cases where it still makes sense:
- a warehouse that must keep scanning when the internet link drops (but the right answer there is offline-first clients from chapter 6 plus cloud servers),
- a contractual requirement that data never leave a building,
- or a workload so large and stable that hardware economics flip.
None describes a wholesale apparel brand.
The failure is dull and specific. A cleaner unplugs the server to plug in a vacuum. The machine comes back up; the RAID array does not. (RAID means several disks combined so that one can fail without losing data. It protects against a dead drive. It does nothing about a sudden power cut mid-write.) The backup turns out to be on a USB drive plugged into that same server, in that same cupboard. Office hardware inherits every risk of the office, including the ones nobody thought to write down.
A decision framework you can actually apply
Run your situation through these criteria before you touch a dashboard. The answers are usually obvious once written down, which is the point of writing them down.
Team size and on-call capacity. How many people can competently log into a Linux server at 2am and diagnose a stuck database? If the answer is zero or one, you must be on a managed platform. One person cannot form a rotation, and that person will eventually be on a plane, asleep, or ill.
Compliance requirements. Wholesale apparel brands rarely handle regulated data beyond personal information about employees and buyer contacts, plus payment data that should never touch your servers at all. (Chapter M covers the legal frame. Keep card handling with Stripe or your factor, the finance company that buys your invoices and chases payment, so card numbers never touch your systems and the PCI DSS card-security rules stay out of your scope.)
If a large retail customer sends you a security questionnaire, managed vendors with SOC 2 reports make answering it dramatically easier. SOC 2 is an audit report, written by an accounting firm, describing how a vendor controls security and availability. Customers ask for it because an independent auditor has checked the claims. You cannot produce a SOC 2 report for a VPS you administer yourself.
Data residency. If you sell into the EU and process personal data of EU residents, where the database physically sits matters. Managed vendors let you pick a region at project creation and generally will not let you change it afterwards without a migration. Get this right on day one — the section near the end of this chapter goes into detail.
Cost and the failure modes you can tolerate
Cost at your actual scale. Not at the scale you imagine in three years. A wholesale brand with 25 internal users, a few hundred retail customers, and a few million rows is a small database by any modern measure. Managed pricing at that scale is a rounding error against payroll.
Failure modes you can tolerate. Write down what happens to the business if the ERP is unavailable for one hour on an ordinary weekday, for one hour during market week, and for a full day. For most wholesale brands the answers are "annoying", "expensive", and "seriously damaging" respectively. That shape argues for maximum reliability during market and tolerable reliability the rest of the year, which is what managed platforms give you cheaply.
| Criterion | Managed PaaS + managed DB | Managed DB + container host | Self-run VPS | On-premise |
|---|---|---|---|---|
| Minimum viable team | 1 developer | 1 strong developer | 2+ with Linux experience | 2+ plus site access |
| Hours/month of ops work | 1–3 | 4–8 | 8–20 | 15–40 |
| Time to first deploy | Under an hour | Half a day | 1–3 days | 1–3 weeks |
| Security patching | Vendor | Split: you patch images | You, all of it | You, plus firmware |
| Hardware failure | Invisible | Invisible | Host migrates or you rebuild | You drive to the office |
| SOC 2 evidence for customers | Vendor reports | Vendor DB report only | None; you write policies | None |
| Realistic RTO | Minutes | Minutes | 1–4 hours | 4–48 hours |
| Cost advantage | None at small scale | Slight at medium scale | Large in dollars, negative in hours | Negative overall |
| Vendor lock-in risk | Moderate | Low | None | None |
The hour ranges in that table are working estimates for a system this size, not measured industry data; exercise 2 at the end of the chapter has you replace them with your own stopwatch figures.
Even so, read the table honestly and one conclusion falls out for almost every brand: start fully managed. The dollar difference is small, the hour difference is enormous, and the hours are the thing you cannot buy back.
Move off managed only when a specific, measured problem forces you to, and when you do, move the app before you move the database. Never the database first.
Cost modeling done properly
Most cost comparisons on the internet compare server rental prices and stop there. At the scale a wholesale brand runs at, engineering time dominates the total, and it is usually the line nobody writes down. The model below is like-for-like, and your own time is one of the lines in it.
The workload. One wholesale apparel brand running the ERP in this book:
- 25 internal users,
- 8 iPads syncing at market,
- a 40 GB PostgreSQL database,
- roughly 300 GB of monthly outbound data transfer,
- around 2 million application requests per month,
- one background job worker running pg-boss (a job queue that keeps its queue inside PostgreSQL, covered in chapter 9),
- and 7 days of point-in-time recovery.
The time rate. Use your own loaded cost — salary plus payroll taxes, benefits, equipment, and the share of overhead the business carries for one engineer. Below I use $100/hour as a deliberately conservative placeholder for a founder-engineer's opportunity cost. Substitute your own number and the conclusion gets stronger, not weaker.
Every price below was checked against the vendor's own pricing page in July 2026. Cloud pricing changes frequently. Re-verify before you commit, and treat each figure as a shape rather than a quote. Where a vendor publishes only a metered rate, I show the arithmetic so you can redo it with your own usage.
| Line item (as of July 2026) | Fully managed (Vercel + Supabase) | Managed DB + container host (Supabase + Railway) | Self-run VPS (DigitalOcean) |
|---|---|---|---|
| App hosting | Vercel Pro, 3 seats × $20: $60 | Railway Pro, 2 seats × $20: $40 | App droplet 4 vCPU / 8 GB: $48 |
| App compute (metered) | Included in plan credit | ~1.5 vCPU + 3 GB RAM: ~$61 | Included in droplet |
| Database | Supabase Pro $25 + Large compute $110 − $10 credit: $125 | Same: $125 | DB droplet 4 vCPU / 8 GB: $48 |
| Database disk beyond 8 GB | 32 GB × $0.125: $4.00 | $4.00 | Included in droplet (160 GiB) |
| Point-in-time recovery | Supabase PITR add-on, 7 days: $100 | $100 | pgBackRest to Backblaze B2, 200 GB × $6.95/TB: $1.40 |
| Egress / bandwidth | 300 GB is inside Vercel's 1 TB; Supabase 50 GB over its 250 GB × $0.09: $4.50 | Railway 300 GB × $0.05 = $15.00, plus Supabase $4.50: $19.50 | Included (5,000 GiB per droplet) |
| Offline sync service | PowerSync Pro, from: $49 | PowerSync Open Edition self-hosted, ~0.5 vCPU + 1 GB: ~$20 | PowerSync Open Edition on app droplet: $0 |
| Warm standby / replica | Not needed (vendor HA) | Not needed | Standby droplet 2 vCPU / 4 GB: $24 |
| Monitoring & paging | Better Stack Responder, annual billing: $29 | $29 | $29 |
| DNS & domain | Registrar at cost, amortized: ~$1 | ~$1 | ~$1 |
| Infrastructure subtotal | $372.50 | $399.50 | $151.40 |
| Ops hours per month | 2 h | 6 h | 12 h |
| Engineer time at $100/h | $200 | $600 | $1,200 |
| True monthly cost | $572.50 | $999.50 | $1,351.40 |
| One-off setup hours | 4 h ($400) | 16 h ($1,600) | 40 h ($4,000) |
| First-year total | $7,270 | $13,594 | $20,217 |
The self-run column has the cheapest servers and the most expensive year. That is the whole argument in one table. It also quietly buys you less:
- no vendor SOC 2 report,
- no automatic failover (the platform noticing a dead machine and switching to a healthy one without you),
- no vendor engineer looking at the machine,
- and a warm standby, a second database kept continuously up to date and waiting, that you have to switch over by hand.
Where these numbers come from
Every figure in that table was checked in July 2026, and the arithmetic behind each one is worth walking through. Vercel's Pro plan is a $20/month platform fee that includes one deploying seat, $20 of usage credit, 1 TB of Fast Data Transfer, and 10 million edge requests; extra seats are $20/month each, which is how three seats reach $60.
Railway charges $20 per seat per month and publishes both round monthly rates ($20 per vCPU, $10 per GB of RAM) and the meter behind them, $0.00000772 per vCPU-second and $0.00000386 per GB-RAM-second. Those round figures assume a 30-day month; a 730-hour month is 2,628,000 seconds, so one vCPU works out at about $20.29 and one GB of RAM at about $10.14. Multiply by your own measured averages instead of trusting mine.
DigitalOcean's Basic droplets are $48/month for 4 vCPU / 8 GB / 160 GiB SSD with 5,000 GiB of transfer, and $24/month for 2 vCPU / 4 GB / 80 GiB with 4,000 GiB.
Backblaze B2 storage starts at $6.95 per TB per month with free egress up to three times your average stored volume, then $0.01/GB; Cloudflare R2 is an alternative at $0.015 per GB-month with free egress. Both are object storage: files you upload to a vendor's service and fetch back over the internet, the way Amazon S3 works, rather than a disk attached to your server.
Cheaper alternatives: Hetzner and Fly.io
Hetzner is materially cheaper than DigitalOcean for equivalent specifications, and it is the usual answer when someone says "but I can get a server for a few euros". Their prices load dynamically and their model line-up changes, so I am not quoting specifications here — use the live cloud calculator.
Two billing rules from their own documentation are stable and worth knowing (checked July 2026): automatic backups are billed at a flat 20% of the server's price, and their worked example uses a server at €3.29/month, which gives €3.29 × 0.2 = €0.658/month for backups. Swapping DigitalOcean for Hetzner might roughly halve the $151.40 and would barely move the $1,351.40, which is the point.
Fly.io sits between Railway and a VPS. As of July 2026 their published prices are:
- $2.02/month for the smallest shared-CPU machine (one shared vCPU with 256 MB of memory),
- about $5 per GB of RAM per 30 days,
- $0.15/GB/month for volumes — a volume being a disk that stays attached to a machine and survives restarts —
- $0.08/GB/month for snapshots of those volumes with the first 10 GB free each month,
- and $0.02/GB for outbound transfer in North America and Europe.
Paid support starts at $29/month. It is a reasonable Shape 2 choice.
Put your workload assumptions in a spreadsheet with a cell for hourly rate and a cell for ops hours. When someone proposes moving to save money, change the numbers instead of arguing. The spreadsheet settles it in about four minutes.
What changes at 5× scale
At five times this workload — 200 GB database, 1.5 TB egress, heavier compute — the managed columns grow roughly linearly on compute and disk while the self-run column grows slowly, because a bigger VPS costs much less than five small ones.
Egress is what catches people out on the managed platforms. Supabase egress beyond the included 250 GB is $0.09/GB, and Vercel Fast Data Transfer beyond the Pro plan's included 1 TB is priced by region, from $0.15/GB up to $0.35/GB (both as of July 2026).
At 1.5 TB of Vercel transfer that is 500 GB over the allowance: an extra $75/month at the cheapest regional rate, and more if your traffic lands in the expensive ones, for bandwidth a VPS provider bundles free.
Even then, a brand five times this size has hired somebody, so ops hours become a salary line instead of your own evenings. The crossover point where self-hosting wins moves with headcount. Server prices barely shift it.
The circumstances that justify leaving managed
Be precise about this. Vague dissatisfaction does not count. These are the five real reasons.
1. A measured cost problem with a measured saving. Your platform bill exceeds roughly 15–20% of the fully-loaded cost of the engineer who would run the alternative, and you have modeled the alternative including their time. Below that threshold, migrating loses money.
2. A capability the platform cannot provide. You need a PostgreSQL extension the managed vendor does not allow, a long-running process the platform kills, a specific kernel setting, or hardware the platform does not sell (large local NVMe flash storage, a GPU). Check first whether the vendor has a workaround — most "impossible on managed" claims turn out to be an unread extensions list.
3. A hard data residency or contractual requirement. A customer contract or a regulator requires data to sit somewhere no managed vendor offers, or requires you to hold the encryption keys in a way no vendor supports. This is rare in wholesale apparel and common in defense and healthcare.
4. Vendor risk you cannot accept. The vendor has changed pricing punitively, has an unacceptable outage record for your business, or is small enough that you doubt it will exist in three years. Judge this on evidence.
5. Latency to a physical location. Your workload requires sub-10ms round trips to equipment in a specific building. A warehouse conveyor controller might. An ERP does not.
If none apply, the pull you are feeling is about how you want to see yourself as an engineer, and the engineering itself is not asking for the move. That is a fine motive on a side project and a bad one for the system that books your customers' orders.
Self-hosting, step by step: provisioning and hardening
Everything from here to the end of the container section assumes you have decided, for one of those five reasons, to run your own machine. The walkthrough targets Ubuntu 24.04 LTS, which receives standard security maintenance until May 2029. Ubuntu 26.04 LTS was released in April 2026 with standard security maintenance to May 2031, and the same commands apply (dates from Ubuntu's release-cycle page, checked July 2026). Use an LTS release, never an interim one.
Provision the machine
Create the VPS in your provider's dashboard. Choose the region closest to most of your users, pick the LTS image, and, the setting people get wrong, add your SSH public key at creation time so the machine never has a root password to guess. Enable the provider's snapshots if they are cheap. A snapshot is a frozen copy of the whole disk, handy for a quick rollback, and it does not replace the backup strategy later in this chapter.
Vendor dashboards change constantly, but the concepts are stable: image, size, region, SSH key, firewall, backups. If the UI does not match, search the provider's docs for "create a droplet" (DigitalOcean), "create a server" (Hetzner), or "launch an instance" (AWS).
# On your laptop. Generate a key dedicated to this server.
ssh-keygen -t ed25519 -C "erp-admin-2026" -f ~/.ssh/erp_admin
# Copy the public half to the new machine (root, first login only).
ssh-copy-id -i ~/.ssh/erp_admin.pub root@203.0.113.10
# Log in and confirm you are where you think you are.
ssh -i ~/.ssh/erp_admin root@203.0.113.10
hostnamectl
ssh-keygen creates a key pair: erp_admin is the private half that never leaves your laptop, and erp_admin.pub is the public half you hand out freely. Ed25519 is the current recommended algorithm — short, fast, and strong. When it asks for a passphrase, set one; it encrypts the private key on disk so a stolen laptop is not an instant compromise.
ssh-copy-id appends the public key to the server's authorized keys list. After the third command you should see a shell prompt ending in #, and hostnamectl should print the operating system and kernel version. 203.0.113.10 is a documentation-only address reserved for examples; substitute your server's real IP everywhere it appears in this chapter.
Create a non-root user
# Still as root on the server.
adduser --gecos "" deploy
usermod -aG sudo deploy
rsync --archive --chown=deploy:deploy ~/.ssh /home/deploy/
# Verify from your laptop in a SECOND terminal, before closing
# the first one.
ssh -i ~/.ssh/erp_admin deploy@203.0.113.10 \
'sudo -n true && echo sudo-ok'
adduser creates the account and prompts for a password. Set it to something long and store it in your password manager. You need that password for sudo, the command that runs one command as the administrator, and not for logging in — logins are by key. usermod -aG sudo grants administrative rights. The rsync line copies root's authorized keys to the new user with correct ownership.
The verification command should print sudo-ok; if sudo asks for a password instead, run it without -n and type the password you just set. Keep your root session open until the check succeeds. If you lock yourself out before confirming, you are rebuilding the machine.
Lock down SSH
# /etc/ssh/sshd_config.d/99-erp-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowUsers deploy
X11Forwarding no
MaxAuthTries 3
LoginGraceTime 20
ClientAliveInterval 300
ClientAliveCountMax 2
Modern Ubuntu reads drop-in files from /etc/ssh/sshd_config.d/, so you add a file instead of editing the main config. The filename must end in .conf or it is ignored silently. PermitRootLogin no stops direct root login. PasswordAuthentication no and KbdInteractiveAuthentication no together mean only keys work, which eliminates the overwhelming majority of internet-wide SSH attacks. AllowUsers deploy restricts logins to one account — if you later add a second administrator, add their username here too or they cannot get in. ClientAlive* disconnects sessions whose network has vanished.
sudo install -m 600 -o root -g root /dev/null \
/etc/ssh/sshd_config.d/99-erp-hardening.conf
sudo nano /etc/ssh/sshd_config.d/99-erp-hardening.conf # paste
sudo sshd -t # syntax check; silence means valid
sudo systemctl restart ssh
sudo systemctl status ssh --no-pager
The install line creates the file owned by root and readable only by root before you put anything in it. sshd -t validates the configuration without applying it. Always run it: a typo plus a restart equals a locked door. After the restart, open a fresh terminal and log in again before closing your existing session, so you still have a way back in if something is wrong.
One note specific to Ubuntu 22.10 and later: SSH is socket-activated, meaning systemd itself holds the network port open and starts the SSH service only when a connection arrives. So if you ever change the listening port, editing Port in the SSH config alone does nothing — you must also override ssh.socket.
The simpler advice is to leave SSH on port 22 and rely on keys plus fail2ban. Moving the port hides you from bulk scanning and from nothing else.
Firewall
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable # answer "y" to the disruption warning
sudo ufw status verbose
UFW ("uncomplicated firewall") is a friendly front end to the kernel's packet filter. The default policy denies every inbound connection and allows outbound, then you punch three holes: SSH, HTTP, and HTTPS. Add the allow OpenSSH rule before you enable the firewall, because ufw enable warns that it may disrupt existing SSH connections and then applies the rules immediately.
Port 80 stays open because the ACME HTTP challenge uses it and because you want to redirect plain HTTP to HTTPS. PostgreSQL's port 5432 is deliberately not open — the database should only ever be reachable over localhost (the machine talking to itself, address 127.0.0.1) or a private network. ufw status verbose should list exactly those three allowed services and show the default policies.
If your provider also offers a network-level firewall (DigitalOcean Cloud Firewalls, Hetzner Firewalls, AWS security groups), configure it identically. Two layers is correct: the host firewall protects you if the cloud firewall is misconfigured, and vice versa.
Automatic security updates
sudo apt update && sudo apt install -y unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
That installs the daily upgrade job and walks you through enabling it. Two configuration files control the behavior. The first says how often to run; the second says what may be installed.
# /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::AutocleanInterval "7";
The numbers are days between runs, so "1" means daily and "0" would disable that step. The autoclean line clears the downloaded package cache weekly so it does not eat disk.
# /etc/apt/apt.conf.d/50unattended-upgrades (relevant lines only)
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
"${distro_id}ESMApps:${distro_codename}-apps-security";
"${distro_id}ESM:${distro_codename}-infra-security";
};
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-WithUsers "false";
Unattended-Upgrade::Automatic-Reboot-Time "03:30";
Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Mail "ops@yourbrand.com";
Those three origins are the ones Ubuntu ships enabled by default: the ordinary security pocket plus the two Expanded Security Maintenance pockets, which carry fixes for a wider set of packages if the machine is attached to an Ubuntu Pro subscription. Security fixes apply themselves; feature updates wait for you. Automatic-Reboot "true" with a 03:30 window means kernel updates actually take effect instead of sitting pending for eight months. Automatic-Reboot-WithUsers "false" skips the reboot if somebody is logged in, which prevents rebooting mid-incident. Point Mail at an address you read, and make sure the machine can actually send mail or that line does nothing.
sudo unattended-upgrade -v --dry-run
The dry run simulates the whole process and prints what it would install, changing nothing. You should see it list the configured origins and either name some packages or report nothing to do. Run this once after configuring so you know the config parses. If your app cannot survive an unannounced 03:30 restart, that is a design problem to fix, and not a reason to disable reboots.
fail2ban
sudo apt install -y fail2ban
sudo systemctl enable --now fail2ban
fail2ban reads authentication logs and blocks IP addresses that fail repeatedly. It ships with defaults in jail.conf. Never edit that file — package upgrades overwrite it. Put your settings in jail.local, which overrides it and survives upgrades.
# /etc/fail2ban/jail.local
[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 5
backend = systemd
destemail = ops@yourbrand.com
banaction = ufw
ignoreip = 127.0.0.1/8 ::1
[sshd]
enabled = true
mode = aggressive
maxretry = 3
bantime = 24h
These settings mean: three failed SSH attempts within ten minutes gets you blocked for 24 hours, and the block is applied through UFW so it appears in your firewall rules. ignoreip keeps the machine from banning itself. If you have a static office IP, add it there too, so a fat-fingered morning cannot lock out the whole team. With key-only authentication already enforced, fail2ban is mostly about cutting log noise and CPU waste from constant scanning.
sudo systemctl restart fail2ban
sudo fail2ban-client status
sudo fail2ban-client status sshd
The restart picks up jail.local. The first status command lists active jails; you should see sshd. The second shows how many attempts have been detected and how many IPs are currently banned. On a public IP address the banned count will be non-zero within hours, which is a useful demonstration of why key-only auth matters.
Those steps protect a freshly built machine and nothing more. They do nothing about a vulnerable dependency you installed last month or a port you opened while debugging and forgot. Put a recurring calendar entry to re-run the checks in the security operations section below.
Self-hosting: PostgreSQL
Install
sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt install -y postgresql-18 postgresql-client-18 \
postgresql-contrib-18
psql --version
sudo systemctl status postgresql --no-pager
Ubuntu's own repositories carry an older PostgreSQL. The apt.postgresql.org.sh script adds the official PostgreSQL Global Development Group repository, which is the correct source for production; it will show you what it is about to add and ask for confirmation.
PostgreSQL 18 was released on 25 September 2025 and is supported until 14 November 2030, while PostgreSQL 14 reaches end of life on 12 November 2026, so do not start anything new on 14 (dates from the project's versioning policy, checked July 2026).
postgresql-contrib brings extensions including pg_stat_statements, which records how long each kind of query takes and which you will need later. After installation the service should already be running and psql --version should print 18.x. (psql is PostgreSQL's command-line client — the program you type SQL into.)
Create the database and roles
# Generate three passwords first and store them in your
# password manager. Do not invent them by hand.
openssl rand -base64 24
openssl rand -base64 24
openssl rand -base64 24
# Keep these statements out of your shell and psql history.
export HISTFILE=/dev/null
sudo -u postgres psql --set=HISTFILE=/dev/null <<'SQL'
CREATE ROLE erp_app LOGIN PASSWORD 'PASTE_GENERATED_1';
CREATE ROLE erp_migrate LOGIN PASSWORD 'PASTE_GENERATED_2';
CREATE ROLE erp_readonly LOGIN PASSWORD 'PASTE_GENERATED_3';
CREATE DATABASE erp OWNER erp_migrate;
\c erp
REVOKE ALL ON SCHEMA public FROM PUBLIC;
GRANT USAGE ON SCHEMA public TO erp_app, erp_readonly;
ALTER DEFAULT PRIVILEGES FOR ROLE erp_migrate IN SCHEMA public
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO erp_app;
ALTER DEFAULT PRIVILEGES FOR ROLE erp_migrate IN SCHEMA public
GRANT SELECT ON TABLES TO erp_readonly;
ALTER DEFAULT PRIVILEGES FOR ROLE erp_migrate IN SCHEMA public
GRANT USAGE, SELECT ON SEQUENCES TO erp_app;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
SQL
Three roles, three jobs:
erp_migrateowns the schema and is the only role that can change it, so a compromised application cannot drop a table.erp_appreads and writes rows but owns nothing.erp_readonlyis for your BI tool and ad-hoc analysis.
REVOKE ALL ON SCHEMA public FROM PUBLIC removes the historical default that let any role create objects, and the ALTER DEFAULT PRIVILEGES lines mean future tables get the right grants automatically. This is least privilege — give every account the smallest set of powers that lets it do its job — applied at the database, and it pairs with the row-level security work in chapter 5, where the database itself decides which rows each account may see.
Two handling notes, because this block contains secrets. The lines between <<'SQL' and SQL are a heredoc: the shell feeds them to psql as if you had typed them in. Quoting the marker stops the shell expanding anything inside, and setting HISTFILE keeps the passwords out of both your shell history and ~/.psql_history. Passwords are still visible to anyone who can read the terminal scrollback, so close the window afterwards.
If you prefer never to type a password at all, create the roles without one and set each afterwards with the interactive \password erp_app command inside psql, which prompts without echoing and never writes the value to a file.
Tune for the workload
# /etc/postgresql/18/main/conf.d/10-erp.conf
# Sized for a dedicated 4 vCPU / 8 GB machine.
listen_addresses = 'localhost'
max_connections = 100
shared_buffers = 2GB
effective_cache_size = 6GB
work_mem = 12MB
maintenance_work_mem = 512MB
random_page_cost = 1.1
effective_io_concurrency = 200
wal_level = replica
max_wal_size = 4GB
min_wal_size = 1GB
checkpoint_completion_target = 0.9
wal_compression = on
autovacuum_vacuum_scale_factor = 0.05
autovacuum_analyze_scale_factor = 0.02
autovacuum_naptime = 30s
autovacuum_max_workers = 4
shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.max = 5000
pg_stat_statements.track = top
track_io_timing = on
log_min_duration_statement = 500ms
log_checkpoints = on
log_lock_waits = on
log_temp_files = 0
log_autovacuum_min_duration = 0
log_line_prefix = '%m [%p] %q%u@%d app=%a '
Take these in groups. listen_addresses = 'localhost' means PostgreSQL is unreachable from the network; combined with the closed firewall port, that is two independent protections.
shared_buffers at 25% of RAM and effective_cache_size at 75% are the standard starting ratios — the first is memory PostgreSQL reserves, the second a hint about what the operating system is caching. work_mem is allocated per sort per query, so 12 MB across 100 connections each running several sorts is the worst case; keep it modest and raise it per-session for heavy reports.
random_page_cost = 1.1 tells the planner, the part of PostgreSQL that decides how to execute each query, that reading a page from a random place on disk is nearly as cheap as reading the next one in order. That is true on SSD and false on the spinning disks the default assumed.
The autovacuum settings matter for an ERP with an append-only ledger (a table you only ever add rows to, never edit — chapter 1 builds one for inventory) and busy order tables.
When you update or delete a row, PostgreSQL leaves the old version behind until it is certain no query still needs it; autovacuum is the background job that later reclaims that space, and the leftovers are called dead rows. The defaults wait until 20% of a table is dead rows — two million dead rows of wasted space on a ten-million-row table before anything happens. Dropping to 5% makes vacuum run more often and each run cheaper.
The logging block gives you the raw material for diagnosis: every query over 500ms, every lock wait, every temp file spill, every autovacuum run. Because log_min_duration_statement writes the query text to the log, treat the PostgreSQL log directory as containing potentially personal data and control who can read it.
sudo -u postgres psql -c "SELECT pg_reload_conf();"
sudo systemctl restart postgresql # needed for the settings below
sudo -u postgres psql -c "SHOW shared_buffers;"
sudo -u postgres psql -c "SHOW shared_preload_libraries;"
Most settings reload without a restart. shared_preload_libraries, shared_buffers, wal_level, and max_connections can only be set at server start, so restart once after the initial configuration. The final two commands confirm the new values took effect. If either still prints the old value, your file is in the wrong directory or has a syntax error, and journalctl -u postgresql will say which.
Connection pooling
# /etc/pgbouncer/pgbouncer.ini
[databases]
erp = host=127.0.0.1 port=5432 dbname=erp
[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction
max_client_conn = 500
default_pool_size = 20
reserve_pool_size = 5
reserve_pool_timeout = 3
server_idle_timeout = 600
query_wait_timeout = 20
ignore_startup_parameters = extra_float_digits
PgBouncer accepts many cheap client connections and shares them across a few expensive PostgreSQL connections. pool_mode = transaction hands a real database connection to a client only for the length of one transaction — one BEGIN to one COMMIT, and then gives it to somebody else. That is the right mode for a web app, and it means your code must not depend on anything that outlives a single transaction.
Prepared statements are the usual casualty: a prepared statement is a query the server parses once and then reuses by name, and the name lives on the connection. PgBouncer can track protocol-level named prepared statements itself when max_prepared_statements is non-zero (current versions default it to 200). Supabase's hosted pooler does not, and its documentation tells you to turn prepared statements off in your client library, which is what chapter 10 walks through.
max_client_conn = 500 with default_pool_size = 20 means 500 app connections share 20 database connections. query_wait_timeout = 20 makes queries fail fast when the pool is exhausted rather than piling up invisibly. listen_addr = 127.0.0.1 keeps the pooler off the network, and auth_file holds hashed credentials, so give it chmod 640 and chown postgres:postgres before you start the service.
This mirrors what Supabase does for you with Supavisor: transaction mode on port 6543, session mode and direct connections on 5432. Their published limits are a useful sizing reference. As of July 2026:
- a Micro instance allows 60 direct connections and 200 pooler clients;
- Small 90 and 400;
- Medium 120 and 600;
- Large 160 and 800;
- XL 240 and 1,000;
- 2XL 380 and 1,500.
If you are self-hosting, aim for the same shape: few real connections, many pooled clients.
Self-hosting: reverse proxy, TLS, and DNS
DNS records
Type Name Content Proxy TTL
A @ 203.0.113.10 DNS only Auto
A www 203.0.113.10 DNS only Auto
A erp 203.0.113.10 DNS only Auto
CAA @ 0 issue "letsencrypt.org" Auto
TXT @ "v=spf1 include:_spf.google.com ~all" Auto
TXT _dmarc "v=DMARC1; p=none;
rua=mailto:dmarc@yourbrand.com" Auto
@ means the domain itself (the "apex"); www and erp are subdomains. Set the proxy status to "DNS only" (the gray cloud in Cloudflare) while you issue certificates, because a proxied record hides your origin server and complicates the ACME HTTP challenge.
The CAA record declares that only Let's Encrypt may issue certificates for your domain, which stops a different certificate authority from being tricked into issuing one. The two TXT records are email authentication, covered in the security section below; add the DKIM record your email provider generates for you at the same time.
TTL ("time to live") is how long resolvers may cache the answer — leave it on automatic except in the hour before a planned IP change, when you should lower it to 300 seconds first.
If you host the app on Vercel rather than a VPS, the records differ: the apex gets an A record whose value Vercel shows you in the project's Domains settings, and each subdomain gets a CNAME to a per-project target that looks like d1d4fc829fe7bc7c.vercel-dns-017.com. Read those values from the dashboard rather than copying them from anywhere else, including this book — Vercel has changed the apex IP before and will again. Wildcard domains (*.yourbrand.com) are the one case that requires you to move nameservers to Vercel entirely.
Caddy with automatic TLS
sudo apt install -y debian-keyring debian-archive-keyring \
apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
| sudo gpg --dearmor \
-o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf \
'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
| sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
sudo chmod o+r /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install -y caddy
Caddy is a web server that obtains and renews TLS certificates automatically, with no cron job and no certbot. Those commands are Caddy's own published apt instructions, and two details matter.
The keyring filename must be exactly caddy-stable-archive-keyring.gpg, because the sources list Cloudsmith gives you refers to that path by name — save it as anything else and apt update fails with an unsigned-repository error. The two chmod o+r lines are also required: apt drops privileges when it fetches, so a root-only keyring produces the same failure.
Caddy is chosen over nginx here for one reason: certificate management is the part of self-hosting that most often breaks silently, and Caddy removes it as a category of failure. The Next.js self-hosting guide recommends a reverse proxy in front of the app whichever one you pick, because a proxy "can handle malformed requests, slow connection attacks, payload size limits, rate limiting, and other security concerns, offloading these tasks from the Next.js server."
# /etc/caddy/Caddyfile
{
email ops@yourbrand.com
}
erp.yourbrand.com {
encode zstd gzip
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
reverse_proxy localhost:3001 localhost:3002 {
lb_policy least_conn
health_uri /api/health
health_interval 5s
health_timeout 2s
fail_duration 30s
flush_interval -1
}
log {
output file /var/log/caddy/erp.access.log {
roll_size 50mb
roll_keep 14
}
format json
}
}
The global block sets the contact address Let's Encrypt uses for expiry warnings. Naming the site erp.yourbrand.com is enough for Caddy to fetch a certificate on first start and renew it forever, and to redirect plain HTTP to HTTPS automatically. encode compresses responses.
The header block adds standard hardening:
- HSTS (HTTP Strict Transport Security) tells browsers to refuse plain HTTP for this hostname for a year,
nosniffstops browsers guessing a file's type,DENYforbids putting your pages in someone else's frame,- and
-Serverremoves the version banner.
Be deliberate about includeSubDomains: it applies the rule to every subdomain of the host, so do not add it until every subdomain has a certificate.
The reverse_proxy line lists two upstreams, which is the trick that gives you zero-downtime deploys without extra tooling. Caddy load-balances between two copies of the app, actively health-checks both every five seconds, and takes an unhealthy one out of rotation for 30 seconds. Restart them one at a time and no request is dropped.
flush_interval -1 tells Caddy to pass each piece of the response straight through instead of collecting it first. That is required for streaming — pages that arrive in pieces as the server produces them, which is how React Suspense shows a loading placeholder and then fills it in — to work at all. The Next.js docs call out the same requirement for nginx, where you disable buffering with the X-Accel-Buffering: no header instead.
sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl reload caddy
curl -I https://erp.yourbrand.com
caddy validate parses the file without applying it. reload applies changes with no dropped connections. The curl -I should return HTTP/2 200 along with your security headers once the app is running. If it returns a certificate error, the usual causes are DNS not yet pointing at this machine, port 80 blocked, or a Cloudflare proxy in the way.
Let's Encrypt's published limits as of July 2026 allow:
- 50 certificates per registered domain every 7 days,
- 5 certificates for the same exact set of names every 7 days,
- 5 authorization failures per name per account per hour,
- and 300 new orders per account every 3 hours.
Debugging in a loop can exhaust those quickly, so point Caddy at Let's Encrypt's staging environment while you iterate. Renewals driven by ACME Renewal Information are exempt from the limits, so ordinary operation never hits them.
Self-hosting: running the app and deploying without downtime
Build output
// next.config.ts
import type { NextConfig } from "next";
import path from "node:path";
const nextConfig: NextConfig = {
output: "standalone",
// Monorepo: trace files from the workspace root, not apps/web.
outputFileTracingRoot: path.join(__dirname, "../../"),
deploymentId: process.env.DEPLOYMENT_VERSION,
experimental: {
serverActions: { bodySizeLimit: "4mb" },
},
};
export default nextConfig;
output: "standalone" tells Next.js to emit a self-contained folder under .next/standalone containing a minimal server and only the node_modules actually used. You ship that folder instead of the whole repository, which cuts deployment size by an order of magnitude.
A monorepo keeps several packages — the web app, the database package, shared code — inside one Git repository, and in that layout you must also set outputFileTracingRoot to the workspace root, or the trace misses the shared packages and the app crashes on first request with a missing-module error.
deploymentId protects against version skew, the situation where a browser is still holding files from the previous build while the server has moved on. During a rolling deploy, restarting your instances one at a time so the site never goes fully dark, that browser gets a full page reload instead of a cryptic error.
Next.js also needs NEXT_SERVER_ACTIONS_ENCRYPTION_KEY set to the same value on every instance. The docs specify a base64-encoded value with a valid AES key length — 16, 24, or 32 bytes, and Next.js generates 32-byte keys by default.
A Server Action is a function in your server code that the browser is allowed to call directly, and Next.js encrypts the values it closes over. Without a shared key, a Server Action encrypted by one process cannot be decrypted by the other, and users see "Failed to find Server Action" errors at random.
The monorepo layout has one consequence you have to get right, because getting it wrong produces a site with no CSS and no images. The standalone output preserves the workspace structure, so the entry point is apps/web/server.js, not server.js at the root. Static assets must land at apps/web/.next/static and apps/web/public relative to that root. The unit file and deploy script below use those paths; if your app lives at the repository root instead, drop the apps/web/ prefix everywhere.
A systemd template unit
# /etc/systemd/system/erp@.service
[Unit]
Description=ERP Next.js app on port %i
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/srv/erp/current
Environment=NODE_ENV=production
Environment=PORT=%i
Environment=HOSTNAME=127.0.0.1
EnvironmentFile=/etc/erp/env
ExecStart=/usr/bin/node apps/web/server.js
Restart=always
RestartSec=2
KillSignal=SIGTERM
TimeoutStopSec=30
StandardOutput=journal
StandardError=journal
SyslogIdentifier=erp-%i
# Sandboxing
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
RestrictSUIDSGID=true
ReadWritePaths=-/srv/erp/current/.next/cache
ReadWritePaths=-/srv/erp/current/apps/web/.next/cache
MemoryMax=2G
[Install]
WantedBy=multi-user.target
The @ in the filename makes this a template: erp@3001.service and erp@3002.service are two instances of it, and %i expands to the port. Restart=always means a crash brings it straight back. KillSignal=SIGTERM with TimeoutStopSec=30 gives Next.js the graceful drain window its docs ask for (they recommend 10–30 seconds), so in-flight requests finish and after() callbacks run before the process exits.
HOSTNAME=127.0.0.1 binds the app to localhost only, so it is unreachable except through Caddy. The sandboxing block makes the filesystem read-only apart from the cache directories and prevents privilege escalation. Note the leading - on both ReadWritePaths lines: without it, systemd refuses to start the unit if the directory does not exist yet, which is exactly what happens on your first deploy. MemoryMax=2G stops a memory leak taking the whole machine down with it.
sudo install -d -m 750 -o deploy -g deploy /etc/erp
sudo install -m 640 -o deploy -g deploy /dev/null /etc/erp/env
sudo systemctl daemon-reload
sudo systemctl enable --now erp@3001 erp@3002
systemctl status erp@3001 --no-pager
The first two lines create a secrets directory readable only by the deploy user and an empty environment file to fill in with your DATABASE_URL, API keys, and NEXT_SERVER_ACTIONS_ENCRYPTION_KEY. Mode 640 means the file never becomes world-readable, which matters because everything in it is a credential. daemon-reload makes systemd notice the new unit. enable --now starts both instances and marks them to start at boot. Status should show active (running) for each — the first run will fail until a release exists at /srv/erp/current, which the next section creates.
# /etc/sudoers.d/erp-deploy (install with: visudo -f)
# Let the deploy user restart only the two app instances.
deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart erp@3001, \
/usr/bin/systemctl restart erp@3002, \
/usr/bin/systemctl status erp@3001, \
/usr/bin/systemctl status erp@3002
The deploy script restarts services, and the deploy user cannot do that without permission. Grant exactly these four commands and nothing else — a blanket NOPASSWD: ALL would make the deploy account equivalent to root, which defeats the point of having created it. Always edit sudoers files with sudo visudo -f /etc/sudoers.d/erp-deploy, because visudo refuses to save a file with a syntax error. A broken sudoers file locks every user out of sudo on the machine.
Zero-downtime deploy script
#!/usr/bin/env bash
# /srv/erp/deploy.sh — run as the deploy user
# WARNING: this discards any uncommitted change in /srv/erp/repo.
# That directory is a deploy checkout, not a place to edit code.
set -euo pipefail
REPO=/srv/erp/repo
RELEASES=/srv/erp/releases
STAMP=$(date +%Y%m%d%H%M%S)
TARGET="$RELEASES/$STAMP"
echo "==> fetching"
git -C "$REPO" fetch --prune
git -C "$REPO" reset --hard origin/main
DEPLOYMENT_VERSION=$(git -C "$REPO" rev-parse --short HEAD)
export DEPLOYMENT_VERSION
echo "==> building $DEPLOYMENT_VERSION"
cd "$REPO"
pnpm install --frozen-lockfile
pnpm turbo run build --filter=@erp/web
echo "==> staging release"
mkdir -p "$TARGET"
cp -r apps/web/.next/standalone/. "$TARGET/"
cp -r apps/web/.next/static "$TARGET/apps/web/.next/static"
cp -r apps/web/public "$TARGET/apps/web/public"
mkdir -p "$TARGET/apps/web/.next/cache"
echo "==> migrating database"
set -a; . /etc/erp/env; set +a
pnpm --filter=@erp/db migrate:deploy
echo "==> swapping symlink"
ln -sfn "$TARGET" /srv/erp/current.new
mv -Tf /srv/erp/current.new /srv/erp/current
wait_healthy () {
local port=$1
for i in $(seq 1 40); do
if curl -fsS "http://127.0.0.1:$port/api/health" >/dev/null
then
echo " port $port healthy"; return 0
fi
sleep 1
done
echo " port $port NEVER became healthy"; return 1
}
for PORT in 3001 3002; do
echo "==> restarting $PORT"
sudo systemctl restart "erp@$PORT"
wait_healthy "$PORT"
sleep 6 # let Caddy re-add it before touching the next one
done
echo "==> pruning old releases"
ls -1dt "$RELEASES"/* | tail -n +6 | xargs -r rm -rf
echo "==> deployed $DEPLOYMENT_VERSION"
Read this top to bottom. set -euo pipefail stops the script at the first error instead of carrying on regardless. The git reset --hard is the one destructive line: it throws away anything uncommitted in the deploy checkout, which is correct for a machine that only ever mirrors origin/main and wrong if you have been editing files on the server. Do not edit files on the server.
Each deploy builds into a new timestamped directory, so nothing is overwritten in place. The three cp lines put the standalone output, the static assets, and the public folder in the layout the monorepo build expects; the mkdir creates the cache directory that the systemd unit wants to write to.
Migrations run before the symlink swap, which is why chapter 2's rule about backward-compatible migrations matters: for a few seconds the old code runs against the new schema. A symlink is a path that points at another path, so /srv/erp/current is really a signpost to whichever release directory is live. mv -Tf repoints that signpost in a single step that no other process can catch half-finished, so there is no instant where current does not exist.
Instances then restart one at a time, with a health check that polls for up to 40 seconds and a six-second pause so Caddy re-adds the instance before the other goes down. If wait_healthy fails, set -e aborts the script with the second instance still serving the old code. Keeping five releases gives you a rollback: point the symlink at the previous directory and restart both units.
// apps/web/app/api/health/route.ts
import { NextResponse } from "next/server";
import { pool } from "@erp/db";
export const dynamic = "force-dynamic";
export async function GET() {
const started = Date.now();
try {
const r = await pool.query("SELECT 1 AS ok");
return NextResponse.json({
status: "ok",
db: r.rows[0].ok === 1,
version: process.env.DEPLOYMENT_VERSION ?? "unknown",
ms: Date.now() - started,
});
} catch {
return NextResponse.json(
{ status: "degraded" },
{ status: 503 },
);
}
}
The health endpoint has to actually check something. Returning 200 OK unconditionally is worse than useless, because it tells your load balancer that a broken instance is fine. This version runs a trivial query against the database, so a process that has lost its connection pool reports 503 and gets pulled from rotation. Keep the query trivial: a health check that runs a real report will time out under load and cause the outage it was meant to detect.
The error branch deliberately returns no detail. Database errors routinely contain hostnames, usernames, and fragments of SQL, and this endpoint is reachable from the internet through Caddy — log the exception server-side instead, and never echo it to the caller.
Log rotation
# /etc/logrotate.d/erp
/var/log/caddy/*.log {
daily
rotate 30
compress
delaycompress
missingok
notifempty
create 640 caddy caddy
sharedscripts
postrotate
systemctl reload caddy >/dev/null 2>&1 || true
endscript
}
Caddy writes access logs to files, so logrotate compresses them daily and keeps 30 days. The postrotate reload makes Caddy reopen its file handles, without which it keeps writing to the renamed file and the new one stays empty. Mode 640 keeps request logs off-limits to other users on the box; they contain URLs, and URLs contain customer identifiers.
# /etc/systemd/journald.conf.d/10-erp.conf
[Journal]
Storage=persistent
SystemMaxUse=2G
MaxRetentionSec=30day
Your app writes to standard output, which systemd captures into the journal, so this second file caps the journal at 2 GB and 30 days. Without both mechanisms, a full disk is the single most common self-hosted outage, and it usually arrives during your busiest week. Chapter 9's structured logging guidance applies unchanged: log JSON to stdout and let the platform handle transport.
sudo logrotate -d /etc/logrotate.d/erp # dry run
sudo systemctl restart systemd-journald
journalctl -u erp@3001 -f --output=cat
df -h /
logrotate -d is a debug run that prints what it would do without doing it. The journalctl command tails your app's logs live, which is the command you will type more than any other. df -h / shows disk usage — check it now and set an alert at 80%.
Containers from zero
Images versus containers
An image is a file: a frozen, layered snapshot of a filesystem plus metadata saying what command to run. A container is a running process started from an image, with its own isolated view of the filesystem, network, and process list. The relationship is the same as a class and an object, or a recipe and a meal. Images are immutable and shareable; containers are disposable and can be thrown away without losing the image.
The practical value is that an image built on your laptop contains exactly the same Node version, system libraries, and application code as the one in production, so "works on my machine" disappears as a category. The cost is another layer of tooling to keep patched. A container running a two-year-old base image has two-year-old vulnerabilities inside it, and the host's automatic updates will not touch them.
A Dockerfile for the app
# apps/web/Dockerfile
FROM node:24-bookworm-slim AS base
ENV PNPM_HOME=/pnpm PATH=/pnpm:$PATH
RUN corepack enable
FROM base AS deps
WORKDIR /app
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
COPY apps/web/package.json apps/web/
COPY packages/db/package.json packages/db/
RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm install --frozen-lockfile
FROM base AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=deps /app/apps/web/node_modules ./apps/web/node_modules
COPY --from=deps /app/packages/db/node_modules \
./packages/db/node_modules
COPY . .
ARG DEPLOYMENT_VERSION
ENV DEPLOYMENT_VERSION=$DEPLOYMENT_VERSION
ENV NEXT_TELEMETRY_DISABLED=1
RUN pnpm turbo run build --filter=@erp/web
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production PORT=3000 HOSTNAME=0.0.0.0
ENV NEXT_TELEMETRY_DISABLED=1
RUN groupadd -g 1001 nodejs \
&& useradd -u 1001 -g nodejs -m nextjs
COPY --from=build --chown=nextjs:nodejs \
/app/apps/web/.next/standalone ./
COPY --from=build --chown=nextjs:nodejs \
/app/apps/web/.next/static ./apps/web/.next/static
COPY --from=build --chown=nextjs:nodejs \
/app/apps/web/public ./apps/web/public
USER nextjs
EXPOSE 3000
COPY --chown=nextjs:nodejs apps/web/healthcheck.mjs ./
HEALTHCHECK --interval=10s --timeout=3s --start-period=20s \
CMD ["node", "healthcheck.mjs"]
CMD ["node", "apps/web/server.js"]
This is a multi-stage build. Each FROM starts a new stage, and COPY --from= pulls only finished artifacts forward, so compilers, source code, and dev dependencies never reach the final image. The deps stage installs from just the lockfile and manifests, so Docker reuses that cached layer whenever dependencies have not changed — the difference between a 20-second and a 4-minute build. The runner stage switches to an unprivileged user with USER nextjs, because a container running as root that gets exploited is a much worse day.
Two version notes, both current as of July 2026. Node 24 is an active Long Term Support line; Node 25 is already end-of-life and Node 26 is the Current release, so neither belongs in production. Corepack, which the second line enables, is bundled with Node from 14.19.0 up to but not including 25.0.0 — when you move past Node 24 you will need to install pnpm explicitly instead. The health check lives in its own small file rather than a one-line node -e string, because quoting a JavaScript expression inside a Dockerfile HEALTHCHECK is a reliable source of silent breakage.
// apps/web/healthcheck.mjs
const url = "http://127.0.0.1:3000/api/health";
try {
const res = await fetch(url, { signal: AbortSignal.timeout(2500) });
process.exit(res.ok ? 0 : 1);
} catch {
process.exit(1);
}
Exit code 0 means healthy and anything else means unhealthy, which is the contract Docker and every orchestrator expect. The timeout matters: without it a hung server leaves the health check hanging too, and the container is never marked unhealthy.
# .dockerignore (at the repository root)
node_modules
**/node_modules
.next
**/.next
.git
.env
.env.*
*.log
coverage
This file is not optional. Without it, COPY . . in the build stage copies your laptop's node_modules over the ones installed inside the image, which at best wastes minutes and gigabytes and at worst produces a broken image with binaries compiled for macOS. The .env entries matter more: they stop your real secrets being baked into an image layer, where they survive forever and are readable by anyone who can pull the image.
SHA=$(git rev-parse --short HEAD)
docker build -f apps/web/Dockerfile \
--build-arg DEPLOYMENT_VERSION="$SHA" \
-t "erp-web:$SHA" .
docker images erp-web
docker run --rm -p 3000:3000 --env-file .env.local "erp-web:$SHA"
The build tags the image with the Git commit, which is the only tagging scheme that lets you answer "what exactly is running in production?" months later. Never deploy :latest. docker images should show a final image of a few hundred megabytes for a Next.js standalone build, the exact figure depends on your dependencies, and if yours is over a gigabyte, the multi-stage split or the .dockerignore is not working.
The docker run starts it locally with your environment file so you can check it before pushing anywhere. Passing secrets with --env-file keeps them out of your shell history and out of docker inspect output, which is where -e KEY=value puts them.
Compose for local development
# compose.yaml — local development only.
# These credentials are deliberately weak and must never be
# reused on any machine reachable from the internet.
services:
db:
image: postgres:18-bookworm
environment:
POSTGRES_USER: erp
POSTGRES_PASSWORD: devpassword
POSTGRES_DB: erp
command: >
postgres -c shared_preload_libraries=pg_stat_statements
-c log_min_duration_statement=200ms
ports:
- "127.0.0.1:54322:5432"
volumes:
- pgdata:/var/lib/postgresql/data
- ./packages/db/seed:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U erp -d erp"]
interval: 5s
timeout: 3s
retries: 10
redis:
image: redis:7-alpine
ports:
- "127.0.0.1:63790:6379"
web:
build:
context: .
dockerfile: apps/web/Dockerfile
target: build
command: pnpm --filter=@erp/web dev
environment:
DATABASE_URL: postgres://erp:devpassword@db:5432/erp
REDIS_URL: redis://redis:6379
ports:
- "127.0.0.1:3000:3000"
volumes:
- ./apps:/app/apps
- ./packages:/app/packages
# Keep the image's installed dependencies visible; the two
# bind mounts above would otherwise hide them.
- /app/apps/web/node_modules
- /app/packages/db/node_modules
- /app/apps/web/.next
depends_on:
db:
condition: service_healthy
volumes:
pgdata:
Compose describes services, a private network joining them, and named volumes for data that must survive a restart. Services address each other by name, which is why DATABASE_URL says @db:5432. Every published port is bound to 127.0.0.1 so the containers are reachable from your laptop and from nowhere else — plain "54322:5432" would expose an empty-password-adjacent development database to your coffee shop's network. Host ports are also deliberately unusual (54322, 63790) so this stack does not collide with a locally installed PostgreSQL or the Supabase CLI.
The three bare paths in the web volumes list are anonymous volumes, and they fix a trap that costs people an afternoon. Bind-mounting ./apps over /app/apps hides everything the image installed underneath it, including node_modules, so the dev server starts and immediately fails to resolve its own dependencies. Listing the nested paths on their own re-exposes the image's copies.
The healthcheck plus depends_on: condition: service_healthy stops the app booting before the database accepts connections, which otherwise produces a crash loop on every fresh clone. Anything in docker-entrypoint-initdb.d runs once when the volume is first created — that is where seed data goes, meaning a small set of realistic example rows so a fresh clone has something to look at.
docker compose up -d
docker compose ps
docker compose logs -f web
docker compose exec db psql -U erp -d erp -c '\dt'
# DESTRUCTIVE: -v deletes the pgdata volume and every local row.
docker compose down -v
The value of this file is onboarding: a new developer clones the repository, runs one command, and has a working database with seed data in about ninety seconds. docker compose ps should show all three services running and the database healthy. Be careful with the last line. docker compose down on its own stops the containers and keeps your data; adding -v deletes the volume and every row in your local database. That is what you want when the schema has drifted and a disaster at any other time.
When orchestration is worth it, and Kubernetes
Orchestration earns its complexity when you have more containers than you can hold in your head, when you need automatic rescheduling across many machines, and when a dedicated person owns the platform. In practice that means dozens of services, several teams deploying independently, and a full-time infrastructure engineer on the payroll.
You have one Next.js app, one worker, one database, and one sync service. Running Kubernetes for that is the wrong answer, and the reason is concrete. Kubernetes brings:
- a control plane to operate (the cluster's own brain, which decides what runs where),
- a networking model to understand (services, plus ingress controllers that let outside traffic in, plus the plugin that gives every container an address),
- a storage abstraction to get wrong,
- role-based access policies that decide who may do what,
- an upgrade treadmill roughly every four months,
- and more YAML configuration than you have application code.
Managed Kubernetes removes some of the control-plane work and none of the rest — Fly.io charges $75/month per cluster before any compute or volumes, as of July 2026, which is a fair signal of the overhead involved. Every hour spent there is an hour not spent on the available-to-sell calculation your sales team needs. (Available-to-sell, or ATS, is how much of a style you can still promise a buyer: chapter 8 builds it.)
The right ladder is: managed platform, then a single container host, then two container hosts behind a load balancer, and only then an orchestrator. Most brands never reach step three.
Four things do it: rehearsed restores, real monitoring, least-privilege access, and a deploy you trust. You can have all four on a single VPS or on Vercel. Adopting Kubernetes delivers none of them by itself, and plenty of teams run a cluster with no tested backup behind it.
Database operations you own regardless of host
Managed hosting removes the machine. It does not remove your responsibility for connection limits, backup verification, upgrade timing, or knowing which metrics matter. These four are yours on Vercel and Supabase exactly as much as on a VPS.
Connections and pooling
PostgreSQL allocates a process per connection, and each one costs memory whether it is doing anything or not. A serverless platform can spawn hundreds of concurrent function instances, each wanting a connection, and exhaust the database in seconds. This is the most common way a Next.js app takes down its own database.
Chapter 10 covers the client-side configuration; the operational rules are these:
- Route serverless and edge workloads through a transaction-mode pooler — Supavisor on port 6543 on Supabase, PgBouncer in transaction mode if you self-host.
- Route long-lived processes (your pg-boss worker, migrations,
pg_dump) through a direct connection, because transaction mode breaks anything that has to outlive a single transaction: session-level settings,LISTEN/NOTIFY(PostgreSQL's built-in message channel, where one session waits on a connection for another to send it something), advisory locks held across several statements, and prepared statements unless the pooler is configured to track them. - Size the pool so that total backend connections across app, worker, replicas, and your own psql sessions stays under
max_connections, leaving free the handful of slots PostgreSQL holds back for superuser logins so an administrator can still get in when everything else is full.
-- Who is connected, and are any of them stuck?
SELECT state,
count(*) AS conns,
max(now() - state_change) AS longest
FROM pg_stat_activity
WHERE datname = current_database()
GROUP BY state
ORDER BY conns DESC;
-- Idle-in-transaction sessions hold locks and block vacuum.
SELECT pid, usename, application_name,
now() - xact_start AS open_for, left(query, 80) AS query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
AND now() - xact_start > interval '1 minute'
ORDER BY xact_start;
The first query groups connections by state. A healthy app shows mostly idle with a handful active. The second finds the dangerous case: a transaction opened and then abandoned, usually by code that hit an exception between BEGIN and COMMIT. Those sessions hold their locks indefinitely and stop autovacuum from cleaning up any row version newer than their snapshot, which is how a database quietly bloats over a weekend. If this query returns rows regularly, fix the code path. As an emergency measure set idle_in_transaction_session_timeout, which kills such sessions automatically after a chosen interval.
Dumps versus continuous archiving
A pg_dump is a logical snapshot. The PostgreSQL manual describes its output as "a snapshot of the database at the time pg_dump began running", and notes that dumps "can generally be re-loaded into newer versions of PostgreSQL, whereas file-level backups and continuous archiving are both extremely server-version-specific." That makes it portable, restorable one table at a time, and the right tool for moving a database between providers or seeding staging.
It is the wrong tool as your only backup, because your recovery point is whenever the dump ran. A nightly dump means up to 24 hours of lost orders. Continuous archiving instead copies each write-ahead log segment off the machine as it is completed, on top of periodic base backups, so recovery can replay to any chosen moment. That turns "we lost a day" into "we lost 90 seconds." Run both: dumps for portability and for restoring one accidentally-truncated table, continuous archiving for actual disaster recovery.
# Nightly logical dump, directory format, parallel-friendly
pg_dump -Fd -j 4 -Z 6 \
--no-owner --no-privileges \
-d "$DATABASE_URL_DIRECT" \
-f /var/backups/erp/dump-$(date +%F)
# Restore one table into a scratch database
createdb erp_scratch
pg_restore -d erp_scratch -j 4 \
--table=purchase_orders \
/var/backups/erp/dump-2026-07-24
-Fd is directory format, the only one that supports parallel dump with -j, which matters once the database is tens of gigabytes. -Z 6 compresses. --no-owner --no-privileges makes the dump restorable into a database with different role names, which is what you want when moving between providers.
Watch the connection string: it uses the direct URL, because pg_dump needs a session-mode connection and will misbehave through a transaction pooler. Restore into a scratch database instead of the live one, as shown, so a mistake in the table name cannot overwrite production. Create /var/backups/erp owned by the postgres user with mode 700 first, because a dump is a complete copy of your customer data sitting in a file.
Continuous archiving with pgBackRest
# /etc/pgbackrest/pgbackrest.conf
# chown root:postgres, chmod 640 — this file holds two secrets.
[erp]
pg1-path=/var/lib/postgresql/18/main
pg1-port=5432
[global]
repo1-type=s3
repo1-s3-endpoint=s3.us-west-004.backblazeb2.com
repo1-s3-region=us-west-004
repo1-s3-bucket=yourbrand-erp-backups
repo1-s3-key=YOUR_APPLICATION_KEY_ID
repo1-s3-key-secret=YOUR_APPLICATION_KEY
repo1-path=/pgbackrest
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=A_LONG_RANDOM_PASSPHRASE_KEEP_THIS_SAFE
repo1-retention-full=4
repo1-retention-diff=6
repo1-retention-archive-type=diff
process-max=4
compress-type=zst
compress-level=6
start-fast=y
delta=y
archive-async=y
spool-path=/var/spool/pgbackrest
log-level-console=info
log-level-file=detail
[global:archive-push]
process-max=2
Use pgBackRest. Hand-rolled archive commands built on cp silently lose WAL under load, and the PostgreSQL manual itself warns that GNU cp "will return status zero when -i is used and the target file already exists, which is not the desired behavior" — in other words, the copy quietly fails and PostgreSQL is told it succeeded.
A stanza is one database cluster's backup configuration. The repository lives in object storage — Backblaze B2 from $6.95/TB/month with free egress up to three times your stored volume, or Cloudflare R2 at $0.015/GB-month with free egress (both checked July 2026). Both are S3-compatible, meaning they answer the same commands Amazon S3 does, which is why repo1-type=s3 works with either.
repo1-cipher-type encrypts backups at rest with a passphrase you hold, which is what makes it safe to put your company's data in someone else's bucket. Store that passphrase in your password manager: without it the backups are unrecoverable, and losing it is the same as having no backups at all. This file contains both the storage credentials and the encryption passphrase in plain text, so its permissions are part of the configuration, not an afterthought.
# /etc/postgresql/18/main/conf.d/20-archive.conf
archive_mode = on
archive_command = 'pgbackrest --stanza=erp archive-push %p'
archive_timeout = 60
archive_timeout = 60 forces a WAL segment switch every minute even on a quiet system, bounding worst-case data loss to about 60 seconds. wal_level is already set to replica in 10-erp.conf, so it is not repeated here; keeping each setting in exactly one file saves you an afternoon of wondering which value won.
sudo install -d -m 750 -o postgres -g postgres /var/spool/pgbackrest
sudo install -d -m 750 -o postgres -g postgres /var/log/pgbackrest
sudo chown root:postgres /etc/pgbackrest/pgbackrest.conf
sudo chmod 640 /etc/pgbackrest/pgbackrest.conf
# archive_mode can only be set at server start: RESTART, not reload.
sudo systemctl restart postgresql
sudo -u postgres pgbackrest --stanza=erp stanza-create
sudo -u postgres pgbackrest --stanza=erp check
sudo -u postgres pgbackrest --stanza=erp --type=full backup
sudo -u postgres pgbackrest --stanza=erp info
Order matters here and one step catches almost everyone. The spool and log directories must exist and be owned by the postgres user before archiving starts, because archive-async=y writes to the spool. The permissions lines lock down the config file. Then comes the restart: archive_mode is one of the settings PostgreSQL says "can only be set at server start", so a reload leaves archiving switched off while everything looks fine. pgBackRest's own guide says the same thing — the cluster must be restarted before you take a backup.
After the restart, stanza-create initializes the repository. check verifies that PostgreSQL can push WAL and that pgBackRest can read it back; run it after any configuration change and treat a failure as an outage-level problem, because silent archive failure is how people discover they have no backups. The full backup will take minutes to hours depending on size. info prints the backup set with timestamps and WAL ranges, and that output is what you screenshot for your customers' security questionnaires.
# /etc/systemd/system/pgbackrest-full.timer
[Unit]
Description=Weekly full pgBackRest backup
[Timer]
OnCalendar=Sun 02:00
RandomizedDelaySec=900
Persistent=true
[Install]
WantedBy=timers.target
Pair this with a matching pgbackrest-full.service unit that runs the backup command as the postgres user, plus a second timer running --type=diff nightly, and enable both with systemctl enable --now pgbackrest-full.timer. Use systemd timers instead of cron, the traditional Unix scheduler: timers log to the journal, they support Persistent=true so a run missed while the machine was off fires after the next boot, and systemctl list-timers shows you the next scheduled run at a glance. RandomizedDelaySec spreads load if you ever run several machines.
The restore drill
An untested backup is a rumor. Run this drill quarterly and after every major change, on a separate machine, with a timer running so you learn your real RTO rather than your imagined one.
#!/usr/bin/env bash
# restore-drill.sh — run on a scratch VM, NEVER on production.
# It deletes the contents of $PGDATA. Read every line first.
set -euo pipefail
DRILL_HOST=erp-drill # expected hostname of the scratch VM
if [ "$(hostname)" != "$DRILL_HOST" ]; then
echo "Refusing to run: expected host $DRILL_HOST" >&2
exit 1
fi
START=$(date +%s)
TARGET_TIME="2026-07-24 14:07:00+00"
PGDATA=/var/lib/postgresql/18/drill
# One-time setup on the scratch VM:
# sudo pg_createcluster 18 drill --port 5433
# sudo pg_ctlcluster 18 drill stop
echo "==> 1. what do we have?"
sudo -u postgres pgbackrest --stanza=erp info
echo "==> 2. restore base backup to $TARGET_TIME"
sudo systemctl stop postgresql@18-drill || true
sudo -u postgres find "${PGDATA:?}" -mindepth 1 -delete
sudo -u postgres pgbackrest --stanza=erp \
--pg1-path="$PGDATA" \
--type=time --target="$TARGET_TIME" \
--target-action=promote \
--delta restore
echo "==> 3. start and let it replay WAL"
sudo systemctl start postgresql@18-drill
until sudo -u postgres psql -p 5433 -c 'SELECT 1' >/dev/null 2>&1
do
sleep 2; echo " still recovering..."
done
echo "==> 4. prove the data is right"
sudo -u postgres psql -p 5433 -d erp <<'SQL'
SELECT max(created_at) AS latest_row FROM order_lines;
SELECT count(*) AS orders FROM orders;
SELECT sum(qty_delta) AS ledger_balance FROM inventory_ledger;
SQL
echo "==> RTO measured: $(( $(date +%s) - START )) seconds"
The hostname guard at the top is the most important line in the script. Everything below it deletes a data directory, and a restore drill pasted into the wrong terminal is a self-inflicted disaster. The find ... -delete form is used instead of rm -rf because it will not follow a mistyped path into somewhere unexpected, and ${PGDATA:?} makes the shell abort if the variable is ever empty.
Step 1 lists available backups so you pick a base that predates your target time. Step 2 restores it with --type=time, which writes the recovery settings and the recovery.signal file for you; --delta only replaces files that differ, making repeat drills much faster.
The commented pg_createcluster lines are one-time setup: on Debian and Ubuntu the systemd unit postgresql@18-drill only exists once that cluster has been created, and the drill runs on port 5433 so it cannot collide with a real instance. Step 3 starts the instance and waits while PostgreSQL replays WAL.
Step 4 is the part people skip and the part that matters: query real business tables and confirm the numbers. latest_row should sit just before your target time, and the ledger balance should match what you recorded from production. The final line prints your measured RTO. Write it down — if it is 45 minutes and your stated RTO is 15, you now know which of the two is wrong.
A backup counts only once you have restored from it. Everything you have not restored is a file you hope is usable. Put the drill in the calendar with a named owner, record the measured RTO each time, and treat a failed drill as a production incident.
Upgrades
PostgreSQL ships minor releases at least once every three months containing only bug fixes, security fixes, and data-corruption fixes. Applying them requires stopping the server, installing new binaries, and starting it — no dump, no reload. The project's own versioning policy states that "the community considers performing minor upgrades to be less risky than continuing to run an old minor version", and recommends always running the current minor release. Schedule minor upgrades monthly during a quiet window. On Supabase this is a button in the dashboard; take the prompt when it appears.
Major versions arrive roughly annually and need pg_upgrade or a dump and reload. Do not chase them. Wait for at least the .2 or .3 minor release, then plan a real maintenance window. Never upgrade a major version during market week, during month-end close, or in the two weeks before a large retailer's EDI onboarding deadline. (EDI, electronic data interchange, is the structured file format large retailers use to send you orders and receive your invoices; chapter 4 covers it.)
# Major upgrade, self-hosted. Example: PostgreSQL 17 -> 18.
# Take a fresh full backup FIRST and verify it with a restore.
# 1. Check compatibility days ahead. Changes nothing.
sudo -u postgres /usr/lib/postgresql/18/bin/pg_upgrade \
--old-datadir=/var/lib/postgresql/17/main \
--new-datadir=/var/lib/postgresql/18/main \
--old-bindir=/usr/lib/postgresql/17/bin \
--new-bindir=/usr/lib/postgresql/18/bin \
--check
# 2. On the day: BOTH clusters must be stopped first.
sudo systemctl stop postgresql
# 3. The real run. --link is fast but one-way.
sudo -u postgres /usr/lib/postgresql/18/bin/pg_upgrade \
--old-datadir=/var/lib/postgresql/17/main \
--new-datadir=/var/lib/postgresql/18/main \
--old-bindir=/usr/lib/postgresql/17/bin \
--new-bindir=/usr/lib/postgresql/18/bin \
--link
# 4. Start the new cluster, then rebuild planner statistics.
sudo systemctl start postgresql@18-main
sudo -u postgres /usr/lib/postgresql/18/bin/vacuumdb \
--all --analyze-in-stages
# 5. Only after verifying the application works:
sudo -u postgres ./delete_old_cluster.sh
Step 1 runs every compatibility test and changes nothing; run it days before the real upgrade and fix whatever it complains about. Step 2 is the step people forget: pg_upgrade refuses to run while either cluster is up, and an old cluster left running after a --link upgrade will corrupt the new one.
--link uses hard links instead of copying data files — a hard link is a second name for the same file on disk, so nothing is duplicated — turning an hour into a couple of minutes, at the cost of making rollback impossible once started. That is why you take the fresh, verified backup first.
Step 4's vacuumdb --analyze-in-stages matters just as much: pg_upgrade does not carry over planner statistics (the collected facts about how many rows a table holds and how the values are spread), and without them the first queries after the upgrade pick terrible plans and look exactly like an outage. Step 5 deletes the old data directory, and you run it only once the application has been serving real traffic happily.
On Debian and Ubuntu there is a friendlier wrapper, pg_upgradecluster 17 main, which handles the cluster bookkeeping, port assignment, and configuration copying for you. It can either dump the data and reload it into the new cluster, or run pg_upgrade in place, so state which you want: --method=dump for the slow path that leaves the old cluster untouched, or --method=upgrade --link for the fast one. For a 40 GB database on a quiet Sunday, the slow path is fine, and the reduced chance of a mistake is worth more than the saved minutes.
The metrics that matter
| Metric | Where to find it | Healthy | Act when |
|---|---|---|---|
| Cache hit ratio | pg_stat_database: blks_hit / (blks_hit + blks_read) | > 99% | Below 95% sustained: raise shared_buffers or add RAM |
| Connections in use | pg_stat_activity row count | < 60% of max_connections | Above 80%: pool harder before scaling up |
| Idle in transaction | pg_stat_activity where state = that | 0 over 1 minute | Any: fix the code path holding it |
| Dead tuples | pg_stat_user_tables.n_dead_tup | < 10% of live | Over 20%: tune autovacuum for that table |
| Longest transaction age | max(now() - xact_start) | Seconds | Over 5 minutes unexpectedly: investigate immediately |
| Replication lag | pg_stat_replication.replay_lag | < 1 second | Over 30 seconds: standby cannot keep up |
| Deadlocks | pg_stat_database.deadlocks | 0 | Any increase: review lock ordering (chapter 3) |
| Slowest queries | pg_stat_statements by mean_exec_time | Stable set | A new entry at the top: something regressed |
| Disk free | df -h or vendor dashboard | > 30% | Below 20%: act now, below 10% is an emergency |
| WAL archive lag | pgbackrest --stanza=erp check | Passing | Any failure: you are not protected |
-- Paste this into a saved dashboard query. One screen, all of it.
SELECT
(SELECT round(100.0*sum(blks_hit)/nullif(sum(blks_hit+blks_read),0),2)
FROM pg_stat_database) AS cache_pct,
(SELECT count(*) FROM pg_stat_activity) AS conns,
(SELECT count(*) FROM pg_stat_activity
WHERE state='idle in transaction') AS idle_in_txn,
(SELECT coalesce(max(now()-xact_start),'0'::interval)
FROM pg_stat_activity WHERE xact_start IS NOT NULL) AS oldest_txn,
(SELECT sum(deadlocks) FROM pg_stat_database) AS deadlocks,
(SELECT pg_size_pretty(pg_database_size(current_database())))
AS db_size;
Six numbers on one line. Run it manually when something feels slow, and wire it into whatever dashboard you use so it sits on a screen you already look at. If you only ever check one thing, check oldest_txn — a transaction that has been open for hours is upstream of an enormous number of other symptoms, from bloat to replication lag to failed vacuum.
Scaling, in the order it actually happens
Scaling has a correct order, and the order is cheapest-and-most-effective first. People skip to the expensive steps because they are more interesting, and end up paying for a bigger machine to run the same missing index badly.
| Step | The signal that it is time | Typical effect | Cost |
|---|---|---|---|
| 1. Fix the query | One entry dominates pg_stat_statements by total time | 10–1000× | A few hours |
| 2. Add an index | EXPLAIN shows a sequential scan on a large table in a hot path | 10–100× | An hour, plus write overhead |
| 3. Pool connections | Connection count near max_connections; "too many clients" errors | Removes a hard ceiling | Hours, or a config toggle |
| 4. Cache | The same expensive read repeated with unchanged inputs | 2–50× on those paths | Days, plus invalidation bugs |
| 5. Scale up (vertical) | CPU or memory sustained above 70%, with 1–4 already done | 2–4× | Minutes and money |
| 6. Read replica | Reporting queries measurably slowing order entry | Offloads reads | Doubles DB spend; adds lag bugs |
| 7. Anything exotic | You have exhausted 1–6 and can prove it | Varies | Very high |
Query optimization first. Turn on pg_stat_statements, sort by total_exec_time, and look at the top five. In almost every young ERP one query dominates, and it is usually an ATS or inventory rollup (chapter 8) computing over the whole ledger when it needs one customer's slice.
Indexes second. Run EXPLAIN (ANALYZE, BUFFERS) on the slow query and read the plan for a sequential scan over a large table — a sequential scan means PostgreSQL read every row one after another because no index could narrow the search. Add the index with CREATE INDEX CONCURRENTLY so writes are not blocked, and check afterwards that it did not fail and leave an invalid index behind. Indexes are not free: each one slows every insert and update on that table, so drop the ones pg_stat_user_indexes shows as never scanned.
Pooling third, covered above. Caching fourth, and be careful — cache invalidation is where correctness bugs come from, and an ERP that shows a buyer stale availability creates an oversell: you have promised stock that does not exist, and somebody has to make that phone call. Cache things that are genuinely static: currency rates for the day, size-scale definitions, a season's style list. Do not cache available-to-sell.
Scaling up, replicas, and everything after
Vertical scaling fifth. On Supabase this is a dropdown: as of July 2026, Micro $10/month, Small $15, Medium $60, Large $110, XL $210, 2XL $410. Doubling compute is a two-minute change with a short restart, and it buys you time to do steps 1–4 properly. It is a legitimate move when a season launch is next week.
Read replicas sixth. They cost roughly what your primary costs. Supabase runs a read replica "on the same Compute size as the primary database", gives it a disk "1.25x the size of the primary disk to account for WAL archives", and states plainly that read replicas "are not covered by the Spend Cap", so a replica added in a hurry can quietly double your bill.
They also introduce replication lag, meaning a write followed immediately by a read on the replica can return stale data. Route only lag-tolerant, read-only work there: BI dashboards (business intelligence — the reporting screens management looks at), exports, the analytics in chapter L.
Supabase's own guidance on high CPU is to optimize your usage before buying more machine, and it presents replicas as a way to take heavy analytical queries off the primary, to put data closer to users in another part of the world, and to add redundancy. Treat a replica as workload isolation, not as a cheap way to buy throughput.
Everything else after. Sharding (splitting one database across several machines by customer or by date), multi-region writes, CQRS (keeping separate models for writing and for reading), event-sourced read models, a separate analytics warehouse. Each adds a permanent tax on every future feature, and a wholesale apparel ERP at brand scale almost never needs them.
High availability and disaster recovery
Recovery point objective is how much data you accept losing. If your RPO is five minutes, you are saying that losing the last five minutes of order entry is survivable. Recovery time objective is how long you accept being down. If your RTO is four hours, you are saying the business can operate for four hours without the ERP.
Set these honestly, because they are the only inputs that determine what you must buy. A brand that demands zero downtime and zero data loss, then declines to pay for either, has stated a wish. A target is a number somebody has funded.
| Target pair | What it requires | Roughly what it costs | Sensible for |
|---|---|---|---|
| RPO 24h / RTO 24h | Nightly dump to object storage | < $5/month | Nothing running a real business |
| RPO 5min / RTO 4h | Continuous WAL archiving + rehearsed restore | $5–20/month self-run; $100/month Supabase PITR | Most small brands — start here |
| RPO 1min / RTO 1h | WAL archiving + a warm standby you promote manually | + one standby instance | Brands where a half-day outage costs real money |
| RPO ~0 / RTO minutes | Synchronous replication + automatic failover + tested runbook | 2–3× database spend, plus real expertise | Rarely justified at brand scale |
| RPO 0 / RTO ~0 | Multi-region active-active | Enormous, plus application redesign | Not you |
For almost every brand reading this, row two is right, with a seasonal exception: during market week and during the two weeks of shipping crunch, temporarily raise your ambition. Add a standby, put a second person on call, and freeze deploys. Then relax it again. Reliability can be seasonal, and in this industry it should be.
High availability means surviving a component failure without human action. Disaster recovery means restoring service after something worse — a region outage, a deleted database, a ransomware event. They are different problems, and high availability does not give you disaster recovery. A replicated cluster faithfully replicates a DROP TABLE to every replica in milliseconds. The only protection is a backup stored somewhere the damage cannot reach: a different account, a different provider, a different region.
A disaster recovery plan template
DISASTER RECOVERY PLAN — <Brand> ERP
Owner: <name> Deputy: <name> Reviewed: <date>
Next review: <date +6 months>
1. TARGETS
RPO: 5 minutes. RTO: 4 hours (business hours)
RTO: 8 hours (nights/weekends)
Market week override: RTO 1 hour, second responder on call.
2. WHAT WE PROTECT
Postgres database ......... pgBackRest, B2 bucket, region us-west
Object storage (images) ... versioned bucket + lifecycle rules
Application code .......... GitHub, mirrored to <second remote>
Secrets ................... 1Password vault "ERP Infra"
Backup encryption pass .... 1Password, separate vault, 2 holders
DNS ....................... Cloudflare, 2 admins with hardware keys
3. WHO TO CALL
Primary on-call ........... <name> <phone>
Deputy .................... <name> <phone>
Hosting vendor ............ <support URL / plan / account id>
Business decision maker ... <name> <phone>
4. SCENARIOS AND FIRST ACTIONS
S1 App down, DB healthy
-> check /api/health, roll back to previous release,
confirm in monitoring. Target 15 min.
S2 Database unreachable
-> check vendor status page; if vendor incident, post
customer notice and wait; if ours, see S3.
S3 Data corrupted or wrongly deleted
-> FREEZE WRITES (put app in read-only mode).
Identify the timestamp before the damage.
Restore to a NEW instance at that time (drill script).
Verify counts. Repoint DATABASE_URL. Unfreeze.
S4 Whole region unavailable
-> restore latest backup into <secondary region>,
update DNS, accept RPO up to 5 minutes.
S5 Credentials compromised
-> rotate DB passwords, API keys, and OAuth secrets;
revoke all sessions; review audit log for the window.
5. COMMUNICATION
Internal: #erp-incidents Slack channel, updates every 30 min.
External: status page + email to affected buyers if the
outage exceeds 2 hours during business hours.
Template: what is broken, what works, what we are doing,
next update time. No speculation on cause.
6. AFTER
Write the timeline within 48 hours. Blameless.
One action item per contributing cause, each with an owner
and a date. Review at the next ops meeting.
7. DRILL LOG
<date> — S3 restore drill, measured RTO 38 min, PASS
<date> — ...
Keep this in the repository as a Markdown file so it is versioned, and keep a printed copy or an offline copy in your password manager, because a plan stored only in the system that is down cannot be read when you need it.
The drill log at the bottom is the part that makes it credible: four dated entries prove a capability, an empty log proves nothing. "Blameless" in section 6 means the write-up describes what the system allowed to happen, not who typed the command — people report problems honestly only when reporting is safe. Chapter 9's operational practices and chapter N's change management process both plug into that section.
A single environment variable that makes the app reject all writes with a friendly banner lets you freeze the system the instant you suspect data damage, without taking it offline. Buyers can still look things up. Build it early; you will use it during migrations too.
Security operations
Security for a small team comes down to a few habits performed consistently. There is no product you can buy that substitutes for them.
Patching. The operating system patches itself via unattended-upgrades. Your container base images do not — rebuild and redeploy at least monthly even when your code has not changed, so the base image picks up fixes. Your Node dependencies do not either.
SHA=$(git rev-parse --short HEAD)
# Fail the build on a high or critical advisory.
pnpm audit --audit-level=high
# Bump minor and patch versions across every workspace package.
pnpm dlx npm-check-updates -u --target minor --workspaces --root
pnpm install
# Scan the built image for OS-level vulnerabilities.
docker scout cves "erp-web:$SHA" \
--only-severity critical,high
pnpm audit checks your dependency tree against published advisories; --audit-level accepts low, moderate, high, or critical, and setting it to high in CI — continuous integration, the automated checks that run on every commit — means a serious advisory fails the build while noise does not.
The npm-check-updates line needs --workspaces --root in a monorepo, or it silently updates only the package.json in the current directory and you will wonder why nothing changed. Run it on a schedule, not in a panic after an advisory lands, and let the test suite from chapter 9 tell you what broke.
docker scout scans the built image for vulnerabilities in system packages, which is the layer pnpm audit cannot see — scan the commit-tagged image you actually built, not :latest. Also enable Dependabot or Renovate on the repository, both watch your dependencies and open a pull request when a newer version appears, so upgrades arrive as changes you can review instead of chores you must remember.
Rotating secrets on a schedule
Secret rotation. Keep an inventory:
- database passwords,
- API keys for every integration in chapter 4,
- the JWT signing secret,
- webhook signing secrets,
- the pgBackRest encryption passphrase,
- and your object-storage credentials.
A JWT (JSON Web Token) is a small signed string a server hands a logged-in user as proof of identity; the signing secret is what makes the signature unforgeable, so leaking it lets an attacker mint a token for any user.
Rotate on a schedule, annually is defensible for a small team, and immediately whenever someone with access leaves or a laptop is lost. Rotation is only feasible if your app reads secrets from the environment rather than from committed files, so design for that from the beginning. Support two valid values at once for anything with external consumers: issue the new key, deploy, switch callers, then revoke the old one.
Access, identity, and second factors
Least privilege for infrastructure access. Nobody uses a shared login. Every human has their own account with their own SSH key or their own single sign-on identity. (Single sign-on, or SSO, means one company identity provider logs you into everything; the two standard protocols for it are SAML, the older XML-based one that large enterprises ask for, and OIDC, the modern JSON-based layer built on OAuth that most developer tools use.) Production database access uses the read-only role by default, with write access requested deliberately. Cloud console access is scoped per person, not one root account shared in a spreadsheet.
Multi-factor authentication — a second proof of identity beyond the password — is mandatory on the domain registrar, the DNS provider, the cloud console, the code host, and the password manager, because those five accounts can be used to take everything else.
The common second factors are TOTP (time-based one-time password: the six digits your authenticator app rotates every 30 seconds) and WebAuthn (the standard behind hardware security keys and passkeys, where a device holds a private key and proves possession without ever transmitting a shared secret). Prefer WebAuthn where the vendor supports it, because TOTP codes can be phished in real time and WebAuthn responses are bound to the site's domain and cannot be replayed elsewhere.
Keeping an audit trail
Audit logging. You need to answer "who changed this price, and when?" and "who logged into the server last month?" The first is application-level and comes free from the append-only ledger in chapter 1. The second is infrastructure-level: keep journalctl -u ssh history, enable your cloud provider's audit log, and ship both somewhere immutable — storage configured so that nobody, including you, can edit or delete an entry for a set period.
If an enterprise customer wants your logs delivered into their own SIEM (security information and event management system — the central tool a security team uses to collect and search logs), that is a paid feature almost everywhere: WorkOS, for example, lists audit log streaming at $125/month per SIEM connection as of July 2026. For a small brand, 90 days of authentication and deploy events in object storage is proportionate and cheap.
Email authentication and deliverability
Email authentication is infrastructure security, because your ERP sends order confirmations and invoices that must arrive. Three DNS records do the work:
- SPF (Sender Policy Framework) is a TXT record listing which servers may send mail for your domain.
- DKIM (DomainKeys Identified Mail) publishes a public key so receivers can verify a cryptographic signature your provider adds to each message.
- DMARC (Domain-based Message Authentication, Reporting and Conformance) tells receivers what to do when SPF and DKIM fail, and where to send reports.
Since 1 February 2024, Google has required every sender to authenticate with SPF or DKIM, publish valid forward and reverse DNS, use TLS for transmission, format messages per RFC 5322, and "keep spam rates reported in Postmaster Tools below 0.30%" — Google's own advice is to stay under 0.10% and never touch 0.30%.
Senders of 5,000 or more messages a day to Gmail also need all three of SPF, DKIM, and DMARC, with the From header domain aligned to either the SPF or the DKIM domain, plus one-click unsubscribe on marketing and subscribed mail (requirements checked July 2026).
Publish one SPF record listing every sender, the DKIM record your provider gives you, and a DMARC record starting at p=none. Read the aggregate reports for a few weeks until you have found every legitimate sender, then tighten to quarantine and finally reject. Jumping straight to reject will silently kill your invoices.
On-call for a very small team
With two or three people, formal rotations are theater. What you need is a short, written agreement about what wakes someone up and what does not, plus genuine permission to ignore everything in the second category.
| Event | Response | Why |
|---|---|---|
| App returns 5xx to real users for 5+ minutes | Page immediately, any hour | Business is stopped |
| Database unreachable | Page immediately | Everything depends on it |
| Disk above 90% on the DB machine | Page immediately | Writes will stop, and recovery is slow |
| WAL archiving failing for 30+ minutes | Page immediately | You are unprotected right now |
| TLS certificate expires in under 72 hours | Page during the day; escalate at 24h | Total outage on expiry, but predictable |
| Payment or EDI integration erroring repeatedly | Business hours, unless it is a shipping cut-off day | Recoverable by replay (chapter 4) |
| Background job queue backing up | Business hours if under 1 hour of backlog | pg-boss retries; work is not lost |
| Raised error rate on one non-critical page | Next working day | Annoying, not stopping anyone |
| Slow query alert | Next working day | Degradation, not outage |
| Dependency vulnerability advisory | Next working day, or next sprint if low | Rushed patching causes outages |
Set up alerting that respects that table, and budget for it properly. Better Stack's free tier, as published in July 2026, covers 10 monitors and heartbeats and one status page, but alerts only by Slack and email, which is no use at 3am.
Phone call and SMS alerting comes with a paid Responder seat at $34/month billed monthly, or $29/month billed annually, per responder; that also raises check frequency to as often as every 30 seconds. Extra capacity is sold in blocks: 50 more monitors at $25/month ($21 annually), 10 more heartbeats at $20/month ($17 annually). Whatever tool you choose, price the paging channel rather than assuming a free tier covers it.
Heartbeat monitoring is the feature people miss: your nightly backup job pings a URL on success, and if the ping does not arrive, you get alerted. That catches the silent failure — the job that stopped running three weeks ago and never said anything.
Escalation for two people is simple: primary gets paged, and if there is no acknowledgement in ten minutes it goes to the deputy. Configure that in the monitoring tool so it happens automatically and does not depend on someone noticing.
Avoiding burnout is an engineering problem more than a scheduling one. Every page that turns out to be noise must be fixed or its alert deleted in the same week — a person woken three times for nothing will sleep through the fourth, real one. Alternate weeks even with two people, and make the off-week genuinely off.
After any night-time page, the responder starts late or takes the next day; write that into the agreement so nobody has to ask. Publish the market-week exception in advance: during the four highest-stakes weeks of the year both people are reachable, in exchange for a quiet January. And keep raising the bar for what can page you at all — every alert you eliminate through better engineering is a night of sleep you get back permanently.
If only one person can fix production, the system is down whenever that person is on a plane, ill, or asleep somewhere with no signal. Before you self-host anything, make sure a second human has the access, has the runbook (the written step-by-step procedure for each kind of failure), and has actually performed a restore drill themselves.
Data residency and where the servers physically are
"The cloud" is a set of buildings. When you create a Supabase project or a Vercel deployment you pick a region, and that determines which building your customers' data sits in and which country's authorities can compel access to it.
GDPR and cross-border transfers
The GDPR (General Data Protection Regulation) is the EU law governing personal data about people in the EU; it applies to you if you handle such data, wherever your company is.
Under it, transferring personal data out of the EU requires a legal basis: either the destination has an adequacy decision from the European Commission (Article 45) or you have appropriate safeguards such as Standard Contractual Clauses — the European Commission's pre-approved contract wording between a data exporter and importer (Article 46).
The EU–US Data Privacy Framework, adopted by the Commission in July 2023 and still in force as of July 2026, provides an adequacy route for certified US organizations. It has been challenged in the EU courts, and its two predecessors, Safe Harbor and Privacy Shield, were both struck down. For a brand with meaningful EU business, keep EU personal data in an EU region so that nothing depends on the current framework surviving.
Picking a region and tracking every copy
Practical rules for a wholesale brand:
Choose the region where most of your data subjects are, and choose it once. A data subject is a person the data is about: your buyers, their staff, your own employees. Changing a managed database's region later means a full migration with downtime. If your buyers are mostly European, use an EU region even if your engineering team is in New York. Latency to the app matters far less than you think, a 90ms round trip is invisible in an order entry screen, while a residency mistake is expensive to unwind.
Know where every copy lives, not just the primary. Backups, read replicas, log aggregators, error trackers, analytics tools, and your email provider all hold copies. A database in Frankfurt with backups in Virginia, and error traces containing customer names in an American SaaS, is a US data flow whatever the primary region says. Write a one-page inventory listing every system that stores personal data and the country it sits in, and revisit it whenever you add a vendor.
Check what your customers require. Large European retailers increasingly ask about hosting location in their vendor questionnaires. Being able to answer "EU region, EU backups, sub-processor list attached" wins that conversation quickly.
Match your sub-processor list to reality. Your privacy policy names the third parties that process personal data on your behalf; those vendors are your sub-processors. If you add an error tracker or swap email providers, that list changes. Chapter M covers the contractual side; the infrastructure side is keeping the list accurate.
Edge networks and points of presence
One nuance: content delivery networks such as Cloudflare, and Vercel's own edge network, cache and process requests at points of presence, the small clusters of servers they run in dozens of cities, all over the world. (A CDN is exactly that: a global network of caching servers that serves copies of your files from wherever the visitor happens to be.)
Static assets traveling through an edge node is normally fine; personal data processed at an arbitrary edge location is a different question. Keep anything that reads or writes customer records in your chosen region — in Next.js terms, the Node.js runtime pinned to a region for data routes, and the edge only for stateless work.
Migrating between managed and self-hosted
You may need to move later, in either direction. PostgreSQL logical replication — where the source database streams the actual row changes to another database, which applies them as it receives them — makes a near-zero-downtime move genuinely achievable, and the app tier is easy once the database is settled.
The naive approach — pg_dump, restore, repoint — requires downtime proportional to your database size, which on 40 GB means tens of minutes, with disk and network speed setting the exact figure. That is acceptable at 2am in January and unacceptable in February. Logical replication reduces the cutover to seconds.
Before you start, three prerequisites that are easy to miss:
- Every table you replicate needs a primary key, or a
REPLICA IDENTITY FULLsetting, or updates and deletes will fail to replicate while inserts appear to work fine — a failure mode that looks like success for hours. - The source needs a dedicated role with the
REPLICATIONattribute and a matchingpg_hba.confline permitting it from the target's address over SSL. - And the connection string below must use
sslmode=requireat minimum, because it carries your whole database across the public internet.
-- ON THE SOURCE (current production)
CREATE ROLE repl WITH REPLICATION LOGIN PASSWORD 'GENERATED';
ALTER SYSTEM SET wal_level = 'logical'; -- restart required
-- then: sudo systemctl restart postgresql
CREATE PUBLICATION erp_pub FOR ALL TABLES;
SELECT * FROM pg_publication;
-- Any table without a primary key needs this, or its UPDATEs
-- and DELETEs will never reach the target:
-- ALTER TABLE some_table REPLICA IDENTITY FULL;
-- ON THE TARGET (new home), after loading schema only:
-- pg_dump --schema-only -d $SOURCE | psql -d $TARGET
CREATE SUBSCRIPTION erp_sub
CONNECTION 'host=old-db.internal dbname=erp
user=repl password=GENERATED sslmode=require'
PUBLICATION erp_pub
WITH (copy_data = true, streaming = true);
-- Watch it catch up (run on the SOURCE)
SELECT slot_name,
active,
pg_size_pretty(
pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)
) AS behind
FROM pg_replication_slots;
Publish everything on the source, load the schema on the target, then create a subscription that first copies all existing rows and afterwards streams changes continuously. wal_level can only be set at server start, so the ALTER SYSTEM line does nothing until you restart — plan that restart, it is your only downtime before cutover. The final query shows how far behind the target is; when behind settles at a few kilobytes and stays there, you are ready.
Two things logical replication does not copy: sequence values and schema changes. Freeze migrations during the move, and advance sequences at cutover or your first insert will collide with an existing ID. The subscription's connection string is stored in the target's catalog in plain text and visible to superusers, so rotate the repl password once the migration is finished.
# CUTOVER — expect 30 to 90 seconds of write downtime
# 1. Put the app in read-only mode.
curl -X POST https://erp.yourbrand.com/admin/readonly \
-H "Authorization: Bearer $ADMIN_TOKEN"
# 2. Wait for replication lag to hit zero (query above).
# 3. Advance every sequence on the target to match its table.
# Print the statements first, read them, THEN run them.
psql "$TARGET" -Atc "
SELECT format(
'SELECT setval(%L, COALESCE((SELECT max(%I) FROM %s), 1));',
s.oid::regclass::text,
a.attname,
t.oid::regclass::text)
FROM pg_class s
JOIN pg_depend d
ON d.objid = s.oid
AND d.classid = 'pg_class'::regclass
AND d.refclassid = 'pg_class'::regclass
AND d.deptype IN ('a','i')
JOIN pg_class t ON t.oid = d.refobjid
JOIN pg_attribute a
ON a.attrelid = t.oid AND a.attnum = d.refobjsubid
WHERE s.relkind = 'S';" | tee /tmp/setval.sql
psql "$TARGET" -f /tmp/setval.sql
# 4. Sanity check row counts on both sides.
for T in orders order_lines inventory_ledger customers; do
echo -n "$T source="
psql "$SOURCE" -Atc "SELECT count(*) FROM $T" | tr -d '\n'
echo -n " target="
psql "$TARGET" -Atc "SELECT count(*) FROM $T"
done
# 5. Repoint the app, redeploy, drop read-only mode.
# 6. Keep the old database running, read-only, for 7 days.
Step 1 stops writes so the target can reach a consistent state; reads keep working, so nobody is locked out. Step 3 is the one people skip, and skipping it fails minutes after cutover with duplicate key errors. The query walks pg_depend, PostgreSQL's internal record of which objects depend on which, to find the sequence behind each table column.
Guessing at tablename_id_seq names breaks on identity columns, renamed tables, and sequences shared between tables; reading the catalog does not. It writes the generated statements to a file so you can read them before executing, instead of piping straight into a second psql. Step 4 prints matching counts side by side; investigate any mismatch before continuing, not afterwards. Step 6 matters: keep the old database alive and read-only for a week so you can compare anything that looks wrong, and only then delete it.
Moving the application is easier in both directions. From Vercel to a VPS, the work is the standalone build described earlier plus replacing platform features — image optimization works under next start, but the page cache is per-instance by default and needs a shared cache handler across instances, and NEXT_SERVER_ACTIONS_ENCRYPTION_KEY and deploymentId must match everywhere. From a VPS back to Vercel, the work is mostly deletion. That asymmetry tells you where the ongoing effort lives.
Design so that moving is possible and then do not move casually. Standard PostgreSQL rather than vendor-specific extensions, secrets in environment variables, storage behind an interface, and no reliance on any single platform's proprietary runtime — that discipline costs almost nothing and preserves every future option.
Field notes & further reading
- PostgreSQL: Continuous Archiving and Point-in-Time Recovery — the authoritative description of WAL archiving, base backups,
recovery_target_time, and therecovery.signalfile, including the warning about naivecp-based archive commands. Read this once slowly before you configure any backup tool. - pgBackRest User Guide — a complete Debian and Ubuntu walkthrough covering stanza creation, retention policy, encrypted S3-compatible repositories, and point-in-time restore with
--type=time. It also states the restart requirement after enablingarchive_mode. - Next.js: How to self-host your application — official guidance on standalone output, reverse proxies, streaming and buffering, cache handlers for multiple instances, deployment IDs for version skew, the Server Actions encryption key, and the recommended 10–30 second graceful shutdown window.
- PostgreSQL Versioning Policy — the release and end-of-life table, plus the project's statement that "the community considers performing minor upgrades to be less risky than continuing to run an old minor version." Use it to plan your upgrade calendar.
- Supabase: Compute and Disk — published direct and pooler connection limits per compute size, disk types and per-GB pricing, and IOPS provisioning. The best public reference for sizing a Postgres instance against a real connection budget.
- Google: Email sender guidelines — the current requirements for authentication, TLS, the 0.30% spam-rate threshold, and one-click unsubscribe for senders above 5,000 messages a day. Your ERP sends invoices; this page determines whether they arrive.
- Caddy: Reverse proxy quick-start — the shortest path to a reverse proxy with automatic HTTPS, plus the Caddyfile syntax used in this chapter.
- Let's Encrypt Rate Limits — 50 certificates per registered domain every 7 days, 5 certificates for the same exact set of names every 7 days, 5 authorization failures per name per account per hour, 300 new orders per account every 3 hours, and the ARI renewal exemption. Worth knowing before you debug a certificate problem in a loop.
1. Run a real restore drill against your existing setup. Whatever you are on today, restore your production database to a scratch environment at a timestamp of your choosing — on Supabase, use the PITR restore into a new project; self-hosted, use the drill script from this chapter. Time it from decision to first successful business query. Then run three checks: the count of orders, the latest row timestamp in your ledger, and the sum of an inventory balance. Write the measured RTO and the three numbers into a drill log file committed to your repository.
2. Build and containerize the app, then compare. Write the Dockerfile, .dockerignore, and compose.yaml from this chapter for your own project. Get docker compose up -d to bring up PostgreSQL with seed data and your app together from a clean clone. Then provision the cheapest VPS your provider sells, run the hardening sequence (non-root user, key-only SSH, UFW, unattended-upgrades, fail2ban), install PostgreSQL, put Caddy in front, and deploy the app once with the systemd template and the sudoers drop-in. Keep a stopwatch running the whole time and record the total hours.
3. Rebuild the cost table with your own numbers. Take the table in this chapter, replace every price with one you have checked today on the vendor's own pricing page, replace $100/hour with your real loaded cost, and replace the ops-hours estimates with the figure your stopwatch produced in exercise 2. Note the date you checked each price next to it.
When you are done you should have: a dated drill log entry proving you can restore your database and stating your true recovery time; a one-command local development environment; and a personal, measured cost model you can use to make the managed-versus-self-hosted decision with your own data rather than anybody else's opinion.