How to Migrate Data From One Postgres Instance to Another

Thu, 03 Jul 2025 00:57:00 +0000

My first version of willow.camp ran a Postgres instance on the same container as the app. Before going to production, I had to migrate the data from the local Postgres instance to the managed instance.

I had a whopping 50-some records in the database. So, of course, could have used the willow_camp_cli to migrate them, but I wanted to do it the hard way. Because sometimes the hard way is fun. At least, it’s fun when it’s much lower stakes than working on a large production app.

Here is how I migrated the data from one database to another using an SSH tunnel, pg_dump and pg_restore.

Maintenance Mode

I had all of two real-life users when I did this. So I didn’t need maintenance mode, but I went ahead and used it anyway. hatchbox has a maintenance mode toggle in the UI that I turned on.

Set Up a New User

Digital Ocean sets up a managed database with a root user. I wanted to create an app-specific user with access to the necessary databases.

I connected to the managed database using psql and created a user, granting the necessary permissions.

PGPASSWORD=CLUSTER_PASSWORD psql \
  -U cluster_admin \
  -h digitaloceanhost.com \
  -p 25060 \
  -d defaultdb

CREATE USER "willow-camp" WITH PASSWORD 'NEW_PASSWORD';

-- Grant connect permission to the database
GRANT CONNECT ON DATABASE defaultdb TO "willow-camp";

-- Grant usage on the schema
GRANT USAGE ON SCHEMA public TO "willow-camp";

-- Grant table permissions (choose what you need)
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO "willow-camp";

-- Grant permissions on sequences (for auto-increment/serial columns)
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO "willow-camp";

-- Grant permissions on future tables (optional)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO "willow-camp";
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT ON SEQUENCES TO "willow-camp";

Export Data from the Old Database

To do this, I created an SSH tunnel to the hatchbox instance that hosted my app and database. Then, I used pg_dump to export the data. -L in the command below sets up port forwarding so that my local port 5432 points to the hatchbox server where Postgres is running.

ssh -L 5432:localhost:5432 deploy@10.0.1.100

Then I created a pg_dump on my local machine through the tunnel. Note that localhost:5432 now points through the tunnel to the hatchbox database.

pg_dump -Fc "postgresql://app_user:PASSWORD@localhost:5432/myapp_production" > myapp_primary_20250625_185627.dump

Import Data to the New Database

I made a common mistake here. A pg_dump file includes statements like ALTER TABLE public.ar_internal_metadata OWNER TO user_a1b2c3d4e5f6;, which assigns ownership of a table to a user, but this user only existed in my hatchbox database.

I worked around ownership issues by using the --no-owner and --no-priviliges options.

PGPASSWORD=CLUSTER_PASSWORD pg_restore \
  -U cluster_admin \
  -h digitaloceanhost.com \
  -p 25060 \
  -d defaultdb \
  --no-owner \
  --no-privileges \
  -v \
  myapp_primary_20250625_185627.dump

Verify the Data

Once pg_restore had done its job, I connected to the new database again using the psql command above and issued a few queries to check the data.

SELECT relname as table_name, n_live_tup as current_rows FROM pg_stat_user_tables ORDER BY current_rows DESC;

Connect the App to the New Database

Finally, I updated the database environment variables in hatchbox, which rebooted the app. I performed a quick rails console check to ensure the database was connected, then I took the app out of maintenance mode.

Once the app was back up, I could see queries hitting the managed instance in the DigitalOcean UI, and I tore down the old database on my hatchbox server.

Should I use a Managed Postgres Instance?

Tue, 01 Jul 2025 13:04:00 +0000

I asked myself this question while developing willow.camp and decided to use a managed instance before sharing willow.camp with the world. In this post, I’ll explain my thought process and how it changed as I finished off willow.camp’s blogging features.

Hatching a New App

Once I had a rough version of willow.camp coded up, I wanted to deploy it so that I could test features like subdomains, multiple users (real people, not just me and some database seeds), and eventually custom domains.

I kicked off a new app on hatchbox.io using DigitalOcean. hatchbox wires everything up to deploy a Rails app off the main branch of a GitHub repository. All I needed was a database, so I clicked “PostgreSQL” in hatchbox, so that my Rails app and database would run on the same server. In just a few clicks, willow.camp was up and running with 1 vCPU, 1 GB of RAM, and 25 GB of storage.

Going to Production

Once I had most of the features I wanted, I had to make a decision. What do I do if my server crashes? The environment and data are ephemeral. It is easy enough to get the web app up and running again, but I’d lose all the data in my database.

My requirements for the database were:

Automated backups at a minimum in a daily interval. Despite making willow.camp for markdown, I end up saving drafts right in the UI.
Access to backups for a limited period. I don’t need much more than two or three days of backups.
Preferably, minimal configuration.

I had three options:

Take periodic snapshots of the server that I can restore in case of data loss. I didn’t see an automated way to take snapshots, so I dropped this from the list.
Mount a storage volume: This would separate the storage from the container on which my app was running. This seemed like an okay option to me but I was also wary of managing a single server with the Rails app and Postgres.
Use a managed Postgres instance: This met all my requirements, but it cost a bit more. I waffled on this and decided to go with it in the end.

Why I Chose a Managed Postgres Instance

In the past, I worked on a production app where the app and database were hosted on the same server. It constantly felt like a precarious situation. The database and the app server were competing for resources.

On willow.camp I only have 1 vCPU and 1 GB of RAM. I could upsize my droplet, but then I’d lose sleep over balancing resources between the two fundamental processes that run the app.

willow.camp is a tiny app and can easily run with the Rails app and Database on the same small server. I wouldn’t need to lose sleep over it, but I’d be thinking about it all the time.

I needed peace of mind that the processes would keep running and my data would be safe. Scaling up a single instance to handle the app and database didn’t sound like a fun hobby for me. It certainly wasn’t a fun job when I worked on the production setup like this in the past.

The resources on this instance are ideal for a small Rails app. With 1 vCPU and 1 GB of RAM, I can run 5 Puma threads. My response time is well under 200 milliseconds (often under 100ms), so it’ll be a while before I need to consider another CPU.

Choosing a managed Postgres instance gave me peace of mind that my data was safe, my app server and my database had isolated resources, and I had the flexibility to change the app server without affecting the data. The extra $10 a month simplifies my tiny infrastructure and lets me focus on the fun parts of willow.camp.

Postgres on enumerator.dev