Docker Postgres 12 'PANIC: could not locate a valid checkpoint record'

Question

I'm using caprover which uses docker to deploy postgresql 12. Right now my postgresql database is constantly restarting and printing this log everytime:

[24] LOG: invalid resource manager ID ...
[24] LOG: invalid primary checkpoint record ...
[24] PANIC: could not locate a valid checkpoint record

The problem is that the container dies immediately so I can't do any docker exec to try to pg_dump or in the worst case scenario pg_resetwal.

Any idea why this is happening? How can I possibly pg_dump or pg_resetwal?

Answer 1

That means that you lost WAL information, that is, you lost a file or file contents from at least one of the files in the pg_wal subdirectory of the PostgreSQL data directory.

If PostgreSQL crashed, it will recover using the transaction log (WAL). This recovery starts from a checkpoint, whose location is found in the control file (global/pg_control). The error message indicates that at the indicated position in the WAL, no checkpoint is found, so PostgreSQL cannot recover.

You have lost some data that is vital to the database. Perhaps a file got deleted, perhaps you have unreliable storage that loses data that an fsync request confirmed to be on disk.

Restore from a backup or hire a PostgreSQL expert to salvage data from the remains of your database.

Answer 2

Follow the instructions from this page to reset your wal logs.

https://www.postgresql.r2schools.com/postgresql-panic-could-not-locate-a-valid-checkpoint-record/

Essentially you have to su as postgres then run the pg_resetwal program from the postgres bin directory using the data directory as the -f parameter. Example:

$ ./pg_resetwal -f "/var/lib/postgresql/11/main"

This should result in:

Write-ahead log reset

WARNING: You may lose some data or tables.