How to Upgrade PostgreSQL, PostGIS and Patroni in Air-Gapped Environments
Upgrade Your PostgreSQL Database: Major Steps and Minor Touches
I’ve moved my blog from https://insanedba.blogspot.com to https://dincosman.com
Please update your bookmarks and follow/subscribe at the new address for all the latest updates and content. More up-to-date content of this post may be available there.
In this blog post, I will show how a complete PostgreSQL environment major update can be done. Minor upgrades are not a tough task; they are no different from a standard Linux package upgrade. As my production environment serves in an airgapped-environment, I will complete all the major upgrades with no internet connection. I've downloaded all the required whl and .rpm packages before the upgrade.
Before starting, a short description of major and minor PostgreSQL upgrades is provided below:
Major Upgrade:
Upgrading PostgreSQL to a new major version, such as moving from 10 to 11 or 12 to 13. Major upgrades bring significant changes, introduce new features, and may require thorough testing and planning due to potential incompatibilities.
Minor Upgrade:
Updating PostgreSQL to a newer minor version within the same major release, like going from 13.1 to 13.2 or 12.5 to 12.6. Minor upgrades focus on bug fixes, security patches, and performance enhancements, typically requiring less planning and downtime than major upgrades.
I will not only perform a PostgreSQL software update, but also update the following extensions: Patroni, pgbackrest, PostGIS, pg_cron, and Oracle_FDW.
My current environment is:
- 2 node Patroni cluster - current release - 3.0.1
- PostgreSQL software - current release - 14.2
- PostGIS - current release - 3.2.7
- pgbackrest - current release - 2.44
- Oracle_fdw - current release - 2.5.0
- pg_cron - current release - 1.4.2
I will upgrade to :
- Patroni - target release - 3.3.0
- PostgreSQL software - target release - 16.2
- PostGIS - target release - 3.4.2
- pgbackrest - target release - 2.51
- Oracle_fdw - target release - 2.6.0
- pg_cron - target release - 1.6.2
First, I will upgrade the current Patroni release on both servers. I have installed Patroni with pip3 before and will use pip3 for the update.
Here it is.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 Packages]# pip3 show patroni | |
| Name: patroni | |
| Version: 3.0.1 | |
| Summary: PostgreSQL High-Available orchestrator and CLI | |
| Home-page: https://github.com/zalando/patroni | |
| Author: Alexander Kukushkin, Polina Bungina | |
| Author-email: akukushkin@microsoft.com, polina.bungina@zalando.de | |
| License: The MIT License | |
| Location: /usr/local/lib/python3.9/site-packages | |
| Requires: python-dateutil, psutil, ydiff, six, PyYAML, urllib3, click, prettytable | |
| Required-by: | |
| [root@posvt01 patroni3.3]# pip3 install --upgrade ./patroni-3.3.0-py3-none-any.whl | |
| WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead. | |
| Processing ./patroni-3.3.0-py3-none-any.whl | |
| Requirement already satisfied, skipping upgrade: urllib3!=1.21,>=1.19.1 in /usr/local/lib/python3.9/site-packages (from patroni==3.3.0) (1.26.14) | |
| Requirement already satisfied, skipping upgrade: PyYAML in /usr/local/lib64/python3.9/site-packages (from patroni==3.3.0) (6.0) | |
| Requirement already satisfied, skipping upgrade: python-dateutil in /usr/local/lib/python3.9/site-packages (from patroni==3.3.0) (2.8.2) | |
| Requirement already satisfied, skipping upgrade: psutil>=2.0.0 in /usr/local/lib64/python3.9/site-packages (from patroni==3.3.0) (5.9.4) | |
| Requirement already satisfied, skipping upgrade: click>=4.1 in /usr/local/lib/python3.9/site-packages (from patroni==3.3.0) (8.1.3) | |
| Requirement already satisfied, skipping upgrade: ydiff>=1.2.0 in /usr/local/lib/python3.9/site-packages (from patroni==3.3.0) (1.2) | |
| Requirement already satisfied, skipping upgrade: prettytable>=0.7 in /usr/local/lib/python3.9/site-packages (from patroni==3.3.0) (3.6.0) | |
| Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil->patroni==3.3.0) (1.16.0) | |
| Requirement already satisfied, skipping upgrade: wcwidth in /usr/local/lib/python3.9/site-packages (from prettytable>=0.7->patroni==3.3.0) (0.2.6) | |
| Installing collected packages: patroni | |
| Attempting uninstall: patroni | |
| Found existing installation: patroni 3.0.1 | |
| Uninstalling patroni-3.0.1: | |
| Successfully uninstalled patroni-3.0.1 | |
| Successfully installed patroni-3.3.0 | |
| [postgres@posvt01 ~]$ pip3 show patroni | |
| Name: patroni | |
| Version: 3.3.0 | |
| Summary: PostgreSQL High-Available orchestrator and CLI | |
| Home-page: https://github.com/zalando/patroni | |
| Author: Alexander Kukushkin, Polina Bungina | |
| Author-email: akukushkin@microsoft.com, polina.bungina@zalando.de | |
| License: The MIT License | |
| Location: /usr/local/lib/python3.9/site-packages | |
| Requires: ydiff, psutil, python-dateutil, prettytable, click, PyYAML, urllib3 | |
| Required-by: |
Upgrading pgbackrest is similar to a standard Linux package upgrade.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 postgres16]# yum upgrade ./pgbackrest-2.51-1PGDG.rhel8.x86_64.rpm ./libssh2-1.8.0-8.module+el8.0.0+5339+ebcc8ca7.1.x86_64.rpm |
Prior to installing the new PostgreSQL release, I'll update the current PostGIS version to the latest one. PostGIS updates can be a bit complicated. Since two different GDAL libraries cannot be installed simultaneously, I utilized the --allowerasing flag. The provided command reflects the setup in my environment. Next, I'll proceed to install PostgreSQL 16 release along with PostGIS 3.4.2 for PostgreSQL 16.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 postgres16]# yum install ./libdeflate-1.9-3.el8.x86_64.rpm ./re2-20190801-1.el8.x86_64.rpm ./libkml-1.3.0-24.el8.x86_64.rpm \ | |
| ./geos312-3.12.1-1PGDG.rhel8.x86_64.rpm ./libgeotiff17-1.7.1-6PGDG.rhel8.x86_64.rpm ./libgeotiff17-devel-1.7.1-6PGDG.rhel8.x86_64.rpm \ | |
| ./proj94-9.4.0-1PGDG.rhel8.x86_64.rpm ./libspatialite50-5.1.0-5PGDG.rhel8.x86_64.rpm ./libspatialite50-devel-5.1.0-5PGDG.rhel8.x86_64.rpm \ | |
| ./uriparser-0.9.7-1.el8.x86_64.rpm ./postgis34_14-3.4.2-3PGDG.rhel8.x86_64.rpm ./gdal38-libs-3.8.5-3PGDG.rhel8.x86_64.rpm \ | |
| ./libqhull_r-2015.2-5.el8.x86_64.rpm ./libarrow-8.0.1-2.el8.x86_64.rpm ./armadillo-12.6.6-1.el8.x86_64.rpm --allowerasing | |
| Dependencies resolved. | |
| ========================================================================================================================================= | |
| Package Architecture Version Repository Size | |
| ========================================================================================================================================= | |
| Installing: | |
| gdal38-libs x86_64 3.8.5-3PGDG.rhel8 @commandline 9.3 M | |
| geos312 x86_64 3.12.1-1PGDG.rhel8 @commandline 1.2 M | |
| libarrow x86_64 8.0.1-2.el8 @commandline 4.7 M | |
| libdeflate x86_64 1.9-3.el8 @commandline 54 k | |
| libgeotiff17 x86_64 1.7.1-6PGDG.rhel8 @commandline 126 k | |
| libgeotiff17-devel x86_64 1.7.1-6PGDG.rhel8 @commandline 33 k | |
| libkml x86_64 1.3.0-24.el8 @commandline 386 k | |
| libqhull_r x86_64 2015.2-5.el8 @commandline 170 k | |
| proj94 x86_64 9.4.0-1PGDG.rhel8 @commandline 2.9 M | |
| re2 x86_64 20190801-1.el8 @commandline 192 k | |
| uriparser x86_64 0.9.7-1.el8 @commandline 71 k | |
| Upgrading: | |
| armadillo x86_64 12.6.6-1.el8 @commandline 42 k | |
| libspatialite50 x86_64 5.1.0-5PGDG.rhel8 @commandline 3.3 M | |
| libspatialite50-devel x86_64 5.1.0-5PGDG.rhel8 @commandline 102 k | |
| postgis32_14 x86_64 3.4.2-3PGDG.rhel8 @commandline 4.0 M | |
| Removing dependent packages: | |
| gdal34-libs x86_64 3.4.3-2.rhel8 @@commandline 28 M | |
| Transaction Summary | |
| ========================================================================================================================================= | |
| Install 11 Packages | |
| Upgrade 4 Packages | |
| Remove 1 Package | |
| Total size: 27 M | |
| Is this ok [y/N]: y | |
| Downloading Packages: | |
| Running transaction check | |
| Transaction check succeeded. | |
| Running transaction test | |
| Transaction test succeeded. | |
| Running transaction | |
| Preparing : 1/1 | |
| Installing : proj94-9.4.0-1PGDG.rhel8.x86_64 1/20 | |
| Running scriptlet: proj94-9.4.0-1PGDG.rhel8.x86_64 1/20 | |
| Installing : geos312-3.12.1-1PGDG.rhel8.x86_64 2/20 | |
| Running scriptlet: geos312-3.12.1-1PGDG.rhel8.x86_64 2/20 | |
| Upgrading : libspatialite50-5.1.0-5PGDG.rhel8.x86_64 3/20 | |
| Running scriptlet: libspatialite50-5.1.0-5PGDG.rhel8.x86_64 3/20 | |
| Installing : libgeotiff17-1.7.1-6PGDG.rhel8.x86_64 4/20 | |
| Running scriptlet: libgeotiff17-1.7.1-6PGDG.rhel8.x86_64 4/20 | |
| Installing : libgeotiff17-devel-1.7.1-6PGDG.rhel8.x86_64 5/20 | |
| Upgrading : libspatialite50-devel-5.1.0-5PGDG.rhel8.x86_64 6/20 | |
| Running scriptlet: libspatialite50-devel-5.1.0-5PGDG.rhel8.x86_64 6/20 | |
| Upgrading : armadillo-12.6.6-1.el8.x86_64 7/20 | |
| Installing : libqhull_r-2015.2-5.el8.x86_64 8/20 | |
| Running scriptlet: libqhull_r-2015.2-5.el8.x86_64 8/20 | |
| Installing : uriparser-0.9.7-1.el8.x86_64 9/20 | |
| Installing : libkml-1.3.0-24.el8.x86_64 10/20 | |
| Installing : re2-20190801-1.el8.x86_64 11/20 | |
| Installing : libarrow-8.0.1-2.el8.x86_64 12/20 | |
| Installing : libdeflate-1.9-3.el8.x86_64 13/20 | |
| Installing : gdal38-libs-3.8.5-3PGDG.rhel8.x86_64 14/20 | |
| Upgrading : postgis34_14-3.4.2-3PGDG.rhel8.x86_64 15/20 | |
| Cleanup : postgis32_14-3.2.7-1.rhel8.x86_64 16/20 | |
| Erasing : gdal34-libs-3.4.3-2.rhel8.x86_64 17/20 | |
| Running scriptlet: gdal34-libs-3.4.3-2.rhel8.x86_64 17/20 | |
| Cleanup : libspatialite50-devel-5.0.1-4.rhel8.x86_64 18/20 | |
| Running scriptlet: libspatialite50-devel-5.0.1-4.rhel8.x86_64 18/20 | |
| Cleanup : libspatialite50-5.0.1-4.rhel8.x86_64 19/20 | |
| Running scriptlet: libspatialite50-5.0.1-4.rhel8.x86_64 19/20 | |
| Cleanup : armadillo-10.8.2-1.el8.x86_64 20/20 | |
| Running scriptlet: armadillo-10.8.2-1.el8.x86_64 20/20 | |
| Verifying : libdeflate-1.9-3.el8.x86_64 1/20 | |
| Verifying : re2-20190801-1.el8.x86_64 2/20 | |
| Verifying : libkml-1.3.0-24.el8.x86_64 3/20 | |
| Verifying : geos312-3.12.1-1PGDG.rhel8.x86_64 4/20 | |
| Verifying : libgeotiff17-1.7.1-6PGDG.rhel8.x86_64 5/20 | |
| Verifying : libgeotiff17-devel-1.7.1-6PGDG.rhel8.x86_64 6/20 | |
| Verifying : proj94-9.4.0-1PGDG.rhel8.x86_64 7/20 | |
| Verifying : uriparser-0.9.7-1.el8.x86_64 8/20 | |
| Verifying : gdal38-libs-3.8.5-3PGDG.rhel8.x86_64 9/20 | |
| Verifying : libqhull_r-2015.2-5.el8.x86_64 10/20 | |
| Verifying : libarrow-8.0.1-2.el8.x86_64 11/20 | |
| Verifying : libspatialite50-5.1.0-5PGDG.rhel8.x86_64 12/20 | |
| Verifying : libspatialite50-5.0.1-4.rhel8.x86_64 13/20 | |
| Verifying : libspatialite50-devel-5.1.0-5PGDG.rhel8.x86_64 14/20 | |
| Verifying : libspatialite50-devel-5.0.1-4.rhel8.x86_64 15/20 | |
| Verifying : postgis32_14-3.4.2-3PGDG.rhel8.x86_64 16/20 | |
| Verifying : postgis32_14-3.4.2-1.rhel8.x86_64 17/20 | |
| Verifying : armadillo-12.6.6-1.el8.x86_64 18/20 | |
| Verifying : armadillo-10.8.2-1.el8.x86_64 19/20 | |
| Verifying : gdal34-libs-3.4.3-2.rhel8.x86_64 20/20 | |
| Upgraded: | |
| armadillo-12.6.6-1.el8.x86_64 libspatialite50-5.1.0-5PGDG.rhel8.x86_64 libspatialite50-devel-5.1.0-5PGDG.rhel8.x86_64 | |
| postgis32_14-3.4.2-3PGDG.rhel8.x86_64 | |
| Installed: | |
| gdal38-libs-3.8.5-3PGDG.rhel8.x86_64 geos312-3.12.1-1PGDG.rhel8.x86_64 libarrow-8.0.1-2.el8.x86_64 | |
| libdeflate-1.9-3.el8.x86_64 libgeotiff17-1.7.1-6PGDG.rhel8.x86_64 libgeotiff17-devel-1.7.1-6PGDG.rhel8.x86_64 | |
| libkml-1.3.0-24.el8.x86_64 libqhull_r-2015.2-5.el8.x86_64 proj94-9.4.0-1PGDG.rhel8.x86_64 | |
| re2-20190801-1.el8.x86_64 uriparser-0.9.7-1.el8.x86_64 | |
| Removed: | |
| gdal34-libs-3.4.3-2.rhel8.x86_64 | |
| postgres=# select postgis.postgis_full_version(); | |
| postgis_full_version | |
| ----------------------------------------------------------------------------------------------------------------------------------------- | |
| ----------------------------------------------------------------------------------------------------------------------------------------- | |
| ---------------------------------------------------------------------------------------------- | |
| POSTGIS="3.4.2 c19ce56" [EXTENSION] PGSQL="140" GEOS="3.12.1-CAPI-1.18.1" PROJ="9.4.0 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj. | |
| org USER_WRITABLE_DIRECTORY=/var/lib/pgsql/.local/share/proj DATABASE_PATH=/usr/proj94/share/proj/proj.db" LIBXML="2.9.7" LIBJSON="0.13.1 | |
| " LIBPROTOBUF="1.3.0" WAGYU="0.5.0 (Internal)" (core procs from "3.2.7 32dc0cc" need upgrade) | |
| postgres=# SELECT postgis.PostGIS_Extensions_Upgrade(); | |
| NOTICE: Updating extension postgis 3.4.2 | |
| postgis_extensions_upgrade | |
| ------------------------------------------------------------------------------------ | |
| Upgrade to version 3.4.2 completed, run SELECT postgis_full_version(); for details | |
| (1 row) | |
| postgres=# select postgis.postgis_full_version(); | |
| postgis_full_version | |
| ----------------------------------------------------------------------------------------------------------------------------------------- | |
| ----------------------------------------------- | |
| POSTGIS="3.4.2 c19ce56" [EXTENSION] PGSQL="140" GEOS="3.12.1-CAPI-1.18.1" PROJ="9.4.0 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj. | |
| org USER_WRITABLE_DIRECTORY=/var/lib/pgsql/.local/share/proj DATABASE_PATH=/usr/proj94/share/proj/proj.db" LIBXML="2.9.7" LIBJSON="0.13.1 | |
| " LIBPROTOBUF="1.3.0" WAGYU="0.5.0 (Internal)" | |
| (1 row) |
Now I will install PostgreSQL 16 release and PostGIS 3.4.2 for PostgreSQL 16.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 postgres16]# yum install ./postgresql16-libs-16.2-1PGDG.rhel8.x86_64.rpm ./postgresql16-16.2-1PGDG.rhel8.x86_64.rpm | |
| [root@posvt01 postgres16]# yum install ./postgresql16-server-16.2-1PGDG.rhel8.x86_64.rpm ./postgresql16-contrib-16.2-1PGDG.rhel8.x86_64.rpm | |
| [root@posvt01 ~]# vi .bash_profile | |
| ... | |
| PATH=$PATH:$HOME/bin:/usr/pgsql-16/bin | |
| [root@posvt01 ~]# yum install ./Packages/postgres16/postgis34_16-3.4.2-3PGDG.rhel8.x86_64.rpm |
At this point, I will proceed to deploy Oracle_FDW and pg_cron extensions for PostgreSQL 16.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 postgres16]# yum install ./postgresql16-devel-16.2-1PGDG.rhel8.x86_64.rpm ./perl-IPC-Run-0.99-1.el8.noarch.rpm \ | |
| ./perl-IO-Tty-1.12-11.el8.x86_64.rpm ./perl-Test-Simple-1.302135-1.el8.noarch.rpm ./perl-Time-HiRes-1.9758-2.el8.x86_64.rpm | |
| [root@posvt01 Packages]# unzip oracle_fdw-ORACLE_FDW_2_6_0.zip | |
| [root@posvt01 Packages]# cd oracle_fdw-ORACLE_FDW_2_6_0/ | |
| [root@posvt01 oracle_fdw-ORACLE_FDW_2_6_0]# export LD_LIBRARY_PATH=/usr/lib/oracle/19.14/client64 | |
| [root@posvt01 oracle_fdw-ORACLE_FDW_2_6_0]# export ORACLE_HOME=/usr/lib/oracle/19.14/client64 | |
| [root@posvt01 oracle_fdw-ORACLE_FDW_2_6_0]# make | |
| [root@posvt01 oracle_fdw-ORACLE_FDW_2_6_0]# make install | |
| /usr/bin/mkdir -p '/usr/pgsql-16/lib' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/share/extension' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/share/extension' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/doc/extension' | |
| /usr/bin/install -c -m 755 oracle_fdw.so '/usr/pgsql-16/lib/oracle_fdw.so' | |
| /usr/bin/install -c -m 644 .//oracle_fdw.control '/usr/pgsql-16/share/extension/' | |
| /usr/bin/install -c -m 644 .//oracle_fdw--1.2.sql .//oracle_fdw--1.0--1.1.sql .//oracle_fdw--1.1--1.2.sql '/usr/pgsql-16/share/extension/' | |
| /usr/bin/install -c -m 644 .//README.oracle_fdw '/usr/pgsql-16/doc/extension/' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@posvt01 pg_cron]# unzip pg_cron-main.zip | |
| [root@posvt01 pg_cron]# cd pg_cron-main/ | |
| [root@posvt01 pg_cron-main]# make | |
| [root@posvt01 pg_cron-main]# make install | |
| /usr/bin/mkdir -p '/usr/pgsql-16/lib' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/share/extension' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/share/extension' | |
| /usr/bin/install -c -m 755 pg_cron.so '/usr/pgsql-16/lib/pg_cron.so' | |
| /usr/bin/install -c -m 644 .//pg_cron.control '/usr/pgsql-16/share/extension/' | |
| /usr/bin/install -c -m 644 .//pg_cron--1.4-1--1.5.sql .//pg_cron--1.3--1.4.sql .//pg_cron--1.5--1.6.sql .//pg_cron--1.2--1.3.sql .//pg_cron--1.0--1.1.sql .//pg_cron--1.4--1.4-1.sql .//pg_cron--1.1--1.2.sql pg_cron--1.0.sql '/usr/pgsql-16/share/extension/' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/lib/bitcode/pg_cron' | |
| /usr/bin/mkdir -p '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| /usr/bin/install -c -m 644 src/pg_cron.bc '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| /usr/bin/install -c -m 644 src/job_metadata.bc '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| /usr/bin/install -c -m 644 src/misc.bc '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| /usr/bin/install -c -m 644 src/task_states.bc '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| /usr/bin/install -c -m 644 src/entry.bc '/usr/pgsql-16/lib/bitcode'/pg_cron/src/ | |
| cd '/usr/pgsql-16/lib/bitcode' && /usr/bin/llvm-lto -thinlto -thinlto-action=thinlink -o pg_cron.index.bc pg_cron/src/pg_cron.bc pg_cron/src/job_metadata.bc pg_cron/src/misc.bc pg_cron/src/task_states.bc pg_cron/src/entry.bc |
Up to this point, we've updated Patroni, pgbackrest, and PostGIS to their latest versions and installed PostgreSQL 16 along with the PostGIS 3.4.2 packages. Following the installation of PostgreSQL 16, I've compiled the pg_cron and Oracle_FDW extensions for the newer PostgreSQL version.
At this stage, it's essential to pay extra attention to a few key points.
We'll begin by stopping Patroni and the PostgreSQL database. From this point onward, downtime commences. Since our database is relatively small in size, we've opted to utilize the pg_upgrade method for the upgrade process. In cases where the database size exceeds 2 TB, the --link or --clone flags may be employed. Alternatively, pg_dump is also a viable option. For situations where no large objects are stored in the database, upgrading with a logical streaming method is feasible. However, it's important to note that as of PostgreSQL 16, large objects are not replicated due to Logical replication restrictions. Unfortunately, there is no workaround for this limitation other than storing data in normal tables.
We'll initiate the creation of a new PostgreSQL cluster database using 'initdb' from the newly installed PostgreSQL 16 directory, ensuring it conforms to the current locale and WAL segment size settings. Following the setup of the new data directory, it's crucial to carefully configure the new PostgreSQL database. During the upgrade phase, it's essential to use a different port for the new version since both databases will be operational simultaneously. Additionally, parameters related to the data directory (such as hba_file and ident_file) need to be verified.
Moreover, it's important to monitor the available memory resources on your server. You may need to adjust the shared_buffers to a lower value temporarily during the upgrade phase to accommodate the operation of two different PostgreSQL cluster databases simultaneously.
The upgrade process begins with running 'pg_upgrade' with the check flag to ensure all necessary checks are passed. Once verified, the 'pg_upgrade' process will be executed. Upon completion of the upgrade phase, 'update_extensions.sql' will be executed, followed by a vacuum analyze operation to generate fresh statistics for the new cluster.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [postgres@posvt01 pg_cron-main]# mkdir -p /mnt/postgres/pgdata/16 | |
| [postgres@posvt01 pg_cron-main]# mkdir -p /mnt/postgres/pg_wal/16 | |
| [root@posvt01 ~]# systemctl stop patroni | |
| [postgres@posvt01 ~]$ /usr/pgsql-16/bin/initdb -U postgres --data-checksums -E 'UTF8' -W --wal-segsize='32' \ | |
| --waldir='/mnt/postgres/pg_wal/16' --lc-ctype='tr_TR.UTF-8' --lc-collate='tr_TR.UTF-8' --pgdata='/mnt/postgres/pgdata/16' | |
| The files belonging to this database system will be owned by user "postgres". | |
| This user must also own the server process. | |
| The database cluster will be initialized with this locale configuration: | |
| provider: libc | |
| LC_COLLATE: tr_TR.UTF-8 | |
| LC_CTYPE: tr_TR.UTF-8 | |
| LC_MESSAGES: en_US.utf8 | |
| LC_MONETARY: en_US.utf8 | |
| LC_NUMERIC: en_US.utf8 | |
| LC_TIME: en_US.utf8 | |
| The default text search configuration will be set to "turkish". | |
| Data page checksums are enabled. | |
| Enter new superuser password: | |
| Enter it again: | |
| fixing permissions on existing directory /mnt/postgres/pgdata/16 ... ok | |
| fixing permissions on existing directory /mnt/postgres/pg_wal/16 ... ok | |
| creating subdirectories ... ok | |
| selecting dynamic shared memory implementation ... posix | |
| selecting default max_connections ... 100 | |
| selecting default shared_buffers ... 128MB | |
| selecting default time zone ... Europe/Istanbul | |
| creating configuration files ... ok | |
| running bootstrap script ... ok | |
| performing post-bootstrap initialization ... ok | |
| syncing data to disk ... ok | |
| initdb: warning: enabling "trust" authentication for local connections | |
| initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. | |
| Success. You can now start the database server using: | |
| /usr/pgsql-16/bin/pg_ctl -D /mnt/postgres/pgdata/16 -l logfile start | |
| [postgres@posvt01 14]$ cp /mnt/postgres/pgdata/14/pg_hba.conf /mnt/postgres/pgdata/16/ | |
| [postgres@posvt01 14]$ cp /mnt/postgres/pgdata/14/pg_ident.conf /mnt/postgres/pgdata/16/ | |
| [postgres@posvt01 14]$ cp /mnt/postgres/pgdata/14/postgresql.conf /mnt/postgres/pgdata/16/ | |
| [postgres@posvt01 14]$ cp /mnt/postgres/pgdata/14/postgresql.base.conf /mnt/postgres/pgdata/16/postgresql.base.conf | |
| -- Check for parameters | |
| [postgres@posvt01 14]$ vi /mnt/postgres/pgdata/16/postgresql.conf | |
| ... | |
| port = '3131' | |
| hba_file = '/mnt/postgres/pgdata/16/pg_hba.conf' | |
| ident_file = '/mnt/postgres/pgdata/16/pg_ident.conf' | |
| unix_socket_directories = '/tmp' | |
| ... | |
| -- An upgrade check | |
| [postgres@posvt01 tmp]$ /usr/pgsql-16/bin/pg_upgrade --check -U postgres -b /usr/pgsql-14/bin/ -B /usr/pgsql-16/bin/ \ | |
| -d /mnt/postgres/pgdata/14/ -D /mnt/postgres/pgdata/16/ -p 3531 -P 3131 -o "-c config_file=/mnt/postgres/pgdata/14/postgresql.conf" \ | |
| -O "-c config_file=/mnt/postgres/pgdata/16/postgresql.conf" --socketdir='/tmp/' | |
| Performing Consistency Checks on Old Live Server | |
| ------------------------------------------------ | |
| Checking cluster versions ok | |
| Checking database user is the install user ok | |
| Checking database connection settings ok | |
| Checking for prepared transactions ok | |
| Checking for system-defined composite types in user tables ok | |
| Checking for reg* data types in user tables ok | |
| Checking for contrib/isn with bigint-passing mismatch ok | |
| Checking for incompatible "aclitem" data type in user tables ok | |
| Checking for presence of required libraries ok | |
| Checking database user is the install user ok | |
| Checking for prepared transactions ok | |
| Checking for new cluster tablespace directories ok | |
| *Clusters are compatible* | |
| [postgres@posvt01 ~]$ /usr/pgsql-16/bin/pg_upgrade -U postgres -b /usr/pgsql-14/bin/ -B /usr/pgsql-16/bin/ \ | |
| -d /mnt/postgres/pgdata/14/ -D /mnt/postgres/pgdata/16/ -p 3531 -P 3131 -o "-c config_file=/mnt/postgres/pgdata/14/postgresql.conf" \ | |
| -O "-c config_file=/mnt/postgres/pgdata/16/postgresql.conf" --socketdir='/tmp/' | |
| Performing Consistency Checks | |
| ----------------------------- | |
| Checking cluster versions ok | |
| Checking database user is the install user ok | |
| Checking database connection settings ok | |
| Checking for prepared transactions ok | |
| Checking for system-defined composite types in user tables ok | |
| Checking for reg* data types in user tables ok | |
| Checking for contrib/isn with bigint-passing mismatch ok | |
| Checking for incompatible "aclitem" data type in user tables ok | |
| Creating dump of global objects ok | |
| Creating dump of database schemas | |
| ok | |
| Checking for presence of required libraries ok | |
| Checking database user is the install user ok | |
| Checking for prepared transactions ok | |
| Checking for new cluster tablespace directories ok | |
| If pg_upgrade fails after this point, you must re-initdb the | |
| new cluster before continuing. | |
| Performing Upgrade | |
| ------------------ | |
| Setting locale and encoding for new cluster ok | |
| Analyzing all rows in the new cluster ok | |
| Freezing all rows in the new cluster ok | |
| Deleting files from new pg_xact ok | |
| Copying old pg_xact to new server ok | |
| Setting oldest XID for new cluster ok | |
| Setting next transaction ID and epoch for new cluster ok | |
| Deleting files from new pg_multixact/offsets ok | |
| Copying old pg_multixact/offsets to new server ok | |
| Deleting files from new pg_multixact/members ok | |
| Copying old pg_multixact/members to new server ok | |
| Setting next multixact ID and offset for new cluster ok | |
| Resetting WAL archives ok | |
| Setting frozenxid and minmxid counters in new cluster ok | |
| Restoring global objects in the new cluster ok | |
| Restoring database schemas in the new cluster | |
| ok | |
| Copying user relation files | |
| ok | |
| Setting next OID for new cluster ok | |
| Sync data directory to disk ok | |
| Creating script to delete old cluster ok | |
| Checking for extension updates notice | |
| Your installation contains extensions that should be updated | |
| with the ALTER EXTENSION command. The file | |
| update_extensions.sql | |
| when executed by psql by the database superuser will update | |
| these extensions. | |
| Upgrade Complete | |
| ---------------- | |
| Optimizer statistics are not transferred by pg_upgrade. | |
| Once you start the new server, consider running: | |
| /usr/pgsql-16/bin/vacuumdb -U postgres --all --analyze-in-stages | |
| Running this script will delete the old cluster's data files: | |
| ./delete_old_cluster.sh | |
| [postgres@posvt01 ~]$ psql -f update_extensions.sql | |
| [postgres@posvt01 ~]$ /usr/pgsql-16/bin/vacuumdb -U postgres --all --analyze-in-stages |
Once the upgrade phase is successfully completed, we may proceed to remove the current Patroni cluster configuration. Before starting Patroni, it's essential to update the current pgbackrest configuration to align with the new pgdata directories, and the stanza should also be upgraded accordingly. After ensuring all preceding steps are confirmed to be completed successfully, we can initialize Patroni after verifying the configuration (version-specific directory related parameters) in the patroni.yml file on both the master and standby servers.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [postgres@posvt01 ~]$ patronictl -c /etc/patroni.yml remove pgcluster | |
| + Cluster: pgcluster (7099446795009447890) -----+ | |
| | Member | Host | Role | State | TL | Lag in MB | | |
| +--------+------+------+-------+----+-----------+ | |
| +--------+------+------+-------+----+-----------+ | |
| Please confirm the cluster name to remove: pgcluster | |
| You are about to remove all information in DCS for pgcluster, please type: "Yes I am aware": Yes I am aware | |
| #-- Environment variables provided below are updated | |
| [postgres@posvt01 ~]$ vi /var/lib/pgsql/.pgsql_profile | |
| ... | |
| export PGDATA=/mnt/postgres/pgdata/16 | |
| export PGHOME=/usr/pgsql-16/ | |
| #-- Environment variables provided below are updated | |
| [postgres@posvt02 ~]$ vi /var/lib/pgsql/.pgsql_profile | |
| ... | |
| export PGDATA=/mnt/postgres/pgdata/16 | |
| export PGHOME=/usr/pgsql-16/ | |
| #-- parameters provided below are updated | |
| [root@posvt01 ~]# vi /etc/patroni.yml | |
| initdb: | |
| - waldir: '/mnt/postgres/pg_wal/16' | |
| ... | |
| postgresql: | |
| ... | |
| data_dir: /mnt/postgres/pgdata/16 | |
| bin_dir: /usr/pgsql-16/bin | |
| ... | |
| basebackup: | |
| ... | |
| waldir: /mnt/postgres/pg_wal/16 | |
| #-- parameters provided below are updated | |
| [postgres@posvt01 ~]$ vi /etc/pgbackrest/pgbackrest.conf | |
| pg1-path=/mnt/postgres/pgdata/16 | |
| ... | |
| #-- parameters provided below are updated | |
| [postgres@posvt02 ~]$ vi /etc/pgbackrest/pgbackrest.conf | |
| pg1-path=/mnt/postgres/pgdata/16 | |
| ... | |
| #-- parameters provided below are updated | |
| [pgbackrest@etcd03 ~]# vi /etc/pgbackrest/pgbackrest.conf | |
| pg1-path=/mnt/postgres/pgdata/16 | |
| pg1-port=3531 | |
| pg1-socket-path=/var/run/postgresql | |
| pg2-host=posvt02.localdomain | |
| pg2-path=/mnt/postgres/pgdata/16 | |
| ... | |
| [pgbackrest@etcd03 ~]$ pgbackrest --stanza=pgcluster --no-online stanza-upgrade | |
| 2024-04-18 21:53:00.857 P00 INFO: stanza-upgrade command begin 2.51: --exec-id=15361-8ec0423f --log-level-console=detail --log-level-file=off --no-online --pg1-host=posvt01.localdomain --pg2-host=posvt02.localdomain --pg1-host-ca-file=/etc/ssl/pgbackrest/root.crt --pg2-host-ca-file=/etc/ssl/pgbackrest/root.crt --pg1-host-cert-file=/etc/ssl/pgbackrest/serverbckp.crt --pg2-host-cert-file=/etc/ssl/pgbackrest/serverbckp.crt --pg1-host-key-file=/etc/ssl/pgbackrest/serverbckp.key --pg2-host-key-file=/etc/ssl/pgbackrest/serverbckp.key --pg1-host-type=tls --pg2-host-type=tls --pg1-host-user=postgres --pg2-host-user=postgres --pg1-path=/mnt/postgres/pgdata/16 --pg2-path=/mnt/postgres/pgdata/16 --pg1-port=3535 --pg2-port=3535 --pg1-socket-path=/var/run/postgresql --pg2-socket-path=/var/run/postgresql --repo1-path=/pgbackrest --stanza=pgcluster | |
| 2024-04-18 21:53:00.872 P00 INFO: stanza-upgrade for stanza 'pgcluster' on repo1 | |
| 2024-04-18 21:53:00.944 P00 DETAIL: statistics: {"socket.client":{"total":1},"socket.session":{"total":1},"tls.client":{"total":1},"tls.session":{"total":1}} | |
| 2024-04-18 21:53:00.944 P00 INFO: stanza-upgrade command end: completed successfully (88ms) | |
| [root@posvt01 ~]# systemctl start patroni | |
| [pgbackrest@etcd03 ~]$ pgbackrest --type=full --stanza=pgcluster backup | |
| [root@posvt02 ~]# systemctl start patroni |
I've provided a sample of a comprehensive PostgreSQL environment update. Choosing to upgrade your major version annually offers a dependable solution. Approximately every quarter, a new minor version becomes available. It's recommended to apply these updates without delay, as they primarily focus on critical bug fixes and are notably easier to install compared to the major release upgrades.
Hope it helps.
Comments
Post a Comment