Barman安装及备份PostgreSQL

时间：2021-12-22 02:39:11

barman特点

  1. 零数据丢失备份。保证用户在只有一台备份服务器的情况下达到零数据丢失。

  2. 与备份服务器合作。允许备份服务器在与主服务器的流式复制不可用时，从barman获取wal文件。

  3. 可靠的监控集成。用于监控和创建有关Barman配置状态的报告。

  4. 可并行的备份和恢复。以更快的速度执行拷贝，加快了整个备份和恢复过程。

  5. 备份目录。允许用户在同一界面中列出、保留、删除、存档和恢复多个完整备份。

  6. 支持多个服务器。允许您从同一位置管理多个服务器的备份和恢复。

  7. 存储备份数据。存储来自多个PostgreSQL实例的备份数据，支持不同的版本。

  8. 完全热备份。从其他服务器执行PostgreSQL服务器的完全热备份，专门为备份和灾难恢复而设计。

  9. 增量备份和恢复。针对新的和修改过的数据生成不同的备份，从而加快处理速度。

  10. 保留策略。配置自定义备份保留策略以满足组织的需要。

  11. 远程恢复。执行PostgreSQL服务器备份的远程恢复。

  12. Wal文件压缩。自定义归档和压缩WAL文件和备份。

  13. 备份信息。生成所有备份的常规和磁盘使用信息。

  14. 地理冗余。给DBA和系统管理员为PostgreSQL数据库的灾难恢复解决方案增加另一层保障。

barman安装

192.168.2.2 pg （postgresql server服务器）

192.168.2.3 barman （barman备份服务器）

postgresql版本11.2

barman版本2.9.1

yum安装postgresql

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/pgdg-centos11-11-2.noarch.rpm yum install -y postgresql11-server-11.2 postgresql11-contrib-11.2 /usr/pgsql-11/bin/postgresql-11-setup initdb #执行数据库初始化

修改postgres.conf配置

listen_addresses = ‘*‘ port = 5432 wal_level = replica #11版本后改为replica archive_mode = on archive_command = ‘cp %p /var/lib/pgsql/11/data/pg_archive/%f‘ max_wal_senders = 10 #建议大于等于3 wal_keep_segments = 64 synchronous_standby_names = ‘*‘ hot_standby = on log_filename = ‘postgresql-%Y%m%d_%H%M.log‘

修改pg_hba.conf配置

local   all             all                                       trust
host    all             all               0.0.0.0/0               md5
host    all             all               ::1/128                 trust
host    replication     barman            0.0.0.0/0               md5
host    replication     streaming_barman  0.0.0.0/0               md5

创建pg_archive，并设置用户组

mkdir /var/lib/pgsql/11/data/pg_archive chown postgres:postgres /var/lib/pgsql/11/data/pg_archive -R

启动postgresql

systemctl enable postgresql-11 systemctl start postgresql-11

创建barman和barman_streaming用户

su postgres createuser -s -P barman #设置密码123456 createuser -P --replication streaming_barman #设置密码123456 #创建.pgpass touch ~/.pgpass #添加 pg:5432:*:streaming_barman:123456 pg:5432:*:barman:123456 #更改权限 chmod 600 ~/.pgpass #验证 psql -c ‘SELECT version()‘ -U barman -h pg postgres

yum安装barman

由于barman是python写的，且有些功能需要使用postgresql的工具，安装时会有些依赖

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/postgresql11-libs-11.2-1PGDG.rhel7.x86_64.rpm yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/python2-psycopg2-2.8.2-1.rhel7.x86_64.rpm yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-2.9-1.rhel7.noarch.rpm yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-cli-2.9-1.rhel7.noarch.rpm

修改barman的公共配置

vi /etc/barman.conf #修改以下配置 parallel_jobs = 2 #并发数 immediate_checkpoint = true #在执行back时是否会自动执行checkpoint network_compression = true #网络传输压缩

修改barman对postgresql server的私有配置

cd /etc/barman.d/ cp streaming-server.conf-template pg.conf #将pg.conf中的[streaming]修改为[pg]

将192.168.2.2 pg添加到/etc/hosts中

barman使用

1.创建复制槽

barman receive-wal --create-slot pg

2.barman check

[[email protected] pg]# barman check pg Server pg: PostgreSQL: OK is_superuser: OK PostgreSQL streaming: OK wal_level: OK replication slot: OK directories: OK retention policy settings: OK backup maximum age: OK (no last_backup_maximum_age provided) compression settings: OK failed backups: OK (there are 0 failed backups) minimum redundancy requirements: FAILED (have 0 backups, expected at least 1) pg_basebackup: OK pg_basebackup compatible: OK pg_basebackup supports tablespaces mapping: OK pg_receivexlog: OK pg_receivexlog compatible: OK receive-wal running: OK archiver errors: OK

其中minimum redundancy requirements: FAILED可忽略，首次配置后还没有全备，执行全备会变为OK

[[email protected] pg]# barman backup pg Starting backup using postgres method for server pg in /var/lib/barman/pg/base/20190821T171724 Backup start at LSN: 0/90000D0 (000000010000000000000009, 000000D0) Starting backup copy via pg_basebackup for 20190821T171724 Copy done (time: 2 seconds) Finalising the backup. This is the first backup for server pg WAL segments preceding the current backup have been found: 000000010000000000000008 from server pg has been removed Backup size: 166.6 MiB Backup end at LSN: 0/B000060 (00000001000000000000000B, 00000060) Backup completed (start time: 2019-08-21 17:17:24.956763, elapsed time: 2 seconds) Processing xlog segments from streaming for pg 000000010000000000000009 00000001000000000000000A

首次配置完会产生minimum redundancy requirements: FAILED、receive-wal running:FAILED，执行barman cron即可变为OK，barman会执行如下过程

[[email protected] pg]# barman cron Starting WAL archiving for server pg Starting streaming archiver for server pg Starting check-backup for backup 20190821T172217 of server pg #正常流复制时的进程 [[email protected] pg]# ps -ef|grep barman root 1278 1 0 17:30 ? 00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg root 1281 1278 0 17:30 ? 00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous #执行全备时的进程 [[email protected] ~]# ps -ef|grep barman root 1325 1 1 17:33 ? 00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg root 1327 1325 0 17:33 ? 00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous root 1331 1035 12 17:33 pts/0 00:00:00 /usr/bin/python2 /usr/bin/barman backup pg root 1334 1331 0 17:33 pts/0 00:00:00 /usr/pgsql-11/bin/pg_basebackup --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup -v --no-password --pgdata=/var/lib/barman/pg/base/20190821T173331/data --no-slot --wal-method=none --checkpoint=fast

所以barman的全备是利用pg_basebackup实现，增量是利用pg_receivewal实现。

barman优点

1.可以与postgresql的master和slaver分离部署，但是barman服务器需要安装postgresql的一些工具，如pg_basebackup、pg_receivewal

2.通过流复制可以同步拉取wal，如果postgresql当前的wal还没close，barman会产生一个.partial文件实时同步，若在恢复时使用该文件需要将扩展后缀去掉

3.barman可以一台服务器管理多个postgresql server，提供本地及远程恢复（远程恢复需要配置ssh免密）

4.提供备份相关文件的检测及详细查询

barman部分功能

1.查看当前复制状态

[[email protected] ~]# barman replication-status pg Status of streaming clients for server ‘pg‘: Current LSN on master: 0/130001A8 Number of streaming clients: 1 1. #1 Sync WAL streamer Application name: barman_receive_wal Sync stage : 3/3 Remote write Communication : TCP/IP IP Address : 192.168.2.3 / Port: 33256 / Host: - User name : streaming_barman Current state : streaming (sync) Replication slot: barman WAL sender PID : 1731 Started at : 2019-08-21 17:33:16.495522 08:00 Sent LSN : 0/130001A8 (diff: 0 B) Write LSN : 0/130001A8 (diff: 0 B) Flush LSN : 0/130001A8 (diff: 0 B)

2.查看备份列表

[[email protected] ~]# barman list-backup pg pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 294.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 64.0 MiB pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB

3.查看postgresql server

[[email protected] ~]# barman show-server pg Server pg: active: True archive_timeout: 3600 archiver: False archiver_batch_size: 0 backup_directory: /var/lib/barman/pg backup_method: postgres backup_options: BackupOptions([‘concurrent_backup‘]) bandwidth_limit: None barman_home: /var/lib/barman barman_lock_directory: /var/lib/barman basebackup_retry_sleep: 30 basebackup_retry_times: 3 basebackups_directory: /var/lib/barman/pg/base check_timeout: 30 checkpoint_timeout: 300 compression: None config_file: /var/lib/pgsql/11/data/postgresql.conf connection_error: None conninfo: host=pg user=barman dbname=postgres current_size: 23808077 current_xlog: 000000010000000000000015 custom_compression_filter: None custom_decompression_filter: None data_checksums: off data_directory: /var/lib/pgsql/11/data description: Example of PostgreSQL Database (Streaming-Only) disabled: False errors_directory: /var/lib/barman/pg/errors hba_file: /var/lib/pgsql/11/data/pg_hba.conf hot_standby: on ident_file: /var/lib/pgsql/11/data/pg_ident.conf immediate_checkpoint: True incoming_wals_directory: /var/lib/barman/pg/incoming is_in_recovery: False is_superuser: True last_backup_maximum_age: None max_incoming_wals_queue: None max_replication_slots: 10 max_wal_senders: 10 minimum_redundancy: 1 msg_list: [] name: pg network_compression: False parallel_jobs: 1 passive_node: False path_prefix: /usr/pgsql-11/bin pg_basebackup_bwlimit: True pg_basebackup_compatible: True pg_basebackup_installed: True pg_basebackup_path: /usr/pgsql-11/bin/pg_basebackup pg_basebackup_tbls_mapping: True pg_basebackup_version: 11.2 pg_receivexlog_compatible: True pg_receivexlog_installed: True pg_receivexlog_path: /usr/pgsql-11/bin/pg_receivewal pg_receivexlog_supports_slots: True pg_receivexlog_synchronous: True pg_receivexlog_version: 11.2 pgespresso_installed: False post_archive_retry_script: None post_archive_script: None post_backup_retry_script: None post_backup_script: None post_delete_retry_script: None post_delete_script: None post_recovery_retry_script: None post_recovery_script: None post_wal_delete_retry_script: None post_wal_delete_script: None pre_archive_retry_script: None pre_archive_script: None pre_backup_retry_script: None pre_backup_script: None pre_delete_retry_script: None pre_delete_script: None pre_recovery_retry_script: None pre_recovery_script: None pre_wal_delete_retry_script: None pre_wal_delete_script: None primary_ssh_command: None recovery_options: RecoveryOptions([]) replication_slot: Record(slot_name=‘barman‘, active=True, restart_lsn=‘0/150000C8‘) replication_slot_support: True retention_policy: None retention_policy_mode: auto reuse_backup: None server_txt_version: 11.2 slot_name: barman ssh_command: None streaming: True streaming_archiver: True streaming_archiver_batch_size: 0 streaming_archiver_name: barman_receive_wal streaming_backup_name: barman_streaming_backup streaming_conninfo: host=pg user=streaming_barman streaming_supported: True streaming_wals_directory: /var/lib/barman/pg/streaming synchronous_standby_names: [‘*‘] systemid: 6727448561719488729 tablespace_bandwidth_limit: None timeline: 1 wal_compression: off wal_level: replica wal_retention_policy: main wals_directory: /var/lib/barman/pg/wals xlog_segment_size: 16777216 xlogpos: 0/150000C8

4.远程执行switch-wal

[[email protected] ~]# barman switch-wal pg The WAL file 000000010000000000000015 has been closed on server ‘pg‘

5.检查backupid

[[email protected] ~]# barman list-backup pg pg 20190821T174304 - Wed Aug 21 17:43:08 2019 - Size: 326.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 310.6 MiB - WAL Size: 16.0 MiB - WAITING_FOR_WALS pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 80.0 MiB pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB [[email protected] ~]# barman check-backup pg 20190821T171724 [[email protected] ~]# barman check-backup pg 20190821T171724s ERROR: Unknown backup ‘20190821T171724s‘ for server ‘pg‘

6.查看文件

–target参数包括：

data：列出data目录的所有文件

standalone：列出全备和需要的wal文件

wal：列出全备开始到当前时间的wal文件

full：等于data   wal

[[email protected] ~]# barman list-files --target wal pg 20190821T173331 /var/lib/barman/pg/wals/0000000100000000/000000010000000000000012 /var/lib/barman/pg/wals/0000000100000000/000000010000000000000013

7.备份恢复

#备份恢复 barman recover <server_name> <backup_id> <postgres server data path> #时间点恢复 barman recover <server_name> <backup_id> <postgres server data path> --target-time TARGET_TIME

还提供：--target-xid TARGET_XID、--target-lsn TARGET_LSN、--target-name TARGET_NAME、--target-immediate

8.查看备份详情

[[email protected] ~]# barman show-backup pg 20190821T173331 Backup 20190821T173331: Server Name : pg Status : WAITING_FOR_WALS PostgreSQL Version : 110002 PGDATA directory : /var/lib/pgsql/11/data Base backup information: Disk usage : 294.6 MiB (310.6 MiB with WALs) Incremental size : 294.6 MiB (-0.00%) Timeline : 1 Begin WAL : 000000010000000000000012 End WAL : 000000010000000000000012 WAL number : 1 Begin time : 2019-08-21 17:33:31 08:00 End time : 2019-08-21 17:33:35.321271 08:00 Copy time : 3 seconds Estimated throughput : 75.6 MiB/s Begin Offset : 96 End Offset : 0 Begin LSN : 0/12000060 End LSN : 0/13000000 WAL information: No of files : 1 Disk usage : 16.0 MiB WAL rate : 12.58/hour Last available : 000000010000000000000013 Catalog information: Retention Policy : not enforced Previous Backup : 20190821T172217 Next Backup : 20190821T174304

参考：

  1. https://severalnines.com/blog/using-barman-backup-postgresql-overview
  2. http://docs.pgbarman.org/release/2.9/

相关文章