Incremental Log Backup

As of version 1.4.0, the PolarDB-X Operator has expanded its capabilities to include incremental log backups. This guide will walk you through the steps to set up incremental log backups for PolarDB-X.

The term "incremental logs" refers to the consistency logs generated on DN nodes, which are akin to MySQL's binlog logs and are typically located in the /data/mysql/log directory inside the DN container.

Prerequisites

1. Update the PolarDB-X Operator to version 1.4.0 or higher.
2. Complete the backup storage configuration, see the document: Backup Storage Configuration.

Initiating Incremental Log Backup

The following describes how to use the PolarDBXBackupBinlog object to perform incremental log backups for PolarDB-X.

Creating a PolarDBXBackupBinlog Object

1. Write a pxc-backup-binlog.yaml file referring to the example below:

apiVersion: polardbx.aliyun.com/v1                 # API group/version
kind: PolarDBXBackupBinlog                         # API kind
metadata:
  name: backupbinlogforpolardb-x                    #Name of the incremental log backup job
spec:
  pxcName: polardb-x                                # Name of the target PolarDB-X cluster to back up
  pxcUid: 8f634de1-5a4e-4e1c-b2dc-e8763384d83a      # UID of the target PolarDB-X cluster to back up
  remoteExpireLogHours: 168                         # Duration (in hours) to retain logs in remote storage (OSS/SFTP)
  localExpireLogHours: 7                            # Duration (in hours) to retain logs locally on the data nodes
  maxLocalBinlogCount: 60                           # Max number of local incremental log files to retain on data nodes
  pointInTimeRecover: true                          # Flag to enable support for point-in-time recovery
  binlogChecksum: CRC32                             # Checksum used for incremental logs
  storageProvider:                                   
    storageName: oss                                # Storage method used, supports sftp and oss
    sink: osssink                                   # Name of the storage configuration item

Parameter Explanation:

• pxcName: The name of the target PolarDB-X cluster to be backed up, required field.
• pxcUid: The UID of the target PolarDB-X cluster to be backed up, optional field, usually not filled.
• remoteExpireLogHours: The number of hours to keep logs at the remote end (OSS or SFTP), optional field, default is 168.
• localExpireLogHours: The number of hours to keep logs locally on the data node, optional field, default is 7.
• maxLocalBinlogCount： The maximum number of incremental log files to keep locally on the data node, optional field, default is 60.
• pointInTimeRecover： Whether to support point-in-time recovery, optional field, default is true.
• binlogChecksum： Checksum for incremental logs, optional field, default is CRC32.
• storageProvider.storageName: Backup set storage method, supports sftp and oss, required field.
• storageProvider.sink: The name of the backup set storage configuration, corresponding to the name field in Backup Storage Configuration, required field.

2.Create the PolarDBXBackupBinlog object with the following command to start incremental log backup:

kubectl create -f pxc-backup-binlog.yaml

3.Check if the incremental log backup is in the running phase:

kubectl get pxcblog

Accessing Incremental Log Backups

Incremental log backup files are stored in the following path, you can check the corresponding files in the SFTP configured host or OSS bucket.

Incremental log metadata file

{root_path}/polardbx-binlogbackup/{namespace}/{pxc_name}/{pxc_uid}/{xstore_name}/{xstore_uid}/{pod_name}/{version}/{batch_name}/binlog-meta/mysql_bin.{number}.txt

Incremental log file

{root_path}/polardbx-binlogbackup/{namespace}/{pxc_name}/{pxc_uid}/{xstore_name}/{xstore_uid}/{pod_name}/{version}/{batch_name}/binlog-file/mysql_bin.{number}

• root_path depends on the storage configuration
  • If sftp is used as storage, this value is sink.rootPath
  • If oss is used as storage, this value is sink.bucket
• polardbx-binlogbackup is a fixed field
• namespace is the namespace of the target PolarDB-X Cluster
• pxc_name is the name of the target PolarDB-X Cluster
• pxc_uid is the UID of the target PolarDB-X Cluster
• xstore_name is the name of the xstore associated with the backup file.
• xstore_uid is the UID of the xstore associated with the backup file.
• pod_name is the name of the pod associated with the backup file.
• version is the version number of the pod associated with the backup file.
• batch_name is the name of the batch directory; each batch contains 1,000 files.
• binlog-file and binlog-meta are fixed directories.
• number is the sequence number of the incremental log file.