Creating CDC Nodes

The PolarDB-X CDC component is integrated within a PolarDB-X instance. To experience the PolarDB-X CDC capabilities, you need to set up a PolarDB-X cluster.

Global Binlog

• Deploying via PXD: Refer to Deploying Cluster via PXD. You can edit CDC related tag values in the topology file to specify the configuration of the CDC cluster.
  • image: The image of the CDC node
  • replica: The number of CDC nodes
  • nodes: Specific configuration for each CDC node
  • resources: Memory and other resources allocated to CDC nodes
• Deploying via K8S: Refer to Deploying via K8S. By default, a CDC node is created, responsible for generating global Binlog.

Multiple Binlog Streams

Multiple Binlog streams currently only support deployment via K8S. Before deploying, you need to have minikube and PolarDB-X Operator environments ready. For environment setup methods, refer to Preparation.

Next, prepare a YAML file describing the PolarDB-X cluster, named create-cluster.yaml, with the following content:

apiVersion: polardbx.aliyun.com/v1
kind: PolarDBXCluster
metadata:
  name: binlogx-example
spec:
  config:
    cdc:
      envs:
        binlogx_stream_group_name: "group1"
        binlogx_stream_count: "3"
        binlogx_transmit_hash_level: "RECORD"
  topology:
    nodes:
      cdc:
        replicas: 2
        xReplicas: 2
        template:
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: 500m
              memory: 500Mi
          image: polardbx/polardbx-cdc:v2.4.0_5.4.19
      cn:
        replicas: 1
        template:
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: 500m
              memory: 1Gi
          image: polardbx/polardbx-sql:v2.4.0_5.4.19
      dn:
        replicas: 2
        template:
          engine: galaxy
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: 500m
              memory: 500Mi
          image: polardbx/polardbx-engine:v2.4.0_8.4.19
      gms:
        template:
          engine: galaxy
          resources:
            limits:
              cpu: "2"
              memory: 4Gi
            requests:
              cpu: 500m
              memory: 500Mi
          image: polardbx/polardbx-engine:v2.4.0_8.4.19

Note: Currently, PolarDB-X Operator only supports setting up a single multi-stream group, and you also need to set up global Binlog at the same time.

The following are configurations related to multiple streams:

• xReplicas: The number of multi-stream nodes
• binlogx_stream_group_name：The name of the multi-stream group
• binlogx_stream_count：The number of streams
• binlogx_transmit_hash_level：The hash rule for distributing multi-stream data, currently supports three rules:
  • RECORD：Hash by row
  • TABLE：Hash by table
  • DATABASE：Hash by database

Use the following command to create the PolarDB-X Cluster object:

kubectl create -f create-cluster.yaml

You should expect to see the following output:

polardbxcluster.polardbx.aliyun.com/binlogx-example created

If you encounter the following error, you may need to refer to Upgrade to update the CRD。

Error from server (BadRequest): error when creating "create-cluster.yaml": 
PolarDBXCluster in version "v1" cannot be handled as a PolarDBXCluster: 
strict decoding error: unknown field "spec.config.cdc"

Use the following command to observe the status of the PolarDB-X Cluster object:

kubectl get pxc binlogx-example -w

NAME              GMS   CN    DN    CDC   COLUMNAR   PHASE      DISK        AGE
binlogx-example   1/1   0/1   2/2   0/4    -         Creating               71s
binlogx-example   1/1   1/1   2/2   0/4    -         Creating               83s
binlogx-example   1/1   1/1   2/2   2/4    -         Creating               106s
binlogx-example   1/1   1/1   2/2   4/4    -         Running    10.7 GiB    2m13s

Once the PHASE status is Running, the PolarDB-X cluster creation is complete.

After successfully launching the cluster, you can use the following command to obtain the names of all Binlog multi-stream Pods:

kubectl get pods -l polardbx/group=g-1