Deploy via PXD

PXD is the deployment tool for PolarDB-X. In addition to supporting the one-click rapid setup of a testing environment locally, it also supports deploying PolarDB-X distributed databases in a Linux cluster using a specified topology.

System Configuration

  1. Install Docker on all machines in the cluster, refer to the documentation: https://docs.docker.com/engine/install/

    Note: In cluster mode, the Docker engine version needs to be greater than or equal to 18.04. After installation, verify with the command docker ps. If you encounter the following error, refer to: How to get docker permissions for non-root users

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http:///var/run/docker.sock/v1.26/images/json: dial unix /var/run/docker.sock: connect: permission denied
  1. Configure passwordless login for cluster machines

    If you have already configured SSH passwordless login on your cluster machines, you can skip this step.

# Generate key pair
ssh-keygen -t rsa

# Copy the public key to the target machine, modify user and IP
ssh-copy-id {user}@{ip}

Install PXD on the Deployment Machine

Choose any machine as the deployment machine, and install PXD on this machine. PXD will use the deployment machine to create the PolarDB-X database within the cluster.

Preparations

1.Install Python3

If Python3 is already installed on your machine, you can skip this step.

Check with the command: which python3, if there is a return, Python3 is installed.

Red Hat, CentOS or Fedora:

yum update -y
yum install -y python3

Debian or Ubuntu:

apt-get update
apt-get install python3.7

2.Create a Python3 virtual environment and activate it

python3 -m venv venv
source venv/bin/activate

It is recommended to use a virtual environment to install the PXD tool.

Install PXD

Before installation, it is recommended to upgrade pip with the following command:

pip install --upgrade pip

Install PXD with the following command:

pip install pxd

Note: Some users may experience slow downloads from PyPI. You can use the following command to install from the Alibaba Cloud mirror: pip install -i http://mirrors.aliyun.com/pypi/simple/ pxd

Prepare PolarDB-X Topology File

PolarDB-X currently includes two series: Enterprise Edition and Standard Edition. You can prepare the corresponding topology file according to your actual needs.

Prepare PolarDB-X Enterprise Edition Topology File

PolarDB-X Enterprise Edition is a distributed architecture cluster that supports larger data volumes, targeting business scenarios with enterprise-level ultra-high concurrency, large-scale data complex queries, and accelerated analysis. Write the following YAML file to specify the name of the PolarDB-X Enterprise Edition cluster and the deployment nodes of GMS, CN, and DN.

For example: A total of 3 machines prepared: 10.168.0.37, 10.168.0.38, 10.168.0.39

  1. 10.168.0.37, deploy GMS and CDC nodes.
  2. 10.168.0.38, 10.168.0.39, deploy two CN/DN nodes each. In the dn section, the host_group represents the deployment machines of one dn node with multiple replicas. For example, for a Paxos three-replica cluster, you need to fill in three IPs.
version: v1
type: polardbx
cluster:
  name: pxc_test
  gms:
    image: polardbx/polardbx-engine-2.0:latest
    host_group: [10.168.0.37]
  cn:
    image: polardbx/polardbx-sql:latest
    replica: 2
    nodes:
      - host: 10.168.0.38
      - host: 10.168.0.39
    resources:
      mem_limit: 4G
  dn:
    image: polardbx/polardbx-engine-2.0:latest
    replica: 2
    nodes:
      - host_group: [10.168.0.38]
      - host_group: [10.168.0.39]
    resources:
      mem_limit: 4G
  cdc:
    image: polardbx/polardbx-cdc:latest
    replica: 1
    nodes:
      - host: 10.168.0.37
    resources:
      mem_limit: 4G

The PolarDB-X Enterprise Edition cluster is created using the above topology file. The topology file includes the following properties:

  • version: Topology file version, no need to modify
  • type: polardbx, no need to modify
  • cluster.name: PolarDB-X cluster name
  • cluster.gms.image: GMS Docker image name, optional. The default is the latest image.
  • cluster.gms.host_group: GMS machine IP list. If you want to create a single replica mode, fill in one IP in the list. If you want to create a three-replica cluster based on Paxos, fill in three IPs. The leader node will be randomly selected from the first two IPs.
  • cluster.cn
    • image: Compute node image name, optional. The default is the latest image.
    • replica: Number of compute nodes, needs to correspond to the number of hosts in nodes.
    • nodes: List of IP addresses of compute nodes.
    • resources: Resources used by compute nodes
      • mem_limit: Memory limit, default 2G
  • cluster.dn
    • image: Data node image name, optional. The default is the latest image.
    • replica: Number of data nodes, needs to correspond to the number of host_group in nodes.
    • nodes: List of host_group of storage nodes. One host_group represents the deployment machine of one dn node with multiple replicas. For example, for a Paxos three-replica cluster, fill in three IPs. The leader node will be randomly selected from the first two IPs.
    • resources: Resources used by storage nodes
      • mem_limit: Memory limit, default 2G
  • cluster.cdc
    • image: CDC node image name, optional. The default is the latest image.
    • replica: Number of CDC nodes, needs to correspond to the number of hosts in nodes.
    • nodes: List of IP addresses of CDC nodes.
    • resources: Resources used by CDC nodes
      • mem_limit: Memory limit, default 2G

Note: If creating a three-replica cluster based on Paxos, pxd version must be >=0.3.0

Prepare PolarDB-X Standard Edition Topology File

PolarDB-X Standard Edition adopts a three-node architecture with one master, one standby, and one log, which is cost-effective. It ensures strong consistency of data through multi-replica synchronous replication. It is designed for online business scenarios with ultra-high concurrency, complex queries, and lightweight analysis.

Note: Standard Edition cluster creation depends on pxd version >=0.6.0

PolarDB-X Standard Edition has no CN, GMS, and CDC. It only has one DN node. Write the following YAML file to specify the name of the PolarDB-X Standard Edition and the topology information of the three nodes.

version: v1
type: polardbx
cluster:
  name: pxc_test
  dn:
    image: polardbx/polardbx-engine-2.0:latest
    replica: 1
    nodes:
      - host_group: [172.16.201.11,172.16.201.11,172.16.201.11]
    resources:
      mem_limit: 2G

The PolarDB-X Standard Edition cluster is created using the above topology file. The topology file includes the following properties:

  • version: Topology file version, no need to modify
  • type: polardbx, no need to modify
  • cluster.name: PolarDB-X cluster name
  • cluster.dn
    • image: Data node image name, optional. The default is the latest image.
    • replica: Number of data nodes, set to 1 by default in the Standard Edition.
    • nodes: List of host_group of storage nodes. One host_group represents the deployment machine of one dn node with multiple replicas. For example, for a Paxos three-replica cluster, fill in three IPs. The leader node will be randomly selected from the first two IPs.
    • resources: Resources used by storage nodes
      • mem_limit: Memory limit, default 2G

Create PolarDB-X Cluster

Execute the following command to deploy PolarDB-X with one click within the cluster:

pxd create -file polardbx.yaml

After deployment, pxd will output the connection method of the PolarDB-X cluster. You can log in to the PolarDB-X database for testing using the MySQL command line.

PXD 创建 PolarDB-X 集群

Note:

  • The password for the PolarDB-X administrator account is randomly generated and only appears once. Please remember to save it.
  • PolarDB-X CN itself is stateless. In the Enterprise Edition, multiple CN nodes will be deployed, and any CN can log in to execute SQL. If load balancing is needed, you can provide a unified access address externally through a load balancing component (such as LVS, HAProxy, or F5).

You can connect via MySQL Client and execute the following SQL to experience the distributed features of PolarDB-X Enterprise Edition. For details on PolarDB-X SQL, please refer to: SQL Overview

Note: The syntax of PolarDB-X Standard Edition is exactly the same as MySQL, so experience SQL is not listed here.

# Check GMS
select * from information_schema.schemata;

# Create a partition table
create database polarx_example partition_mode='partitioning';

use polarx_example;

create table example (
  `id` bigint(11) auto_increment NOT NULL,
  `name` varchar(255) DEFAULT NULL,
  `score` bigint(11) DEFAULT NULL,
  primary key (`id`)
) engine=InnoDB default charset=utf8 
partition by hash(id) 
partitions 8;

insert into example values(null,'lily',375),(null,'lisa',400),(null,'ljh',500);

select * from example;

show topology from example;

# Check CDC
show master status ;
show binlog events in 'binlog.000001' from 4;


# Check DN and CN
show storage ;  
show mpp ;

Check PolarDB-X Cluster Status

Execute the following command to check the status of the PolarDB-X cluster:

pxd list

PXD list 结果

Execute the following command to view more commands and usage of pxd:

pxd --help

results matching ""

    No results matching ""