TPC-C Test Report

Background Information

TPC-C is a widely recognized benchmark suite established by the TPC Council for evaluating the performance of database Online Transaction Processing (oriented towards OLTP capabilities). It involves 10 tables and simulates five types of business transactions: NewOrder (creation of new orders), Payment (payment processing for orders), OrderStatus (inquiry for recent orders), Delivery (order delivery), and StockLevel (analysis of stock shortage status). TPC-C measures the maximum qualified throughput (MQTh) of a system in terms of tpmC (Transactions per Minute), where Transactions are based on the NewOrder Transaction, and the final unit of measure is the number of new orders processed per minute.

This article uses Benchmark-Boot to verify the performance of PolarDB-X.

Note The TPC-C implementation in this article is based on the TPC-C benchmark test but is not comparable with published TPC-C benchmark test results, as the tests in this article do not meet all the requirements of the TPC-C benchmark test.

Benchmark Boot is a one-stop stress testing platform developed for PolarDB-X, currently supporting benchmarks including Sysbench, TPC-C, and TPC-H, and enhances testing efficiency through a graphical interface.

Test Design

Data Volume for the Test

Based on 1000 Warehouses, the main table data volumes are as follows:

• bmsql_order_line 300 million rows
• bmsql_stock 100 million rows
• bmsql_customer、bmsql_history、bmsql_oorder 30 million rows each

Host Resources

PolarDB-X standard edition centralized deployment only requires deploying the storage node DN, no separate compute node CN is needed.

Stress Machine

Test Method

Step 1: Prepare the Stress Machine

Deploy Benchmark-Boot, refer to the documentation: Stress Testing with Benchmark Boot

# Download the image
docker pull polardbx/benchmark-boot:latest

# Start the container
docker run -itd --name 'benchmark-boot' --privileged --net=host \
    -v /etc/localtime:/etc/localtime polardbx/benchmark-boot:latest \
    /usr/sbin/init

# Verify
curl http://127.0.0.1:4121/

For offline deployment, you can download the benchmark-boot image to the OPS machine and upload it to a private repository.

Step 2: Prepare the PolarDB-X Instance for Stress Testing

Create a PolarDB-X instance, for detailed steps please refer to: Deploy by PXD

Instance creation parameters:

version: v1
type: polardbx
cluster:
  name: pxc_test_standard
  dn:
    image: registry:5000/polardbx-engine-2.0:latest
    replica: 1
    nodes:
      - host_group: [10.0.3.244, 10.0.3.245, 10.0.3.246]
    resources:
      mem_limit: 80G

Verify the instance is connectable, open Benchmark-Boot, select 'Run Stress Test' -> 'Database Connection' on the left, fill in the database connection information into Benchmark-Boot, and click "Submit".

Step 3: Adjust Instance Parameters

The PolarDB-X instance created by PXD by default uses an internal my.cnf parameter template with the following parameters set to one:

sync_binlog = 1
innodb_flush_log_at_trx_commit = 1

Step 4: Data Preparation for Stress Testing

Create Database

Open Benchmark-Boot, select 'Run Stress Test' -> 'TPC-C' -> 'Switch Database' on the left, enter the database name tpcc, choose MYSQL mode, and submit.

Import Data

After the database is created, scroll to the bottom of the page and import TPC-C data with a warehouse count of 1000 and a concurrency setting of 500, then wait for the data preparation to complete.

Step 5: Execute Stress Testing

From the left menu, select 'Run Stress Test' -> 'TPC-C' -> 'Run TPC-C', fill in the stress test parameters and start the test.

Test Results

12:11:01,386 [Thread-165] INFO   jTPCC : Term-00, 
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, 
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, Measured tpmC (NewOrders) = 249902.01
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, Measured tpmTOTAL = 554958.7
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, Session Start     = 2023-10-30 12:06:01
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, Session End       = 2023-10-30 12:11:01
12:11:01,387 [Thread-165] INFO   jTPCC : Term-00, Transaction Count = 2775292