MySQL realizes data comparison between two tables by calculating intersection and difference sets

Two tables bank_bill and biz_bill The structure definition of should be consistent , If not completely consistent , Need to be in sql Statement to redefine the alias .

0. Environmental preparation

Bank statement ：

CREATE TABLE `bank_bill` (
  `id` varchar(32) NOT NULL COMMENT ' Serial number ',
  `merchant_no` varchar(32) DEFAULT NULL COMMENT ' Merchant number ',
  `pay_channel` varchar(32) DEFAULT NULL COMMENT ' Payment channel ',
  `trans_type` varchar(32) DEFAULT NULL COMMENT ' Type of transaction ',
  `merchant_order_no` varchar(64) DEFAULT NULL COMMENT ' The order no. ',
  `trans_datetime` varchar(32) DEFAULT NULL COMMENT ' Trading hours ',
  `trans_amount` double(15,2) DEFAULT NULL COMMENT ' Transaction amount ',
  `merchant_account` varchar(32) DEFAULT NULL COMMENT ' Merchant account ',
  `account_change` double(15,2) DEFAULT NULL COMMENT ' Current account amount ',
  `customer_account` varchar(32) DEFAULT NULL COMMENT ' Customer account ',
  `account_type` varchar(32) DEFAULT NULL COMMENT ' Account type ',
  `bank_fee` double(15,2) DEFAULT NULL COMMENT ' Commission Commission ',
  `divide_fee` double(15,2) DEFAULT NULL COMMENT ' Installment service charge ',
  `account_date` varchar(20) DEFAULT NULL COMMENT ' Accounting date ',
  `flow_host` varchar(20) DEFAULT NULL COMMENT ' The serial number of the host ',
  `flow_9014` varchar(32) DEFAULT NULL COMMENT ' Serial number ',
  `ref_order` varchar(64) DEFAULT NULL COMMENT ' Original order No ',
  PRIMARY KEY (`id`),
  UNIQUE KEY `pkindex` (`merchant_no`,`merchant_order_no`) USING BTREE,
  KEY `bill_no_index` (`merchant_order_no`) USING BTREE,
  KEY `trans_datetime` (`trans_datetime`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=' Bank statement ';

Business data ： 

INSERT INTO `bank_bill` VALUES ('21ca1b16d1c44a1b80cd9ca4b463da2d', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156235202105262020412243', '2020-06-10 09:57:33', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095416928604', '');
INSERT INTO `bank_bill` VALUES ('4256f61dc53e485da6bfd3dc6932fb1d', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156397202105252100295721', '2020-06-10 10:06:33', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01100448335484', '');
INSERT INTO `bank_bill` VALUES ('83f446fe484f4e2a8dbb5cbab3f8195e', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156235202105262124027097', '2020-06-10 09:55:55', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095244873600', '');
INSERT INTO `bank_bill` VALUES ('8a15423309974f35b106c09f9ca023b7', '103882280002012', 'wxalipay', 'weixinpay', '102002060000131626202105261638336030', '2020-06-10 09:59:05', '0.01', '22810101941000825', '0.01', '', 'BOCD_DEBIT', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095823067133', '');

Business statement

CREATE TABLE `biz_bill` (
  `id` varchar(32) NOT NULL COMMENT ' Serial number ',
  `merchant_no` varchar(32) DEFAULT NULL COMMENT ' Merchant number ',
  `pay_channel` varchar(32) DEFAULT NULL COMMENT ' Payment channel ',
  `trans_type` varchar(32) DEFAULT NULL COMMENT ' Type of transaction ',
  `merchant_order_no` varchar(64) DEFAULT NULL COMMENT ' The order no. ',
  `trans_datetime` varchar(32) DEFAULT NULL COMMENT ' Trading hours ',
  `trans_amount` double(15,2) DEFAULT NULL COMMENT ' Transaction amount ',
  `merchant_account` varchar(32) DEFAULT NULL COMMENT ' Merchant account ',
  `account_change` double(15,2) DEFAULT NULL COMMENT ' Current account amount ',
  `customer_account` varchar(32) DEFAULT NULL COMMENT ' Customer account ',
  `account_type` varchar(32) DEFAULT NULL COMMENT ' Account type ',
  `bank_fee` double(15,2) DEFAULT NULL COMMENT ' Commission Commission ',
  `divide_fee` double(15,2) DEFAULT NULL COMMENT ' Installment service charge ',
  `account_date` varchar(20) DEFAULT NULL COMMENT ' Accounting date ',
  `flow_host` varchar(20) DEFAULT NULL COMMENT ' The serial number of the host ',
  `flow_9014` varchar(32) DEFAULT NULL COMMENT ' Serial number ',
  `ref_order` varchar(64) DEFAULT NULL COMMENT ' Original order No ',
  PRIMARY KEY (`id`),
  UNIQUE KEY `pkindex` (`merchant_no`,`merchant_order_no`) USING BTREE,
  KEY `bill_no_index` (`merchant_order_no`) USING BTREE,
  KEY `trans_datetime` (`trans_datetime`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=' Business statement ';

  Test data ：

INSERT INTO `biz_bill` VALUES ('21ca1b16d1c44a1b80cd9ca4b463da2d', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156235202105262020412243', '2020-06-10 09:57:33', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095416928604', '');
INSERT INTO `biz_bill` VALUES ('4256f61dc53e485da6bfd3dc6932fb1d', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156397202105252100295721', '2020-06-10 10:06:33', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01100448335484', '');
INSERT INTO `biz_bill` VALUES ('83f446fe484f4e2a8dbb5cbab3f8195e', '103882280002012', 'wxalipay', 'weixinpay', '102002060000156235202105262124027097', '2020-06-10 09:55:55', '0.01', '22810101941000825', '0.01', '', 'OTHERS', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095244873600', '');
INSERT INTO `biz_bill` VALUES ('8a15423309974f35b106c09f9ca023b7', '103882280002012', 'wxalipay', 'weixinpay', '102002060000131626202105261638336030', '2020-06-10 09:59:05', '0.01', '22810101941000825', '0.01', '', 'BOCD_DEBIT', '0.00', '0.00', '2020-06-11', '285702419', '6AECEP01095823067133', '');

1. Calculation bank_bill and biz_bill The difference between the set

bank_bill and biz_bill The difference set of is in bank_bill Presence in table , But in biz_bill Data that does not exist in the table .

mysql> select a.merchant_order_no, b.merchant_order_no from bank_bill a left join biz_bill b on a.merchant_order_no = b.merchant_order_no where b.merchant_order_no is null;
Empty set

#  modify biz_bill Two records in the table merchant_order_no field value , Add... At the end 1
mysql> select a.merchant_order_no, b.merchant_order_no from bank_bill a left join biz_bill b on a.merchant_order_no = b.merchant_order_no where b.merchant_order_no is null;
+--------------------------------------+-------------------+
| merchant_order_no                    | merchant_order_no |
+--------------------------------------+-------------------+
| 102002060000131626202105261638336030 | NULL              |
| 102002060000156235202105262124027097 | NULL              |
+--------------------------------------+-------------------+

For the differences found , According to business needs , The data with deviation can be corrected automatically or manually , When necessary, , Multiple comparisons can also be made . 

2. Calculation biz_bill and bank_bill The difference between the set

biz_bill and bank_bill The difference set of is in biz_bill Presence in table , But in bank_bill Data that does not exist in the table .

mysql> select a.merchant_order_no, b.merchant_order_no from biz_bill a left join bank_bill b on a.merchant_order_no = b.merchant_order_no where b.merchant_order_no is null;
+---------------------------------------+-------------------+
| merchant_order_no                     | merchant_order_no |
+---------------------------------------+-------------------+
| 1020020600001316262021052616383360301 | NULL              |
| 1020020600001562352021052621240270971 | NULL              |
+---------------------------------------+-------------------+
2 rows in set

3. Calculation bank_bill and biz_bill Intersection

When bank_bill and biz_bill Difference set and biz_bill and bank_bill When all the difference sets of are empty , Just explain bank_bill and biz_bill The data is consistent . here a and b as well as a and b The intersection of is the same data .

# When the data of two tables are consistent 
mysql> select a.merchant_order_no, b.merchant_order_no from biz_bill a left join bank_bill b on a.merchant_order_no = b.merchant_order_no where b.merchant_order_no = a.merchant_order_no;
+--------------------------------------+--------------------------------------+
| merchant_order_no                    | merchant_order_no                    |
+--------------------------------------+--------------------------------------+
| 102002060000131626202105261638336030 | 102002060000131626202105261638336030 |
| 102002060000156235202105262020412243 | 102002060000156235202105262020412243 |
| 102002060000156235202105262124027097 | 102002060000156235202105262124027097 |
| 102002060000156397202105252100295721 | 102002060000156397202105252100295721 |
+--------------------------------------+--------------------------------------+

# When two table data parts match 
mysql> select a.merchant_order_no, b.merchant_order_no from biz_bill a left join bank_bill b on a.merchant_order_no = b.merchant_order_no where b.merchant_order_no = a.merchant_order_no;
+--------------------------------------+--------------------------------------+
| merchant_order_no                    | merchant_order_no                    |
+--------------------------------------+--------------------------------------+
| 102002060000156235202105262020412243 | 102002060000156235202105262020412243 |
| 102002060000156397202105252100295721 | 102002060000156397202105252100295721 |
+--------------------------------------+--------------------------------------+

4. Determine the source of statistical data according to business needs

After the data is successfully compared according to the keyword and status information , It can be stored according to the relevant business data location , Read the relevant data information from the corresponding tables , Make statistical summary .

5. Merge two tables into one large table

Merge the data of two tables into a large table , Set a globally unique primary key , In principle, the primary key in the benchmark table is used as the primary key , Fields from both tables are added to one table , Use specific naming methods to distinguish . When the data is added , Can pass where Conditions query a single table with various conditions , To achieve the purpose of taking intersection and union .