Tidb CDC log tables are not eligible to replicate

One 、 background

I've created a new one cdc Synchronous tasks , Upstream tidb, Downstream is mysql, However, it is found that some newly created tables cannot be synchronized to the downstream mysql in

Two 、 screening

1. Check the log

#  The following indicates that this table does not meet the synchronization condition 
tables are not eligible to replicate

2.cdc Synchronization limitations

TiCDC Only tables with at least one valid index can be synchronized , The definition of a valid index is as follows ：

• Primary key (PRIMARY KEY) Is a valid index .
• A unique index that meets the following conditions (UNIQUE INDEX) Is a valid index ：
  • Each column in the index is clearly defined as non empty in the table structure (NOT NULL).
  • The virtual build column does not exist in the index (VIRTUAL GENERATED COLUMNS).

3、 ... and 、 terms of settlement

Scheme 1 、 Create satisfaction cdc Table of synchronization conditions

Option two 、 change cdc To synchronize a table without a valid index

1. Stop task

tiup ctl:v4.0.15 cdc changefeed pause --pd=http://pd_ip:pd_port --changefeed-id=simple-replication-task

2. modify cdc Task profile

vim cdc.yaml
#  Add the following configuration 
enable-old-value = true
force-replicate = true

3. Modify the configuration

tiup ctl:v4.0.15 cdc changefeed  update --pd=http://pd_ip:pd_port --config=./cdc.yaml --changefeed-id=simple-replication-task

4. start-up cdc Mission

tiup ctl:v4.0.15 cdc changefeed resume --pd=http://pd_ip:pd_port --changefeed-id=simple-replication-task

5. Be careful
enable-old-value v4.0.5 Start supporting
force-replicate v4.0.8 Start supporting
For tables without valid indexes ,INSERT and REPLACE Such operations are not reentrant , Therefore, there is a risk of data redundancy .TiCDC The data is only guaranteed to be distributed at least once during synchronization ,
Therefore, enable this feature to synchronize tables without valid indexes , It will definitely lead to data redundancy . If data redundancy is not acceptable , It is recommended to add effective indexes , For example, adding AUTO RANDOM The primary key column of the attribute .

Official document ： Document address