Clickhouse learning (IX) Clickhouse integrating MySQL

---

Integrate mysql database

!!!clickhouse Improper support mysql, It also supports other databases , If necessary, you can check it on the official website

MaterializeMySQL engine

MaterializeMySQL The engine can map mysql One of the database, And it will automatically be ClickHouse Create the corresponding ReplacingMergeTree, It can directly monitor binlog Events are imported directly clickhouse in , No longer use third-party middleware .

• Creating database After that, it will be fully synchronized mysql Tables and data in , After that, it will pass binlog Incremental synchronization
• Will be created for each ReplacingMergeTree Automatic increase _sign( Whether the tag is deleted , The value is 1 and -1) and _version( Listen for adding, deleting and modifying events , If it happens, it will increase ) Field

MySQL Turn on binlog and GTID Pattern

!!! Note the version of the database , It's better to use mysql8.0 Version of , Otherwise, it is likely to report code 537 error

Get into /etc/my.cnf Add information to

Turn on binlog

Turn on GTID Pattern

Ensure the consistency of data synchronization

To configure mysql Of default_authentication_plugin, in order to clickhouse You can create tables

default_authentication_plugin = mysql_native_password

Restart after modification mysql service

[[email protected] etc]# systemctl restart mysqld

test

Get ready MySQL Tables and data

clickhouse Open materialized view

set allow_experimental_database_materialize_mysql=1;

ClickHouse Created in MaterializeMySQL database

CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster]
ENGINE = MaterializedMySQL(‘host:port’, [‘database’ | database], ‘user’, ‘password’) [SETTINGS …]

CREATE DATABASE test_binlog ENGINE =　MaterializeMySQL('192.168.124.26:3306','mybatis','root','123456789');

Check the mapping

Insert data test

stay mysql Insert a piece of data in

Delete the test

Delete successful

You can't see from the table _sign and _version These two fields , Because these two fields are hidden , If you want to check, you have to add this condition yourself

You can see that it just marks the inserted and deleted data , It's not really about data deletion or update

Integrate MySQL surface

It is similar to the operation of integrating databases, but the commands are different

CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
(
    name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1] [TTL expr1],
    name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2] [TTL expr2],
    ...
) ENGINE = MySQL('host:port', 'database', 'table', 'user', 'password'[, replace_query, 'on_duplicate_clause'])
SETTINGS
    [connection_pool_size=16, ]
    [connection_max_tries=3, ]
    [connection_auto_close=true ]
;