Flink SQL knows why (16): dlink, a powerful tool for developing enterprises with Flink SQL

1. Preface

Bloggers in this series are talking about flink sql Relevant practical .

Come to this chapter , In fact, there are many commonly used flink sql Grammar and practical cases have been talked about .

So the principle is , You have to put... In your own company flink sql These are good things to use .

Students engaged in big data development basically know that HUE It says hive sql Thief Shuang . So did you write flink sql Enterprise class web IDE Recommended ？

After blogger research , I found two excellent sharp weapons ：

• Apache Zeppelin

• Dlink

Why introduce Dlink Well ？

Because bloggers are familiar with their developers , So take it first Dlink To try .

I don't say much nonsense , Everyone wants to see the effect first and then how to deploy . Let's take a look at the final results .

2.Dlink Platform effect

2.1. Sign in

Sign in

The login account and password are... By default admin/admin.

2.2.flink sql Development interface

The specific functions are shown in the figure below ：

sql Development

Be careful ：

Pre run is used for quick verification sql Is it right , You can quickly standalone The pattern comes out with a result . Fat intestines are easy to use . Want to see the results of the pre run , stay sql You can't write insert into xxx This paragraph .

It can be seen that it is a fully functional web IDE.

Next, let's see how to install and deploy Dlink 0.4.0.

3. Installation and deployment

3.1.Dlink Of github

The source code goes directly to github Look up , It's open source , Links are as follows .

https://github.com/DataLinkDC/dlink

3.2. Deployment environment preparation

This deployment example is in Mac OS on , Other environments are not tested .

Environmental Science	edition	remarks
jdk	1.8.0_201	web Based on the environment
mysql	8.0+	Storage web IDE Homework , Cluster and other information
nginx	Bloggers use 1.21.1	web Front end access

3.3. Download decompression Dlink Installation package

Bloggers are based on Dlink 0.4.0 Version deployment installed .

First step ： download Dlink 0.4.0 Installation package .

We turn on Dlink 0.4.0 release Notes have a look , Links are as follows ：

https://github.com/DataLinkDC/dlink/releases/tag/0.4.0

Open the above 0.4.0 release Notes After link , Click dlink-release-0.4.0.tar.gz download Dlink 0.4.0 Installation package .

Dlink download

The second step ： decompression Dlink 0.4.0 Install the package .

Dlink

After decompression dlink-release-0.4.0 The directory structure is as follows ：

Notice that this is just  Dlink Related packages and files ,flink The execution of the task will inevitably depend on  flink  Related to the package , This requires you to dlink-release-0.4.0 Create your own directory plugins Folder . And will flink dependent jar Put it in  plugins Under the folder . From https://archive.apache.org/dist/flink/flink-1.13.2/ Download the corresponding flink tar.gz When you get to the local area , After decompressing flink jar All copy To   plugins Under folder .

config/ --  The configuration file 
|- application.yml
lib/ --  External dependencies and Connector
|- dlink-client-1.13-0.4.0.jar
|- dlink-connector-jdbc-1.13-0.4.0.jar
|- dlink-function-0.4.0.jar
|- dlink-metadata-clickhouse-0.4.0.jar
|- dlink-metadata-mysql-0.4.0.jar
|- dlink-metadata-oracle-0.4.0.jar
|- dlink-metadata-postgresql-0.4.0.jar
sql/
|- dlink.sql -- Mysql Initialization script 
auto.sh --  Start stop script 
dlink-admin-0.4.0.jar --  Package 

3.4. To configure Dlink MySQL

Since it's a web IDE, It's bound to store some web Apply relevant information . This information is stored in MySQL Medium .

First step ： establish MySQL dlink library .

Use MySQL-cli Connect MySQL Create a library .

> mysql -u user name  -p password 

mysql> create database dlink;
mysql> show databases;

Dlink MySQL

The second step ： Use Dlink Of sql Script （dlink Catalog \sql Catalog \dlink.sql） initialization Dlink Database table information , The specific initialization command line is as follows .

> mysql -h localhost -u user name  -p password  -Ddlink < dlink.sql
> mysql -u user name  -p password 
mysql> use dlink;
mysql> show tables;

Dlink MySQL Table

The third step ：Dlink It's also a web project , It's using SpringBoot That set of things , So connect MySQL It needs to be done Spring Related configuration .

So we need to go dlink Catalog \config Catalog \application.yml Revision in China MySQL Related configuration . Because bloggers are deployed locally . Therefore, only the user name and password are involved in the change , Save after changing .

application.yml

3.5. start-up web Back end

Dlink It's separate at the front and back , We just configured MySQL Relevant environment , You can start the backend directly , The port occupied by the back end is 8888, The start command is as follows .

sh auto.sh start

stay Mac OS The following problems may be encountered during the next startup ：

bash error

The reason for this error is the following link ：

https://jingyan.baidu.com/article/9f63fb91d014b8c8410f0e7a.html

The solution is as follows ：

Put... Directly auto.sh Copy a new one in another editor auto1.sh Just start it up .

Other commands ：

#  stop it 
sh auto.sh stop
#  restart 
sh auto.sh restart
#  state 
sh auto.sh status

Run log ：

Console output ： In the root directory of the project dlink.log file .

Log archive output ： In the root directory of the project logs Under the table of contents .

3.6. To configure web front end

The front end is all static files ,Dlink Use Nginx As a server for accessing front-end static files .

First step ：Nginx stay Mac OS Installation , The following links ：

https://www.jianshu.com/p/4f433d219ab7

The second step ： stay nginx.conf Configuration in file Dlink Of server Information .

If installed locally , Just put the following paragraph copy To nginx.conf Corresponding server Just configure it .

server {
    listen       9999;
    server_name localhost;
    
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain application/javascript application/x-javascript text/css application/xml text/javascript application/x-httpd-php image/jpeg image/gif image/png;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";
    
    #charset koi8-r;
    
    #access_log logs/host.access.log main;
    
    location / {
        root   html;
        index index.html index.htm;
        try_files $uri $uri/ /index.html;
    }
    
    #error_page 404             /404.html;
    
    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504 /50x.html;
    location = /50x.html {
        root   html;
    }
    
    location ^~ /api {
        proxy_pass http://127.0.0.1:8888;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_set_header   X-Real-IP         $remote_addr;
    }
}

The third step ： Be careful , a key . Be sure to will 【dlink Catalog \html Catalog \ All the files 】 Uploaded to the Nginx Of [html Catalog ] Next . Or visit http://127.0.0.1:9999 I can't see anything .

Step four ：reload Nginx Configuration information .

Step five ： Then we can happy Use Dlink 了 .

4. Summary and prospect

Other features are still being tested by bloggers .

quote Dlink Introduction to the official website ,Dlink Will follow closely Flink Official community development , To promote and develop Flink Strive for the application of , make FlinkSQL The image of your best partner .

5.Dlink Community communication

Welcome to join the community to exchange, share and criticize , You are also welcome to contribute to the community .

QQ Community group ：543709668, Application remarks Dlink, No writing, no approval .

Wechat community group （ recommend ）： add to wenmo_ai , Application remarks Dlink, Invite in .

official account ：DataLink Data center

DataLink Data center