How to use superset to seamlessly connect with MRS for self-service analysis

Abstract ： This article mainly introduces how to do it in MRS Use on top of Superset Data analysis .

This article is shared from Huawei cloud community 《 Use business intelligence software Superset analysis MRS Data best practices 》, author ： Ahhh YeYe .

1. Summary

Superset

Apache Superset It is a modern data exploration and visualization platform . It has powerful functions 、 There are many types of data supported 、 Easy to use 、 Easy to expand 、 Rich visualization ability and many other advantages , stay github There are also 4.6w+ Of star.

MRS

MRS It is a one-stop big data platform provided by Huawei cloud , It basically covers Hadoop Basic components commonly used in ecology , No need for our O & M 、 The trouble of building .

This article mainly introduces how to do it in MRS Use on top of Superset Data analysis .

2. Environmental preparation

Purchase and create elastic ECS in Huawei cloud ECS（ For public images, it is recommended to select CentOS 8.2 64bit）, For installation and operation Superset, And bind the elastic public network IP, It is used to access public network installation dependent packages and Superset service .
MRS Service opening , choice MRS 3.1.0 Version of the normal cluster mode . Pay attention to network and superset Get through

3. superset install

3.1 Log in to purchased Linux Elastic cloud server , Perform the following installation Superset Run dependency package

yum install gcc gcc-c++ libffi-devel openssl-devel cyrus-sasl-devel openldap-devel python36-devel cyrus-sasl-plain

3.2 Execute the following command to upgrade pip edition

pip3 install --upgrade pip

3.3 Perform the following installation python A virtual environment

pip install virtualenv

3.4 Execute the following command to create , And enter python A virtual environment

python3 -m venv venv

. venv/bin/activate

3.5 Perform the following installation python Dependency package

pip install dataclasses

pip install pyhive[hive]

pip install pyhive[presto]

3.6 Perform the following installation Superset

pip install apache-superset

3.7 Execute the following command to initialize database

superset db upgrade

3.8 Execute the following command to create admin user . You need to enter a user name 、FirstName、LastName And email address and password

export FLASK_APP=superset

superset fab create-admin

3.9 Execute the following command to initialize the role and user information

superset init

3.10 Execute the following command to start superset service

superset run -p 8088 -h 0.0.0.0 --with-threads --reload --debugger

3.11 choice “ List of services > Elastic cloud server ECS > The name of the elastic ECs to be operated > Security group ”, single click “ Configuration rules ”. In the configuration rule interface , choice “ The entry direction rule > Add rules ”, Set the protocol port to 8088, The source address is set to access Supereset Of the page's machine IP.

3.12 visit http://ECS elastic IP:8088, And admin The user login , Start using Superset

4. MRS Hive docking

MRS HiveServer adopt ZooKeeper High availability ,Superset Use it directly pyhive Connect HiveServer, Unable to get ZooKeeper Perform service discovery , Therefore, only one connection is configured HiveServer.

4.1 Sign in MRS Administrative console , On the existing cluster page , Click the cluster name to enter MRS Manager page . choice “ Component management > Hive > example ”, see HiveServer Instance node IP

4.2 stay Superset Interface , choice “Data > Databases”, Click... On the right “+” Button to enter create Database page

4.3 stay Add Database Fill in the page Database and SQL Alchemy URI.SQL Alchemy URI The contents to be filled in are “hive://{HiveServer example ip}: port /{hive database name }?auth=NONE”, among ,{HiveServer example ip} by HiveServer Instance business IP,{hive database name } For the... To be connected Hive Database, for example default. Check “Expose in SQL Lab”, single click “Save” Save configuration .

MRS Average cluster hiveserver2 The default port is 10000. View by ： Sign in manager, Click on hive service , Click configure all , Search for hive.server2.thrift.port

superset To configure ：

4.4 choice “SQL Lab > SQL Editor”, Get into Untitled Query page . On the left side “Database” Select the created... From the drop-down menu Database, stay “Select a schema” Select the... To query from the drop-down menu Schema（ namely Hive Of databse, Such as default）, In the middle SQL Enter... In the edit box SQL sentence . And then click “RUN” Button execution SQL, Below Result View the execution results in the tab .

5. MRS SparkJDBC Docking use

5.1 Reference resources hive Access by docking SparkJDBC example ip And port （ Default 22550, Configuration item ：hive.server2.thrift.port）, And then in Superset add to database.

5.2 perform SQL verification

6. Superset Use in MRS PrestoSQL

6.1 stay Manager Interface , choice “ colony > service > Presto > To configure > All configuration ” , Search for configuration items PRESTO_COORDINATOR_FLOAT_IP , obtain Presto Coordinator float IP

port ： Default 7520

6.2 add to Presto database

SQL Alchemy URI Fill in the form of “presto://{Presto Coordinator float IP}:{port}/{catalog name }/{schema name }”, among {Presto Coordinator float IP} by 1 From Presto Coordinator float IP,{catalog name } For the... To be connected Presto catalog,{schema name } by catalog Corresponding schema name , for example hive/default.

6.3 perform Presto SQL verification

Huawei partners and developers conference 2022 The fire is coming , Heavy content can't be missed ！

【 Wonderful activities 】

March forward courageously · Be an all-around Developer →12 Technology live broadcast ,8 High energy output of the great technical treasure , And the code room 、 Many rounds of mysterious tasks such as knowledge competition are waiting for you to challenge . Break through immediately , Open the ultimate prize ！ Click to embark on the promotion of all-round developers ！

【 Technical topics 】

The future has to ,2022 Technical exploration → Huawei's cutting-edge technologies in various fields 、 Heavy open source project 、 Innovative application practice , Standing at the entrance of the intelligent world , Explore how the future shines into reality , Full of dry goods, click to learn

Click to follow , The first time to learn about Huawei's new cloud technology ~

How to use Superset Seamless docking MRS More articles about self-help analysis

5 Channel digital input Di,5 Circuit high current relay output , It can be controlled by computer , Support modbus Protocol industry module , Support and DCS,PLC Seamless docking .
Digital input and output module MRD-5060 have 5 road DI,5 Circuit relay output ,1 road 485 Interface ( Support Modbus RTU), It can be realized 5 road DI( Dry contact input ) turn 485 Collection , It can also be done through 485 control 5 Circuit relay (5A30VDC, ...
How to integrate Android Studio Seamless connection with Huawei software development cloud code warehouse （ Two ）
Last article : How to integrate Android Studio Seamless connection with Huawei software development cloud code warehouse ( One ) The last chapter talked about , How to use Android Studio Based on the software development cloud code warehouse , Create a new project . Next , This chapter continues to talk about building a good project ...
There is a new thing in the code today .js Auto scroll up and down , Seamless docking .
js Auto scroll up and down , Seamless docking . Why do you use this thing ? Because I am doing the modification of the company's official website project . There is a product introduction page , There will be many products in , In the middle of the list . But you can't use paging . So I was thinking if , List number ...
xmake v2.3.1 Release , Seamless docking with other building systems
Recently on xmake A lot of internal refactoring has been done to improve , And a lot of practical new features have been added , Welcome to experience . Project source code Official documents Some new features : One click compilation of other construction system maintenance projects , Achieve seamless docking , And support cross compilation ( such as autotoo ...
iNeuOS Industrial interconnection platform , Machine tool &PLC The hardware gateway is seamlessly connected with the platform , Data interaction
Objective record 1. summary ... 2 2. Platform demonstration ... 2 3. Basic operation of hardware gateway ... 3 3.1 Data collection ... 3 3.2 ...
Kingdee K3 Management software PDA Barcode solutions , Inventory machine and Kingdee K3 Seamless docking
Affirming : The above words are " Wuhan Hanma Technology Co., Ltd " original , Be sure to indicate the source when reprinting . Technology sharing , Communicate with you and me , Common progress !www.hanma-scan.com Original post :http://www.hanma-sc ...
Chinese code counting machine PDA Seamless connection of sixun stocktaking machine sixun barcode data collector batch number commodity stocktaking method
1.1. Inventory Lot No If batch number management is carried out for some commodities , We don't just have to count the total number of items , We also need to distinguish the quantity of a certain batch number of the product , Therefore, the previous batch count workload was very large . Our counting machine PDA Batch number counting is supported ...
Remember a .NET Intelligent robot control system of a manipulator MRS CPU Blast height analysis
One : background 1. Tell a story This is a 6 In mid June, a friend added wx For help dump The story of , His program cpu Blasting height UI stuck , How to solve the problem , The screenshot is as follows : After getting this dump after , I found that this is about a manipulator MRS Program , ha-ha , In the mechanical arm ...
Hua Wei Yun FusionInsight MRS： Help enterprises build “ One enterprise, one lake , One city and one lake ”
Abstract : Hua Wei Yun FusionInsight MRS The next generation of data Lake , Let big data use faster and faster . The easier it is to use . The more used, the more stable . The more you use, the less you need ! Keep the value of data close at hand ! 10 month 30 Japan , With " Hand in hand and win-win · Digital future " For the theme of the first ...
4 Bits or 5 position led Digital display ,485 signal communication modbus, Support any decimal point write , Industry standard settings , Can and plc,dcs, The configuration is perfect , Support custom modification
MRD-5030 have 4 position 8 Segment nixie tube , Support through industrial tagging protocol Modbus(Modbus-RTU) Control display , Support the display of any decimal point . Data is communicated in half duplex mode . Both power port and communication port have anti surge function , lightning protection 600W Protect , can ...

Random recommendation

centos System compilation and installation nginx+php Environment plus independence mysql course
The installation I've seen before nginx+php The environment is full of mysql Database , This is because many webmasters are nginx+php+mysql It's all on the same server , So today we are dealing with it alone , One is nginx+php Environmental Science , then mys ...
linux Process background running and output redirection
The local environment is ubuntu 14.04 With ping www.baidu.com For example : 1. The foreground of the program runs ping www.baidu.com You can see , The screen outputs baidu The result returned 2. Implement the program background ...
extjs In the dynamic to gridpanel Check box assignment
Doing in recently extjs We need to dynamically give gridpanel Check box assignment of I read a lot on the Internet , No more , Finally, find a way to work as follows : RBACformPanel.getSelectionModel().sel ...
ahk Mouse Combo tool
;x = 0 Start clicking ,x = 1 Pause clicking ^::ck_start() ^::ck_end() ck_start() { x = { Click } } ck_end() { x = else x = ...
Animation principle —— painting API
Book name :HTML5-Animation-with-JavaScript Book source :https://github.com/lamberta/html5-animation 1.canvas Of conte ...
data mining ---Matplotib Learning from
What is? matplotlib mat - matrix matrix Two dimensional data - Two dimensional charts plot - drawing lib - libra ...
msvcp140.dll Missing solutions
[ First of all give 99% The easiest way to solve this problem in any situation ]: Download and install Microsoft VC++2015 Version runtime ( English full name :Microsoft Visual C++ 2015 Redistributable Pa ...
XML Of documents DTD To write
<?xml version="1.0" encoding="UTF-8" ?>  <!DOCTYPE ...
Python One of the built-in modules time、random、hashlib、OS、sys、UUID modular
Python Common modules 1.time modular stay Python in , There are usually three ways to express time : Time stamp . Tuples (struct_time). Formatted time string : (1) Time stamp (timestamp) : Generally speaking , Time ...
MATLAB Map toolbox m_map How to install and get started with ( turn ）
reference: http://blog.sina.com.cn/s/blog_8fc890a20102v6pm.html You need to use some mapping tools ,arcgis I'm too lazy to pretend ,GMT(generic m ...

当前位置：网站首页>How to use superset to seamlessly connect with MRS for self-service analysis