当前位置：网站首页>Chapter 5 DML data operation

Chapter 5 DML data operation

2022-07-07 00:46:00 【Sun Zhongming】

The first 5 Chapter DML Data manipulation

5.1 Data import —— important

5.1.1 Loading data into a table （Load）—— The key

1． grammar

hive> load data [local] inpath '/opt/module/datas/student.txt' overwrite | into table student [partition (partcol1=val1,…)]

       
        （1）load data: Represents loading data 
        

        
（2）local: Load data from local to hive surface ; Otherwise, from HDFS Load data to hive surface 
        

        
（3）inpath: Indicates the path to load data 
        

        
（4）overwrite: Represents the data already in the overlay table , Otherwise, it means additional 
        

        
（5）into table: Indicates which table to load 
        

        
（6）student: A specific table 
        

        
（7）partition: It means to upload to the specified partition 
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.

2． Practical operation cases

       
        --  Create a table 
        

        
hive (default)> create table student(id string, name string) row format delimited fields terminated by '\t';
        

        
--  Load the local file into hive
        

        
hive (default)> load data local inpath '/opt/module/datas/student.txt' into table default.student;
        

        
--  load HDFS File to hive in 
        

        
--  Upload files to HDFS
        

        
hive (default)> dfs -put /opt/module/datas/student.txt /user/atguigu/hive;
        

        
--  load HDFS Data on 
        

        
hive (default)> load data inpath '/user/atguigu/hive/student.txt' into table default.student;
        

        
--  Load data to cover existing data in the table 
        

        
--  Upload files to HDFS
        

        
hive (default)> dfs -put /opt/module/datas/student.txt /user/atguigu/hive;
        

        
--  Load data to cover existing data in the table 
        

        
hive (default)> load data inpath '/user/atguigu/hive/student.txt' overwrite into table default.student;
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.
        14.
        15.
        16.
        17.
        18.
        19.
        20.
        21.
        22.
        23.
        24.
        25.
        26.
        27.

5.1.2 Insert data into a table through a query statement （Insert）—— The key

       
        --  Create a partition table 
        

        
hive (default)> create table student(id int, name string) partitioned by (month string) row format delimited fields terminated by '\t';
        

        
--  Basic insert data 
        

        
hive (default)> insert into table  student partition(month='201709') values(1,'wangwu');
        

        
--  Basic mode insertion （ According to the single table query results ）
        

        
hive (default)> insert overwrite table student partition(month='201708') select id, name from student where month='201709';
        

        
--  Multiple insertion mode （ Query results based on multiple tables ）
        

        
hive (default)> from student
        

        
       insert overwrite table student partition(month='201707')
        

        
       select id, name where month='201709'
        

        
       insert overwrite table student partition(month='201706')
        

        
       select id, name where month='201709';
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.
        14.
        15.
        16.
        17.
        18.
        19.
        20.
        21.
        22.
        23.

5.1.3 Create table and load data in query statement （As Select）

See 4.5.1 Chapter create table .

Create a table based on the query results （ The results of the query are added to the newly created table ）

       
        create table if not exists student3
        

        
as select id, name from student;
       
       
        1.
        2.
        3.

5.1.4 Passed while creating the table Location Specify the load data path —— The key

       
        --  Create table , And designated at hdfs Position on 
        

        
hive (default)> create table 
        

        
     if not exists student5(id int, name string)
        

        
       row format delimited fields terminated by '\t'
        

        
       location '/user/hive/warehouse/student5';
        

        
-- Upload data to hdfs On 
        

        
hive (default)> dfs -put /opt/module/datas/student.txt
        

        
/user/hive/warehouse/student5;
        

        
--  Query data 
        

        
hive (default)> select * from student5;
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.
        14.
        15.
        16.
        17.
        18.
        19.

5.1.5 Import Data to designated Hive In the table —— Not much

Be careful ： First use export After export , Then import the data into .

       
        hive (default)> import table student2 partition(month='201709') from '/user/hive/warehouse/export/student';
       
       
        1.

5.2 Export data

5.2.1 Insert export

       
        --  Export the results of the query to local 
        

        
hive (default)> insert overwrite local directory '/opt/module/datas/export/student'
        

        
 select * from student;
        

        
--  Format and export the query results to local 
        

        
hive(default)>insert overwrite local directory '/opt/module/datas/export/student1'
        
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';
        

        
 select * from student;
        

        
--  Export the results of the query to HDFS On ( No, local)
        

        
hive (default)> insert overwrite directory '/user/root/student2'
        

        
      ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' 
        

        
      select * from student;
       
       
        1.
        2.
        3.
        4.
        5.
        6.
        7.
        8.
        9.
        10.
        11.
        12.
        13.
        14.
        15.
        16.
        17.
        18.
        19.
        20.

5.2.2 Hadoop Command export to local

       
        hive (default)> dfs -get /user/hive/warehouse/student/month=201709/000000_0
        
/opt/module/datas/export/student3.txt;
       
       
        1.
        2.

5.2.3 Hive Shell Command export

Basic grammar ：（hive -f/-e Execute statements or scripts > file）

       
        [[email protected] hive]$ bin/hive -e 'select * from default.student;' > /opt/module/datas/export/student4.txt;
       
       
        1.

原网站

版权声明
本文为[Sun Zhongming]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/02/202202130947247368.html