【sqoop】sqoop1.4.7 安装集成CDH5.13

前言

使用apache sqoop 集成CDH5.13.

一、下载

下载地址: http://archive.apache.org/dist/sqoop/

这里使用的连接：

http://archive.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

二、安装

1. 解压

tar -xzvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /home/rong/

2. 重命名

mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop-1.4.7

3. 软连接

ln -s sqoop-1.4.7 sqoop

4. 配置环境变量

vim /etc/profile

添加如下内容：

export SQOOP_HOME=/home/rong/sqoop-1.4.7
export PATH=$SQOOP_HOME/bin:$PATH

5. 配置conf

a. 修改文件名

cp ${SQOOP_HOME}/conf/sqoop-env-template.sh sqoop-env.sh

b. 修改文件内容

vi sqoop-env.sh

设置如下内容：

#Set path to where bin/hadoop is available
#
export HADOOP_COMMON_HOME=/opt/cloudera/parcels/CDH/lib/hadoop

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/opt/cloudera/parcels/CDH/lib/hadoop-yarn

#set the path to where bin/hbase is available
export HBASE_HOME=/opt/cloudera/parcels/CDH/lib/hbase

#Set the path to where bin/hive is available
export HIVE_HOME=/opt/cloudera/parcels/CDH/lib/hive
export HIVE_CONF_DIR=/etc/hive/conf

#Set the path for where zookeper config dir is
export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH/lib/zookeeper
export ZOOCFGDIR=/etc/zookeeper/conf/  
                                           

如果是用的CDH集群，那么这个基本是不需要做变更的

6. 拷贝jar包

cp /opt/cloudera/parcels/CDH/lib/hive/lib/hive-exec.jar /home/rong/sqoop/lib/
cp /opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar /home/rong/sqoop/lib/
cp /opt/cloudera/parcels/CDH/lib/hadoop/hadoop-common.jar /home/rong/sqoop/lib/

验证

${SQOOP_HOME}/bin/sqoop-version