当前位置:网站首页>SparkContext: Error initializing SparkContext解决方法
SparkContext: Error initializing SparkContext解决方法
2022-07-02 09:43:00 【Ostrich5yw】
SparkContext: Error initializing SparkContext解决方法
Spark在配置高可用集群中,出现报错ERROR SparkContext: Error initializing SparkContext. java.net.ConnectException: Call From hadoop102/192.168.10.102 to hadoop102:8020 failed on connection exception: java.net.ConnectException: Connection refused;
这是因为我们配置了Spark日志存储在HDFS中,但是在开启Spark集群后,却没有打开Hadoop,造成提交任务时报错。
解决方案:
- 不再存储事件日志
找到Spark安装目录/conf/spark-defaults.conf文件,如图所示,注释掉相应事件日志部分
- 将事件日志存在本地,而非HDFS中
将上图中第二行的目录替换为Linux本地目录 - 启动Hadoop集群(即HDFS服务)
究其出错原因,还是我们配置了spark日志存储于HDFS中,但是缺没有打开HDFS,所以开启Hadoop集群即可
边栏推荐
- Codeforces 771 div2 B (no one FST, refers to himself)
- 堆(優先級隊列)
- 求16以内正整数的阶乘,也就是n的阶层(0=<n<=16)。输入1111退出。
- Lekao: contents of the provisions on the responsibility of units for fire safety in the fire protection law
- K-Means Clustering Visualization in R: Step By Step Guide
- Implementation of address book (file version)
- 子线程获取Request
- How to Create a Beautiful Plots in R with Summary Statistics Labels
- 输入一个三位的数字,输出它的个位数,十位数、百位数。
- ORB-SLAM2不同线程间的数据共享与传递
猜你喜欢

CONDA common command summary

Jenkins用户权限管理

Depth filter of SvO2 series

Dynamic debugging of multi file program x32dbg

Natural language processing series (II) -- building character level language model using RNN

HOW TO EASILY CREATE BARPLOTS WITH ERROR BARS IN R

CDA data analysis -- Introduction and use of aarrr growth model

PyTorch搭建LSTM实现服装分类(FashionMNIST)

Sort---

【工控老马】西门子PLC Siemens PLC TCP协议详解
随机推荐
YYGH-BUG-04
史上最易懂的f-string教程,收藏這一篇就够了
Map和Set
Mish-撼动深度学习ReLU激活函数的新继任者
(C语言)输入一行字符,分别统计出其中英文字母、空格、数字和其它字符的个数。
测试左移和右移
jenkins 凭证管理
Lekao: contents of the provisions on the responsibility of units for fire safety in the fire protection law
mysql表的增删改查(进阶)
PgSQL string is converted to array and associated with other tables, which are displayed in the original order after matching and splicing
[C language] convert decimal numbers to binary numbers
Read the Flink source code and join Alibaba cloud Flink group..
全链路压测
Gaode map test case
Jenkins用户权限管理
【C语言】杨辉三角,自定义三角的行数
Thesis translation: 2022_ PACDNN: A phase-aware composite deep neural network for speech enhancement
Jenkins voucher management
ES集群中节点与分片的区别
HR wonderful dividing line