当前位置:网站首页>SparkContext: Error initializing SparkContext解决方法
SparkContext: Error initializing SparkContext解决方法
2022-07-02 09:43:00 【Ostrich5yw】
SparkContext: Error initializing SparkContext解决方法
Spark在配置高可用集群中,出现报错ERROR SparkContext: Error initializing SparkContext. java.net.ConnectException: Call From hadoop102/192.168.10.102 to hadoop102:8020 failed on connection exception: java.net.ConnectException: Connection refused;
这是因为我们配置了Spark日志存储在HDFS中,但是在开启Spark集群后,却没有打开Hadoop,造成提交任务时报错。
解决方案:
- 不再存储事件日志
找到Spark安装目录/conf/spark-defaults.conf文件,如图所示,注释掉相应事件日志部分 - 将事件日志存在本地,而非HDFS中
将上图中第二行的目录替换为Linux本地目录 - 启动Hadoop集群(即HDFS服务)
究其出错原因,还是我们配置了spark日志存储于HDFS中,但是缺没有打开HDFS,所以开启Hadoop集群即可
边栏推荐
- Filtre de profondeur de la série svo2
- HOW TO CREATE A BEAUTIFUL INTERACTIVE HEATMAP IN R
- 甜心教主:王心凌
- 子线程获取Request
- Take you ten days to easily finish the finale of go micro services (distributed transactions)
- mysql索引和事务
- How to Easily Create Barplots with Error Bars in R
- 堆(优先级队列)
- File operation (detailed!)
- 基于Arduino和ESP8266的Blink代码运行成功(包含错误分析)
猜你喜欢
Flesh-dect (media 2021) -- a viewpoint of material decomposition
MSI announced that its motherboard products will cancel all paper accessories
Test shift left and right
PyTorch nn. Full analysis of RNN parameters
Filtre de profondeur de la série svo2
(C language) input a line of characters and count the number of English letters, spaces, numbers and other characters.
测试左移和右移
(C语言)输入一行字符,分别统计出其中英文字母、空格、数字和其它字符的个数。
WSL 2 will not be installed yet? It's enough to read this article
5g era, learning audio and video development, a super hot audio and video advanced development and learning classic
随机推荐
【2022 ACTF-wp】
倍增 LCA(最近公共祖先)
Leetcode122 买卖股票的最佳时机 II
(C语言)3个小代码:1+2+3+···+100=?和判断一个年份是闰年还是平年?和计算圆的周长和面积?
BEAUTIFUL GGPLOT VENN DIAGRAM WITH R
Jenkins user rights management
PgSQL string is converted to array and associated with other tables, which are displayed in the original order after matching and splicing
【C语言】十进制数转换成二进制数
JZ63 股票的最大利润
Depth filter of SvO2 series
Lombok common annotations
Post request body content cannot be retrieved repeatedly
初始JDBC 编程
drools决策表的简单使用
YYGH-9-预约下单
PyTorch中repeat、tile与repeat_interleave的区别
H5,为页面添加遮罩层,实现类似于点击右上角在浏览器中打开
甜心教主:王心凌
CDH存在隐患 : 该角色的进程使用的交换内存为xx兆字节。警告阈值:200字节
GGPUBR: HOW TO ADD ADJUSTED P-VALUES TO A MULTI-PANEL GGPLOT