当前位置:网站首页>Spark judges that DF is empty
Spark judges that DF is empty
2022-07-07 20:23:00 【The south wind knows what I mean】
List of articles
Performance test of different schemes
I have the same question , And tested 3 There are three main solutions :
df!= null df.count> 0
df.head(1).isEmpty()
df.rdd.isEmpty
About need 9366ms
About need 5607 millisecond
About need 1921ms
Of course. 3 Species effective , But in terms of performance , This is the same in my machine in terms of execution time DF Found when executing these methods on
Update another method
/** * Judge DataFrame Is it empty * @param df DataFrame * @return true Said is empty or false It means not empty */
def isEmpty(df: DataFrame): Boolean ={
try{
df.head()
false
}catch {
case e: NoSuchElementException =>
println(e.getMessage)
true
}
}
边栏推荐
- Micro service remote debug, nocalhost + rainbow micro service development second bullet
- POJ 1742 coins (monotone queue solution) [suggestions collection]
- The boundary of Bi: what is bi not suitable for? Master data, Martech? How to expand?
- Deep learning model compression and acceleration technology (VII): mixed mode
- TS quick start - Generic
- 使用高斯Redis实现二级索引
- Force buckle 643 Subarray maximum average I
- ISO 26262 - 基于需求测试以外的考虑因素
- 使用高斯Redis实现二级索引
- 有了ST7008, 蓝牙测试完全拿捏住了
猜你喜欢
随机推荐
一文读懂数仓中的pg_stat
EasyGBS级联时,上级平台重启导致推流失败、画面卡住该如何解决?
JNI 初级接触
Force buckle 674 Longest continuous increasing sequence
一. 基础概念
Useful win11 tips
测量楼的高度
机器学习笔记 - 使用Streamlit探索对象检测数据集
I wrote a markdown command line gadget, hoping to improve the efficiency of sending documents by garden friends!
POJ 1742 Coins ( 单调队列解法 )「建议收藏」
Micro service remote debug, nocalhost + rainbow micro service development second bullet
Force buckle 643 Subarray maximum average I
深度学习模型压缩与加速技术(七):混合方式
School 1 of vulnhub
[auto.js] automatic script
Chapter 20 using work queue manager (3)
微服务远程Debug,Nocalhost + Rainbond微服务开发第二弹
CIS芯片测试到底怎么测?
Network principle (1) - overview of basic principles
实战:sqlserver 2008 扩展事件-XML转换为标准的table格式[通俗易懂]