当前位置:网站首页>Spark 判断DF为空
Spark 判断DF为空
2022-07-07 18:15:00 【南风知我意丿】
不同方案性能测试
我有同样的问题,并且测试了3个主要解决方案:
df!= null df.count> 0
df.head(1).isEmpty()
df.rdd.isEmpty
大约需要9366ms
大约需要5607毫秒
大约需要1921ms
当然这3种有效,但是就性能而言,这是我在执行时间方面在我的机器的同一DF上执行这些方法时发现的
更新另一种方法
/** * 判断DataFrame是否为空 * @param df DataFrame * @return true 表示为空 or false 表示非空 */
def isEmpty(df: DataFrame): Boolean ={
try{
df.head()
false
}catch {
case e: NoSuchElementException =>
println(e.getMessage)
true
}
}
边栏推荐
- MSE API学习
- How to implement safety practice in software development stage
- Machine learning notes - explore object detection datasets using streamlit
- 浅尝不辄止系列之试试腾讯云的TUIRoom(晚上有约,未完待续...)
- 开发那些事儿:Go加C.free释放内存,编译报错是什么原因?
- 有用的win11小技巧
- Traversal of Oracle stored procedures
- mock. JS returns an array from the optional data in the object array
- When easygbs cascades, how to solve the streaming failure and screen jam caused by the restart of the superior platform?
- equals 方法
猜你喜欢
写了个 Markdown 命令行小工具,希望能提高园友们发文的效率!
Yolov6:yolov6+win10--- train your own dataset
One click deployment of any version of redis
With st7008, the Bluetooth test is completely grasped
Force buckle 2319 Judge whether the matrix is an X matrix
The boundary of Bi: what is bi not suitable for? Master data, Martech? How to expand?
让这个CRMEB单商户微信商城系统火起来,太好用了!
AIRIOT助力城市管廊工程,智慧物联守护城市生命线
Simulate the implementation of string class
vulnhub之Funfox2
随机推荐
The state cyberspace Office released the measures for data exit security assessment: 100000 information provided overseas needs to be declared
恢复持久卷上的备份数据
Chapter 9 Yunji datacanvas company won the highest honor of the "fifth digital finance innovation competition"!
Creation of kubernetes mysql8
rk3128投影仪lcd显示四周显示不完整解决
【STL】vector
力扣 989. 数组形式的整数加法
大厂经典指针笔试题
JVM GC garbage collection brief
[solution] package 'XXXX' is not in goroot
网络原理(1)——基础原理概述
Read PG in data warehouse in one article_ stat
使用 BR 恢复 Azure Blob Storage 上的备份数据
有了ST7008, 蓝牙测试完全拿捏住了
Open source heavy ware! Chapter 9 the open source project of ylarn causal learning of Yunji datacanvas company will be released soon!
Kubernetes——kubectl命令行工具用法详解
MSE API learning
实战:sqlserver 2008 扩展事件-XML转换为标准的table格式[通俗易懂]
备份 TiDB 集群到持久卷
Useful win11 tips