当前位置:网站首页>Spark 判断DF为空
Spark 判断DF为空
2022-07-07 18:15:00 【南风知我意丿】
不同方案性能测试
我有同样的问题,并且测试了3个主要解决方案:
df!= null df.count> 0
df.head(1).isEmpty()
df.rdd.isEmpty
大约需要9366ms
大约需要5607毫秒
大约需要1921ms
当然这3种有效,但是就性能而言,这是我在执行时间方面在我的机器的同一DF上执行这些方法时发现的
更新另一种方法
/** * 判断DataFrame是否为空 * @param df DataFrame * @return true 表示为空 or false 表示非空 */
def isEmpty(df: DataFrame): Boolean ={
try{
df.head()
false
}catch {
case e: NoSuchElementException =>
println(e.getMessage)
true
}
}
边栏推荐
- JVM GC垃圾回收简述
- Force buckle 1790 Can two strings be equal by performing string exchange only once
- Get webkitformboundary post login
- Equals method
- Gorilla official: sample code for golang to open websocket client
- 【解决】package ‘xxxx‘ is not in GOROOT
- MSE API学习
- Semantic slam source code analysis
- POJ 1742 Coins ( 单调队列解法 )「建议收藏」
- 4G设备接入EasyGBS平台出现流量消耗异常,是什么原因?
猜你喜欢
About cv2 dnn. Readnetfromonnx (path) reports error during processing node with 3 inputs and 1 outputs [exclusive release]
The boundary of Bi: what is bi not suitable for? Master data, Martech? How to expand?
PHP method of obtaining image information
Detailed explanation of Flink parallelism and slot
【哲思与实战】程序设计之道
mysql 的一些重要知识
Machine learning notes - explore object detection datasets using streamlit
关于cv2.dnn.readNetFromONNX(path)就报ERROR during processing node with 3 inputs and 1 outputs的解决过程【独家发布】
使用camunda做工作流设计,驳回操作
ASP.NET学习& asp‘s one word
随机推荐
开发一个小程序商城需要多少钱?
[résolution] le paquet « xxxx» n'est pas dans goroot
关于cv2.dnn.readNetFromONNX(path)就报ERROR during processing node with 3 inputs and 1 outputs的解决过程【独家发布】
vulnhub之Funfox2
Force buckle 2319 Judge whether the matrix is an X matrix
mock. JS returns an array from the optional data in the object array
机械臂速成小指南(十二):逆运动学分析
九章云极DataCanvas公司摘获「第五届数字金融创新大赛」最高荣誉!
Vulnhub tre1
The state cyberspace Office released the measures for data exit security assessment: 100000 information provided overseas needs to be declared
Chapter 9 Yunji datacanvas was rated as 36 krypton "the hard core technology enterprise most concerned by investors"
Force buckle 88 Merge two ordered arrays
Force buckle 1961 Check whether the string is an array prefix
How to implement safety practice in software development stage
浅尝不辄止系列之试试腾讯云的TUIRoom(晚上有约,未完待续...)
力扣 912.排序数组
毕业季|遗憾而又幸运的毕业季
PHP method of obtaining image information
Precautions for cjson memory leakage
Cloud component development and upgrading