当前位置:网站首页>Spark 判断DF为空
Spark 判断DF为空
2022-07-25 15:10:00 【南风知我意丿】
不同方案性能测试
我有同样的问题,并且测试了3个主要解决方案:
df!= null df.count> 0
df.head(1).isEmpty()
df.rdd.isEmpty
大约需要9366ms
大约需要5607毫秒
大约需要1921ms
当然这3种有效,但是就性能而言,这是我在执行时间方面在我的机器的同一DF上执行这些方法时发现的
更新另一种方法
/** * 判断DataFrame是否为空 * @param df DataFrame * @return true 表示为空 or false 表示非空 */
def isEmpty(df: DataFrame): Boolean ={
try{
df.head()
false
}catch {
case e: NoSuchElementException =>
println(e.getMessage)
true
}
}
边栏推荐
- vscode 插件篇收集
- 在win10系统下使用命令查看WiFi连接密码
- Client error: invalid param endpoint is blank
- 继承的实现过程及ES5和ES6实现的区别
- Sublimetext-win10 cursor following problem
- 图片的懒加载
- IP address classification, which determines whether a network segment is a subnet supernetwork
- Spark002---spark任务提交,传入json作为参数
- 35 quick format code
- 万能通用智能JS表单验证
猜你喜欢

System. Accessviolationexception: an attempt was made to read or write to protected memory. This usually indicates that other memory is corrupted

在win10系统下使用命令查看WiFi连接密码

VS2010 add WAP mobile form template

node学习

如何解决Visual Studio中scanf编译报错的问题

npm的nexus私服 E401 E500错误处理记录

Deployment and simple use of PostgreSQL learning

pl/sql 创建并执行oralce存储过程,并返回结果集

SPI传输出现数据与时钟不匹配延后问题分析与解决

39 simple version of millet sidebar exercise
随机推荐
[Nacos] what does nacosclient do during service registration
Realsense ROS installation configuration introduction and problem solving
Boosting之GBDT源码分析
了解一下new的过程发生了什么
[comprehensive pen test] difficulty 4/5, classic application of line segment tree for character processing
Handle Oracle deadlock
推荐10个堪称神器的学习网站
"Ask every day" what is volatile
Melody + realsense d435i configuration and error resolution
Award winning interaction | 7.19 database upgrade plan practical Summit: industry leaders gather, why do they come?
oracle_ 12505 error resolution
45padding不会撑开盒子的情况
ice 100G 网卡分片报文 hash 问题
C语言函数复习(传值传址【二分查找】,递归【阶乘,汉诺塔等】)
Reprint ---- how to read the code?
Vs2010添加wap移动窗体模板
Splice a field of the list set into a single string
转载----如何阅读代码?
[C topic] Li Kou 88. merge two ordered arrays
Leo-sam: tightly coupled laser inertial odometer with smoothing and mapping