当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- How much do you know about the bank deposit business that software test engineers must know?
- [designmode] composite mode
- FFMPEG关键结构体——AVCodecContext
- PV static creation and dynamic creation
- MySql——CRUD
- Wechat applet -- wxml template syntax (with notes)
- 提升工作效率工具:SQL批量生成工具思想
- Go learning - dependency injection
- Uniapp development, packaged as H5 and deployed to the server
- 多线程与高并发(8)—— 从CountDownLatch总结AQS共享锁(三周年打卡)
猜你喜欢
2022-02-13 work record -- PHP parsing rich text
MySQL functions
Single merchant v4.4 has the same original intention and strength!
Go learning --- structure to map[string]interface{}
Tools to improve work efficiency: the idea of SQL batch generation tools
MDK debug时设置数据实时更新
Configuring OSPF GR features for Huawei devices
anconda下载+添加清华+tensorflow 安装+No module named ‘tensorflow‘+KernelRestarter: restart failed,内核重启失败
建立时间和保持时间的模型分析
Leetcode 450 deleting nodes in a binary search tree
随机推荐
Global and Chinese market of digital serial inverter 2022-2028: Research Report on technology, participants, trends, market size and share
GD32F4xx uIP协议栈移植记录
JS can really prohibit constant modification this time!
Miaochai Weekly - 8
小程序技术优势与产业互联网相结合的分析
JS 这次真的可以禁止常量修改了!
认识提取与显示梅尔谱图的小实验(观察不同y_axis和x_axis的区别)
行列式学习笔记(一)
How much do you know about the bank deposit business that software test engineers must know?
FPGA内部硬件结构与代码的关系
Date类中日期转成指定字符串出现的问题及解决方法
Chapter 16 oauth2authorizationrequestredirectwebfilter source code analysis
USB Interface USB protocol
Intranet Security Learning (V) -- domain horizontal: SPN & RDP & Cobalt strike
What is information security? What is included? What is the difference with network security?
7.5 装饰器
如何解决ecology9.0执行导入流程流程产生的问题
Configuring OSPF load sharing for Huawei devices
Yunna | what are the main operating processes of the fixed assets management system
Huawei equipment configuration ospf-bgp linkage