当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- MySQL之函数
- 【DesignMode】适配器模式(adapter pattern)
- 如何解决ecology9.0执行导入流程流程产生的问题
- 权限问题:source .bash_profile permission denied
- Detailed explanation of APP functions of door-to-door appointment service
- notepad++正则表达式替换字符串
- 认识提取与显示梅尔谱图的小实验(观察不同y_axis和x_axis的区别)
- 数据分析思维分析方法和业务知识——分析方法(三)
- 【线上小工具】开发过程中会用到的线上小工具合集
- LeetCode 6006. Take out the least number of magic beans
猜你喜欢

MDK debug时设置数据实时更新
![N1 # if you work on a metauniverse product [metauniverse · interdisciplinary] Season 2 S2](/img/f3/8e237296f5948dd0488441aa625182.jpg)
N1 # if you work on a metauniverse product [metauniverse · interdisciplinary] Season 2 S2

Anconda download + add Tsinghua +tensorflow installation +no module named 'tensorflow' +kernelrestart: restart failed, kernel restart failed

State mode design procedure: Heroes in the game can rest, defend, attack normally and attack skills according to different physical strength values.

小程序技术优势与产业互联网相结合的分析
![Go learning --- structure to map[string]interface{}](/img/e3/59caa3f2ba5bd3647bdbba075ee60d.jpg)
Go learning --- structure to map[string]interface{}

There is no network after configuring the agent by capturing packets with Fiddler mobile phones

notepad++正则表达式替换字符串

Hudi of data Lake (1): introduction to Hudi

FFMPEG关键结构体——AVFrame
随机推荐
Intranet Security Learning (V) -- domain horizontal: SPN & RDP & Cobalt strike
7.5 装饰器
【DesignMode】组合模式(composite mode)
MySQL存储引擎
数据分析思维分析方法和业务知识——分析方法(三)
LeetCode 1598. Folder operation log collector
Single source shortest path exercise (I)
LeetCode 6005. The minimum operand to make an array an alternating array
Anconda download + add Tsinghua +tensorflow installation +no module named 'tensorflow' +kernelrestart: restart failed, kernel restart failed
QT -- thread
Permission problem: source bash_ profile permission denied
FFT 学习笔记(自认为详细)
2022.7.5-----leetcode.729
How to solve the problems caused by the import process of ecology9.0
wx. Getlocation (object object) application method, latest version
【QT】Qt使用QJson生成json文件并保存
wx.getLocation(Object object)申请方法,最新版
Problems and solutions of converting date into specified string in date class
About the slmgr command
Priority queue (heap)