当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- 行列式学习笔记(一)
- 【NOI模拟赛】Anaid 的树(莫比乌斯反演,指数型生成函数,埃氏筛,虚树)
- Pointer - character pointer
- What is information security? What is included? What is the difference with network security?
- 7.5 装饰器
- State mode design procedure: Heroes in the game can rest, defend, attack normally and attack skills according to different physical strength values.
- 数据分析思维分析方法和业务知识——分析方法(二)
- [Online gadgets] a collection of online gadgets that will be used in the development process
- Anconda download + add Tsinghua +tensorflow installation +no module named 'tensorflow' +kernelrestart: restart failed, kernel restart failed
- Intranet Security Learning (V) -- domain horizontal: SPN & RDP & Cobalt strike
猜你喜欢

【NOI模拟赛】Anaid 的树(莫比乌斯反演,指数型生成函数,埃氏筛,虚树)

行列式学习笔记(一)

Problems and solutions of converting date into specified string in date class

权限问题:source .bash_profile permission denied

Go learning - dependency injection

GD32F4xx uIP协议栈移植记录

Notepad + + regular expression replace String

MySql——CRUD

FFMPEG关键结构体——AVFrame

Wechat applet -- wxml template syntax (with notes)
随机推荐
7.5模拟赛总结
FFmpeg抓取RTSP图像进行图像分析
[designmode] adapter pattern
Go learning --- read INI file
数据分析思维分析方法和业务知识——分析方法(三)
JS can really prohibit constant modification this time!
2022-02-13 work record -- PHP parsing rich text
Hardware and interface learning summary
Extracting profile data from profile measurement
提升工作效率工具:SQL批量生成工具思想
Huawei equipment configuration ospf-bgp linkage
2022.7.5-----leetcode. seven hundred and twenty-nine
Leetcode 450 deleting nodes in a binary search tree
The difference of time zone and the time library of go language
Gd32f4xx UIP protocol stack migration record
LeetCode 1598. Folder operation log collector
【QT】Qt使用QJson生成json文件并保存
Global and Chinese market of digital serial inverter 2022-2028: Research Report on technology, participants, trends, market size and share
MySql——CRUD
Global and Chinese markets for hinged watertight doors 2022-2028: Research Report on technology, participants, trends, market size and share