当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- How much do you know about the bank deposit business that software test engineers must know?
- Classical concurrency problem: the dining problem of philosophers
- FFmpeg抓取RTSP图像进行图像分析
- Gd32f4xx UIP protocol stack migration record
- notepad++正则表达式替换字符串
- [online chat] the original wechat applet can also reply to Facebook homepage messages!
- MySQL global lock and table lock
- Hardware and interface learning summary
- Priority queue (heap)
- Key structure of ffmpeg - avframe
猜你喜欢

Recognize the small experiment of extracting and displaying Mel spectrum (observe the difference between different y_axis and x_axis)

OS i/o devices and device controllers

Knowledge about the memory size occupied by the structure

The difference of time zone and the time library of go language

Doppler effect (Doppler shift)

Permission problem: source bash_ profile permission denied

Gavin teacher's perception of transformer live class - rasa project actual combat e-commerce retail customer service intelligent business dialogue robot system behavior analysis and project summary (4

FFMPEG关键结构体——AVFrame
![[online chat] the original wechat applet can also reply to Facebook homepage messages!](/img/d2/1fd4de4bfd433ed397c236ddb97a66.png)
[online chat] the original wechat applet can also reply to Facebook homepage messages!

免费的聊天机器人API
随机推荐
[online chat] the original wechat applet can also reply to Facebook homepage messages!
OS i/o devices and device controllers
[Online gadgets] a collection of online gadgets that will be used in the development process
JS 这次真的可以禁止常量修改了!
notepad++正則錶達式替換字符串
DEJA_ Vu3d - cesium feature set 055 - summary description of map service addresses of domestic and foreign manufacturers
【NOI模拟赛】Anaid 的树(莫比乌斯反演,指数型生成函数,埃氏筛,虚树)
Calculate sha256 value of data or file based on crypto++
7.5 simulation summary
常用API类及异常体系
Key structure of ffmpeg - avformatcontext
The global and Chinese markets of dial indicator calipers 2022-2028: Research Report on technology, participants, trends, market size and share
Global and Chinese market of digital serial inverter 2022-2028: Research Report on technology, participants, trends, market size and share
Go learning --- read INI file
如何解决ecology9.0执行导入流程流程产生的问题
提升工作效率工具:SQL批量生成工具思想
Hudi of data Lake (2): Hudi compilation
Choose to pay tribute to the spirit behind continuous struggle -- Dialogue will values [Issue 4]
2022.7.5-----leetcode.729
Date类中日期转成指定字符串出现的问题及解决方法