当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-25 15:10:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- 6线SPI传输模式探索
- 剑指Offer | 二进制中1的个数
- QT connect, signal, slot and lambda comparison
- Automatically set the template for VS2010 and add header comments
- vscode 插件篇收集
- MySQL sort
- C语言函数复习(传值传址【二分查找】,递归【阶乘,汉诺塔等】)
- 32 use of chrome debugging tools
- Introduction to raspberry Pie: initial settings of raspberry pie
- Visual Studio 2022 查看类关系图
猜你喜欢

"Ask every day" reentrantlock locks and unlocks

39 simple version of millet sidebar exercise

Gbdt source code analysis of boosting

Spark SQL空值Null,NaN判断和处理

6月产品升级观察站

27 classification of selectors

反射-笔记

String type time comparison method with error string.compareto

密码强度验证示例

Pl/sql creates and executes ORALCE stored procedures and returns the result set
随机推荐
[C题目]力扣88. 合并两个有序数组
Introduction to raspberry Pie: initial settings of raspberry pie
ES5写继承的思路
When using jetty to run items, an error is reported: form too large or form too many keys
Scala110-combineByKey
基于OpenCV和YOLOv3的目标检测实例应用
简易轮播图和打地鼠
Handle Oracle deadlock
[C topic] Li Kou 88. merge two ordered arrays
Spark 判断DF为空
oracle_ 12505 error resolution
Implementation of redis distributed lock
ice 100G 网卡分片报文 hash 问题
Login of MySQL [database system]
什么是物联网
如何更新更新数据库中的json值?
pkg_resources动态加载插件
任务、微任务、队列和调度(动画展示每一步调用)
As methods for viewing and excluding dependencies
Unable to start web server when Nacos starts