当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-25 15:10:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
猜你喜欢

"Ask every day" reentrantlock locks and unlocks

Leo-sam: tightly coupled laser inertial odometer with smoothing and mapping

oracle_ 12505 error resolution

Introduction to raspberry Pie: initial settings of raspberry pie

反射-笔记

Fast-lio: fast and robust laser inertial odometer based on tightly coupled IEKF

"Ask every day" how locksupport realizes thread waiting and wakeup

《三子棋》C语言数组应用 --n皇后问题雏形

Boosting之GBDT源码分析

MySQL之事务与MVCC
随机推荐
pkg_resources动态加载插件
Automatically set the template for VS2010 and add header comments
35 quick format code
Universal smart JS form verification
Sudo rosdep init error ROS installation problem solution
SublimeText-win10光标跟随问题
Add the jar package under lib directory to the project in idea
MySQL sort
LeetCode_ Factorization_ Simple_ 263. Ugly number
Vs2010添加wap移动窗体模板
TypeScript学习2——接口
ESXI6.7.0 升级到7.0U3f(2022年7月12 更新)
Content type corresponding to office file
Scala110-combineByKey
node学习
Visual Studio 2022 查看类关系图
Implement a simple restful API server
瀑布流布局
如何解决Visual Studio中scanf编译报错的问题
iframe嵌套其它网站页面 全屏设置