当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- LeetCode 斐波那契序列
- 电机的简介
- USB Interface USB protocol
- Shardingsphere source code analysis
- OS i/o devices and device controllers
- Problems encountered in the database
- Global and Chinese market of valve institutions 2022-2028: Research Report on technology, participants, trends, market size and share
- Global and Chinese markets of POM plastic gears 2022-2028: Research Report on technology, participants, trends, market size and share
- 【DesignMode】适配器模式(adapter pattern)
- Go learning - dependency injection
猜你喜欢
Calculate sha256 value of data or file based on crypto++
Doppler effect (Doppler shift)
Hudi of data Lake (2): Hudi compilation
Teach you to run uni app with simulator on hbuilderx, conscience teaching!!!
anconda下载+添加清华+tensorflow 安装+No module named ‘tensorflow‘+KernelRestarter: restart failed,内核重启失败
How to solve the problems caused by the import process of ecology9.0
Determinant learning notes (I)
【EI会议分享】2022年第三届智能制造与自动化前沿国际会议(CFIMA 2022)
Ffmpeg learning - core module
常用API类及异常体系
随机推荐
There is no network after configuring the agent by capturing packets with Fiddler mobile phones
MySQL存储引擎
uniapp开发,打包成H5部署到服务器
Global and Chinese markets of POM plastic gears 2022-2028: Research Report on technology, participants, trends, market size and share
State mode design procedure: Heroes in the game can rest, defend, attack normally and attack skills according to different physical strength values.
Huawei equipment is configured with OSPF and BFD linkage
[designmode] adapter pattern
What are the functions of Yunna fixed assets management system?
[binary search tree] add, delete, modify and query function code implementation
Mathematical model Lotka Volterra
SQLServer连接数据库读取中文乱码问题解决
Browser local storage
Yunna | what are the main operating processes of the fixed assets management system
Configuring OSPF load sharing for Huawei devices
硬件及接口学习总结
wx. Getlocation (object object) application method, latest version
Global and Chinese markets of universal milling machines 2022-2028: Research Report on technology, participants, trends, market size and share
Wechat applet -- wxml template syntax (with notes)
Multithreading and high concurrency (8) -- summarize AQS shared lock from countdownlatch (punch in for the third anniversary)
Room cannot create an SQLite connection to verify the queries