当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- FPGA内部硬件结构与代码的关系
- Global and Chinese markets for pressure and temperature sensors 2022-2028: Research Report on technology, participants, trends, market size and share
- uniapp开发,打包成H5部署到服务器
- Global and Chinese market of valve institutions 2022-2028: Research Report on technology, participants, trends, market size and share
- Detailed explanation of APP functions of door-to-door appointment service
- Codeforces Round #804 (Div. 2)【比赛记录】
- MDK debug时设置数据实时更新
- Search (DFS and BFS)
- 常用API类及异常体系
- Browser local storage
猜你喜欢

MySql——CRUD

2022-02-13 work record -- PHP parsing rich text
![[designmode] Decorator Pattern](/img/65/457e0287383d0ca9a28703a63b4e1a.png)
[designmode] Decorator Pattern

Location based mobile terminal network video exploration app system documents + foreign language translation and original text + guidance records (8 weeks) + PPT + review + project source code

电机的简介

Classical concurrency problem: the dining problem of philosophers

About the slmgr command

MySQL存储引擎
![N1 # if you work on a metauniverse product [metauniverse · interdisciplinary] Season 2 S2](/img/f3/8e237296f5948dd0488441aa625182.jpg)
N1 # if you work on a metauniverse product [metauniverse · interdisciplinary] Season 2 S2

Hudi of data Lake (1): introduction to Hudi
随机推荐
QT -- thread
FPGA内部硬件结构与代码的关系
Location based mobile terminal network video exploration app system documents + foreign language translation and original text + guidance records (8 weeks) + PPT + review + project source code
Problems and solutions of converting date into specified string in date class
Go learning --- structure to map[string]interface{}
JS can really prohibit constant modification this time!
DEJA_ Vu3d - cesium feature set 055 - summary description of map service addresses of domestic and foreign manufacturers
notepad++正則錶達式替換字符串
AtCoder Beginner Contest 258【比赛记录】
《编程之美》读书笔记
LeetCode 8. String conversion integer (ATOI)
Power query data format conversion, Split Merge extraction, delete duplicates, delete errors, transpose and reverse, perspective and reverse perspective
Shardingsphere source code analysis
【DesignMode】装饰者模式(Decorator pattern)
FFmpeg学习——核心模块
Hudi of data Lake (1): introduction to Hudi
Gd32f4xx UIP protocol stack migration record
Global and Chinese market of valve institutions 2022-2028: Research Report on technology, participants, trends, market size and share
Classical concurrency problem: the dining problem of philosophers
[Chongqing Guangdong education] Chongqing Engineering Vocational and Technical College