当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- MDK debug时设置数据实时更新
- LeetCode 6005. The minimum operand to make an array an alternating array
- Room cannot create an SQLite connection to verify the queries
- [designmode] adapter pattern
- LeetCode 6004. Get operands of 0
- Opencv classic 100 questions
- Location based mobile terminal network video exploration app system documents + foreign language translation and original text + guidance records (8 weeks) + PPT + review + project source code
- LeetCode 6006. Take out the least number of magic beans
- Browser local storage
- Problems encountered in the database
猜你喜欢

【NOI模拟赛】Anaid 的树(莫比乌斯反演,指数型生成函数,埃氏筛,虚树)

Gd32f4xx UIP protocol stack migration record

【EI会议分享】2022年第三届智能制造与自动化前沿国际会议(CFIMA 2022)

Classic CTF topic about FTP protocol
![Atcoder beginer contest 258 [competition record]](/img/e4/1d34410f79851a7a81dd8f4a0b54bf.gif)
Atcoder beginer contest 258 [competition record]

认识提取与显示梅尔谱图的小实验(观察不同y_axis和x_axis的区别)

Huawei equipment configuration ospf-bgp linkage

Basic introduction and source code analysis of webrtc threads

Room cannot create an SQLite connection to verify the queries

notepad++正則錶達式替換字符串
随机推荐
Extracting profile data from profile measurement
Gavin teacher's perception of transformer live class - rasa project actual combat e-commerce retail customer service intelligent business dialogue robot system behavior analysis and project summary (4
FFMPEG关键结构体——AVFrame
An understanding of & array names
Go learning --- read INI file
STM32 configuration after chip replacement and possible errors
MySql——CRUD
MySql——CRUD
DEJA_ Vu3d - cesium feature set 055 - summary description of map service addresses of domestic and foreign manufacturers
MySQL之函数
wx.getLocation(Object object)申请方法,最新版
Wechat applet -- wxml template syntax (with notes)
Permission problem: source bash_ profile permission denied
Codeforces gr19 D (think more about why the first-hand value range is 100, JLS yyds)
硬件及接口学习总结
Multithreading and high concurrency (8) -- summarize AQS shared lock from countdownlatch (punch in for the third anniversary)
Single source shortest path exercise (I)
Atcoder beginer contest 258 [competition record]
FFMPEG关键结构体——AVFormatContext
Global and Chinese markets for hinged watertight doors 2022-2028: Research Report on technology, participants, trends, market size and share