当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
猜你喜欢

Leetcode:20220213 week race (less bugs, top 10% 555)

How much do you know about the bank deposit business that software test engineers must know?

硬件及接口学习总结

Anconda download + add Tsinghua +tensorflow installation +no module named 'tensorflow' +kernelrestart: restart failed, kernel restart failed

建立时间和保持时间的模型分析

FPGA内部硬件结构与代码的关系

Leetcode 450 deleting nodes in a binary search tree

FFMPEG关键结构体——AVFormatContext

Uniapp development, packaged as H5 and deployed to the server

FFT learning notes (I think it is detailed)
随机推荐
[Chongqing Guangdong education] reference materials for Zhengzhou Vocational College of finance, taxation and finance to play around the E-era
Tools to improve work efficiency: the idea of SQL batch generation tools
[designmode] adapter pattern
Leetcode 450 deleting nodes in a binary search tree
Detailed explanation of APP functions of door-to-door appointment service
MySQL之函数
Single merchant v4.4 has the same original intention and strength!
Teach you to run uni app with simulator on hbuilderx, conscience teaching!!!
从底层结构开始学习FPGA----FIFO IP核及其关键参数介绍
LeetCode 6004. Get operands of 0
Atcoder beginer contest 258 [competition record]
NSSA area where OSPF is configured for Huawei equipment
小程序技术优势与产业互联网相结合的分析
硬件及接口学习总结
7.5 decorator
Shardingsphere source code analysis
JS can really prohibit constant modification this time!
[Online gadgets] a collection of online gadgets that will be used in the development process
PV static creation and dynamic creation
关于slmgr命令的那些事