当前位置:网站首页>Spark获取DataFrame中列的方式--col,$,column,apply
Spark获取DataFrame中列的方式--col,$,column,apply
2022-07-06 00:23:00 【南风知我意丿】
Spark获取DataFrame中列的方式--col,$,column,apply
1 官方说明
df("columnName") //在特定数据帧上
col("columnName") //尚未与数据帧关联的通用列
col("columnName.field") //提取结构字段
col("`a.column.with.dots`") //转义。在列名中
$"columnName" //Scala命名列的缩写
expr("a + 1") //由解析的SQL表达式构造的列
lit("abc") //产生文字(常量)值的列
2 使用时涉及到的的包
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- Problems encountered in the database
- Data analysis thinking analysis methods and business knowledge -- analysis methods (II)
- 从底层结构开始学习FPGA----FIFO IP核及其关键参数介绍
- 免费的聊天机器人API
- Power Query数据格式的转换、拆分合并提取、删除重复项、删除错误、转置与反转、透视和逆透视
- 认识提取与显示梅尔谱图的小实验(观察不同y_axis和x_axis的区别)
- How to solve the problems caused by the import process of ecology9.0
- Knowledge about the memory size occupied by the structure
- 如何利用Flutter框架开发运行小程序
- FFMPEG关键结构体——AVFrame
猜你喜欢
FFMPEG关键结构体——AVFrame
数据分析思维分析方法和业务知识——分析方法(三)
AtCoder Beginner Contest 258【比赛记录】
Detailed explanation of APP functions of door-to-door appointment service
The difference of time zone and the time library of go language
Choose to pay tribute to the spirit behind continuous struggle -- Dialogue will values [Issue 4]
【EI会议分享】2022年第三届智能制造与自动化前沿国际会议(CFIMA 2022)
Extracting profile data from profile measurement
Huawei equipment is configured with OSPF and BFD linkage
Basic introduction and source code analysis of webrtc threads
随机推荐
Extracting profile data from profile measurement
Single merchant v4.4 has the same original intention and strength!
JS can really prohibit constant modification this time!
Leetcode 450 deleting nodes in a binary search tree
Tools to improve work efficiency: the idea of SQL batch generation tools
DEJA_VU3D - Cesium功能集 之 055-国内外各厂商地图服务地址汇总说明
PHP determines whether an array contains the value of another array
Global and Chinese market of water heater expansion tank 2022-2028: Research Report on technology, participants, trends, market size and share
Mathematical model Lotka Volterra
Room cannot create an SQLite connection to verify the queries
Configuring OSPF GR features for Huawei devices
Power Query数据格式的转换、拆分合并提取、删除重复项、删除错误、转置与反转、透视和逆透视
FFMPEG关键结构体——AVCodecContext
Leetcode:20220213 week race (less bugs, top 10% 555)
Key structure of ffmpeg - avframe
2022.7.5-----leetcode.729
【QT】Qt使用QJson生成json文件并保存
7.5 装饰器
Global and Chinese markets for pressure and temperature sensors 2022-2028: Research Report on technology, participants, trends, market size and share
[Chongqing Guangdong education] Chongqing Engineering Vocational and Technical College