当前位置:网站首页>How spark gets columns in dataframe --column, $, column, apply
How spark gets columns in dataframe --column, $, column, apply
2022-07-06 00:28:00 【The south wind knows what I mean】
Spark obtain DataFrame The way listed in --col,$,column,apply
1 Official statement
df("columnName") // On a specific data frame
col("columnName") // Generic columns that have not been associated with data frames
col("columnName.field") // Extract structural fields
col("`a.column.with.dots`") // escape . In the column name
$"columnName" //Scala Abbreviations of named columns
expr("a + 1") // Resolved by SQL Expression constructed column
lit("abc") // Generate text ( Constant ) Columns of values
2 Packages involved in use
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- Key structure of ffmpeg -- AVCodecContext
- The global and Chinese markets of dial indicator calipers 2022-2028: Research Report on technology, participants, trends, market size and share
- STM32 configuration after chip replacement and possible errors
- Teach you to run uni app with simulator on hbuilderx, conscience teaching!!!
- STM32按键消抖——入门状态机思维
- DEJA_VU3D - Cesium功能集 之 055-国内外各厂商地图服务地址汇总说明
- Global and Chinese markets for hinged watertight doors 2022-2028: Research Report on technology, participants, trends, market size and share
- Notepad++ regular expression replacement string
- Atcoder beginer contest 258 [competition record]
- 【EI会议分享】2022年第三届智能制造与自动化前沿国际会议(CFIMA 2022)
猜你喜欢

MySql——CRUD

State mode design procedure: Heroes in the game can rest, defend, attack normally and attack skills according to different physical strength values.

认识提取与显示梅尔谱图的小实验(观察不同y_axis和x_axis的区别)

FPGA内部硬件结构与代码的关系

XML配置文件

Priority queue (heap)

Arduino六足机器人

小程序技术优势与产业互联网相结合的分析

There is no network after configuring the agent by capturing packets with Fiddler mobile phones

MySQL存储引擎
随机推荐
Extracting profile data from profile measurement
Browser local storage
Common API classes and exception systems
《编程之美》读书笔记
【DesignMode】适配器模式(adapter pattern)
Spark SQL空值Null,NaN判断和处理
Configuring OSPF GR features for Huawei devices
[Chongqing Guangdong education] Chongqing Engineering Vocational and Technical College
Hudi of data Lake (2): Hudi compilation
Choose to pay tribute to the spirit behind continuous struggle -- Dialogue will values [Issue 4]
数据分析思维分析方法和业务知识——分析方法(二)
[designmode] composite mode
LeetCode 6005. The minimum operand to make an array an alternating array
Hudi of data Lake (1): introduction to Hudi
FFMPEG关键结构体——AVFrame
FPGA内部硬件结构与代码的关系
2022-02-13 work record -- PHP parsing rich text
Power Query数据格式的转换、拆分合并提取、删除重复项、删除错误、转置与反转、透视和逆透视
MySQL storage engine
[designmode] adapter pattern