当前位置:网站首页>How spark gets columns in dataframe --column, $, column, apply
How spark gets columns in dataframe --column, $, column, apply
2022-07-06 00:28:00 【The south wind knows what I mean】
Spark obtain DataFrame The way listed in --col,$,column,apply
1 Official statement
df("columnName") // On a specific data frame
col("columnName") // Generic columns that have not been associated with data frames
col("columnName.field") // Extract structural fields
col("`a.column.with.dots`") // escape . In the column name
$"columnName" //Scala Abbreviations of named columns
expr("a + 1") // Resolved by SQL Expression constructed column
lit("abc") // Generate text ( Constant ) Columns of values
2 Packages involved in use
//
import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
3 Demo
//
scala> val idCol = $"id"
idCol: org.apache.spark.sql.ColumnName = id
scala> val idCol = col("id")
idCol: org.apache.spark.sql.Column = id
scala> val idCol = column("id")
idCol: org.apache.spark.sql.Column = id
scala> val dataset = spark.range(5).toDF("text")
dataset: org.apache.spark.sql.DataFrame = [text: bigint]
scala> val textCol = dataset.col("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset.apply("text")
textCol: org.apache.spark.sql.Column = text
scala> val textCol = dataset("text")
textCol: org.apache.spark.sql.Column = text
边栏推荐
- 剖面测量之提取剖面数据
- DEJA_VU3D - Cesium功能集 之 055-国内外各厂商地图服务地址汇总说明
- [noi simulation] Anaid's tree (Mobius inversion, exponential generating function, Ehrlich sieve, virtual tree)
- Arduino六足机器人
- Global and Chinese markets of POM plastic gears 2022-2028: Research Report on technology, participants, trends, market size and share
- Go learning --- read INI file
- 小程序技术优势与产业互联网相结合的分析
- notepad++正则表达式替换字符串
- 【文件IO的简单实现】
- 【线上小工具】开发过程中会用到的线上小工具合集
猜你喜欢

Intranet Security Learning (V) -- domain horizontal: SPN & RDP & Cobalt strike

Arduino六足机器人

Priority queue (heap)

Gd32f4xx UIP protocol stack migration record

Configuring OSPF load sharing for Huawei devices

The relationship between FPGA internal hardware structure and code

Hardware and interface learning summary

Opencv classic 100 questions

Notepad + + regular expression replace String

Leetcode:20220213 week race (less bugs, top 10% 555)
随机推荐
Global and Chinese markets for hinged watertight doors 2022-2028: Research Report on technology, participants, trends, market size and share
MDK debug时设置数据实时更新
N1 # if you work on a metauniverse product [metauniverse · interdisciplinary] Season 2 S2
SQLServer连接数据库读取中文乱码问题解决
Start from the bottom structure and learn the introduction of fpga---fifo IP core and its key parameters
NLP basic task word segmentation third party Library: ICTCLAS [the third party library with the highest accuracy of Chinese word segmentation] [Chinese Academy of Sciences] [charge]
Hardware and interface learning summary
uniapp开发,打包成H5部署到服务器
MySQL storage engine
The relationship between FPGA internal hardware structure and code
Key structure of ffmpeg -- AVCodecContext
PHP determines whether an array contains the value of another array
Extension and application of timestamp
[EI conference sharing] the Third International Conference on intelligent manufacturing and automation frontier in 2022 (cfima 2022)
[noi simulation] Anaid's tree (Mobius inversion, exponential generating function, Ehrlich sieve, virtual tree)
Browser local storage
免费的聊天机器人API
MySQL global lock and table lock
Configuring OSPF load sharing for Huawei devices
从底层结构开始学习FPGA----FIFO IP核及其关键参数介绍