当前位置:网站首页>Spark: specify the date and output the log of the corresponding date (entry level - simple implementation)
Spark: specify the date and output the log of the corresponding date (entry level - simple implementation)
2022-07-24 14:37:00 【One's cow】
The specified date is 2015.05.20, Output the log of this day .
The code is as follows :( Find the log file yourself )
import org.apache.spark.{SparkConf, SparkContext}
object RDD_Operator_Transform_filter_Test {
def main(args: Array[String]): Unit = {
//TODO Create an environment
val sparkConf = new SparkConf().setMaster("local[*]").setAppName("Operator")
val sc = new SparkContext(sparkConf)
//TODO RDD operator ——filter
val rdd = sc.textFile("datas/apache.log")
val filterRDD = rdd.filter(
line => {
val datas = line.split(" ")
val time = datas(3)
time.startsWith("20/05/2015")
}
)
filterRDD.collect().foreach(println)
//TODO Shut down the environment
sc.stop()
}
}边栏推荐
- Problem handling of repeated restart during Siemens botu installation
- Stack and queue - 20. Valid parentheses
- Rasa 3.x learning series -rasa fallbackclassifier source code learning notes
- Binlog and iptables prevent nmap scanning, xtrabackup full + incremental backup, and the relationship between redlog and binlog
- 多数据源配置下,解决org.apache.ibatis.binding.BindingException: Invalid bound statement (not found)问题
- The difference and relation among list, set and map
- "After 00" is coming! Digital data ushers in a new generation of "codeless" forces
- Kotlin class and inheritance
- bibliometrix: 从千万篇论文中挖掘出最值得读的那一篇!
- Clear all spaces in the string
猜你喜欢

Rest style

Su Chunyuan, founder of science and technology · CEO of Guanyuan data: making business use is the key to the Bi industry to push down the wall of penetration

After five years of contact with nearly 100 bosses, as a headhunter, I found that the secret of promotion was only four words

ISPRS2018/云检测:Cloud/shadow detection based on spectral indices for multi/hyp基于光谱指数的多/高光谱光学遥感成像仪云/影检测

Unity 委托 (Delegate) 的简单理解以及实现

2022 IAA industry category development insight series report - phase II

对话框管理器第二章:创建框架窗口

Kotlin class and inheritance

Learning and thinking about the relevant knowledge in the direction of building network security knowledge base

Maotai ice cream "bucked the trend" and became popular, but its cross-border meaning was not "selling ice cream"
随机推荐
DDD based on ABP -- Entity creation and update
The sliding window of Li Kou "step by step" (209. The smallest sub array, 904. Fruit baskets)
Kotlin class and inheritance
2022 IAA industry category development insight series report - phase II
Rasa 3.x 学习系列-Rasa FallbackClassifier源码学习笔记
IntelliSense of Visual Studio: 'no members available'
Multithreaded common classes
Regular expression and bypass cases
ISPRS2018/云检测:Cloud/shadow detection based on spectral indices for multi/hyp基于光谱指数的多/高光谱光学遥感成像仪云/影检测
PCA of [machine learning]
How vscode debug nodejs
TypeError: 'str' object does not support item assignment
mysql
Rasa 3.x learning series -rasa fallbackclassifier source code learning notes
String - Sword finger offer 58 - ii Rotate string left
Nodejs uses the express framework to post the request message "badrequesterror:request aborted"
The difference and relation among list, set and map
Must use destructuring props assignmenteslint
Moving the mouse into select options will trigger the mouseleave event processing scheme
String - 459. Repeated substrings