当前位置:网站首页>Construction of Flink development environment and wordcount
Construction of Flink development environment and wordcount
2022-06-11 02:53:00 【Xiao Pang is so fierce!】
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>bigdata16</artifactId>
<groupId>com.shujia</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>Flink</artifactId>
<version>1.0</version>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<flink.version>1.11.2</flink.version>
<scala.binary.version>2.11</scala.binary.version>
<scala.version>2.11.12</scala.version>
<log4j.version>2.12.1</log4j.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-walkthrough-common_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Java Compiler -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!-- Scala Compiler -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>package com.shujia.core
import org.apache.flink.streaming.api.scala._
object Demo01WordCount {
def main(args: Array[String]): Unit = {
/**
* establish Flink entrance
*/
val env: StreamExecutionEnvironment= StreamExecutionEnvironment.getExecutionEnvironment
// Default parallelism equals CPU The number of logical cores It is equivalent to a parallelism of the task
env.setParallelism(2)
/**
* adopt Socket Simulate real-time data
* nc -lk 8888
*
* DataStream:Flink The programming model in
*/
val linesDS: DataStream[String] = env.socketTextStream("master", 8888)
// Segment each piece of data
val wordsDS: DataStream[String] = linesDS
.flatMap(line => line.split(","))
// Turn each word into K V Format
val wordsKVDS: DataStream[(String, Int)] = wordsDS.map(word => (word, 1))
// Group according to each word
val keyByDS: KeyedStream[(String, Int), String] = wordsKVDS.keyBy(kv => kv._1)
// Count the number of words
val wordCntDS: DataStream[(String, Int)] = keyByDS.sum(1)
// Print the results
wordCntDS.print()
// Start the task
env.execute("Demo01WordCount")
}
}边栏推荐
- Arduino Uno接JQ8900-16p语音播报模块
- 弄懂了采矿业与碳中和的逻辑,就读懂了矿区无人驾驶的千亿市场
- CPT 102_ LEC 20
- How to state clearly and concisely the product requirements?
- Introduction to the functions of today's headline search webmaster platform (portal)
- 【189. 轮转数组】
- 92. actual combat of completable future
- 扁平数据转tree与tree数据扁平化
- GraphAcademy 课程讲解:《Neo4j 图数据科学基础》
- GCC C inline assembly
猜你喜欢

学习太极创客 — ESP8226 (二)

Live broadcast! Feature matching of orb-slam3 series (mlpnp, word bag model, etc.).

弄懂了采矿业与碳中和的逻辑,就读懂了矿区无人驾驶的千亿市场
![【长时间序列预测】Aotoformer 代码详解之[3]模型整体架构分析](/img/77/30215c363ae8a1324db1cbcaa5324b.png)
【长时间序列预测】Aotoformer 代码详解之[3]模型整体架构分析

CPT 102_ LEC 15

HUST Software Engineering (Experiment 2) -- TDD test driven development experiment.

Learning Tai Chi Maker - esp8226 (II)

The two request sessionids of the same user are inconsistent ----- record the problem

CPT 102_LEC 17
![[C language classic]: inverted string](/img/f3/e6f37b852d22d395314628b73c9be3.jpg)
[C language classic]: inverted string
随机推荐
牛客网:数组中只出现一次的两个数字
Error excluded identifier before '(' token, grpc enumeration class compilation error
Prophet
Problèmes de classe d'outils JDBC
HOSATAPD_ CLI debug using WiFi whitelist
Jetpack compose box control
从绿联冲刺IPO,看手机配件市场沉浮录
CPT 102_ LEC 13-14
Databinding escaping with presentation symbols
cmake常用命令
Unity animator rewind
app 测试 常用 adb 命令集合
[Fibonacci series]
Looking at the ups and downs of the mobile phone accessories market from the green Union's sprint for IPO
AOSP ~ WIFI默认开启 + GPS默认关闭 + 蓝牙默认关闭 + 旋转屏幕关闭
Niuke: two numbers that only appear once in the array
近期学习和更新计划
Common vocabulary of software testing English
6 best WordPress Image optimizer plug-ins to improve WordPress website performance
Explanation of spark common parameters