当前位置:网站首页>Construction of Flink development environment and wordcount
Construction of Flink development environment and wordcount
2022-06-11 02:53:00 【Xiao Pang is so fierce!】
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>bigdata16</artifactId>
<groupId>com.shujia</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>Flink</artifactId>
<version>1.0</version>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<flink.version>1.11.2</flink.version>
<scala.binary.version>2.11</scala.binary.version>
<scala.version>2.11.12</scala.version>
<log4j.version>2.12.1</log4j.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-walkthrough-common_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Java Compiler -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!-- Scala Compiler -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>package com.shujia.core
import org.apache.flink.streaming.api.scala._
object Demo01WordCount {
def main(args: Array[String]): Unit = {
/**
* establish Flink entrance
*/
val env: StreamExecutionEnvironment= StreamExecutionEnvironment.getExecutionEnvironment
// Default parallelism equals CPU The number of logical cores It is equivalent to a parallelism of the task
env.setParallelism(2)
/**
* adopt Socket Simulate real-time data
* nc -lk 8888
*
* DataStream:Flink The programming model in
*/
val linesDS: DataStream[String] = env.socketTextStream("master", 8888)
// Segment each piece of data
val wordsDS: DataStream[String] = linesDS
.flatMap(line => line.split(","))
// Turn each word into K V Format
val wordsKVDS: DataStream[(String, Int)] = wordsDS.map(word => (word, 1))
// Group according to each word
val keyByDS: KeyedStream[(String, Int), String] = wordsKVDS.keyBy(kv => kv._1)
// Count the number of words
val wordCntDS: DataStream[(String, Int)] = keyByDS.sum(1)
// Print the results
wordCntDS.print()
// Start the task
env.execute("Demo01WordCount")
}
}边栏推荐
- Win10 安装Office 2016出现错误代码30204-44怎么处理?
- [Fibonacci series]
- AOSP - Developer mode is enabled by default
- Uni app - one click access to user information
- 13. numeric array
- Arduino Uno接JQ8900-16p语音播报模块
- [resolved] how to fix another update in progress WordPress upgrade error
- error exepected identifier before ‘(‘ token, grpc 枚举类编译错误
- Manon's advanced road - Daily anecdotes
- 怎样简洁明了地说清楚产品需求?
猜你喜欢

軟件測試英語常見詞匯

【大咖秀】博睿数据眼中的AIOps,选择正确的赛道正确的人

ADVANCE.AI首席执行官寿栋将在2022新兴市场品牌出海线上峰会分享跨境电商运用AI技术合规

Niuke: two numbers that only appear once in the array

CPT 102_ LEC 16

RS232/RS485转4G DTU 上传基于Modbus协议的温湿度传感器数据到远程TCP服务器

Can Xiaoxiang life become the "Yonghui" in the discount industry after the completion of the round a financing of tens of millions of yuan?

What is ttfb

【冒泡排序的实现】

Limiting visibility of symbols when linking shared libraries
随机推荐
Learning Tai Chi Maker - esp8226 (II)
AOSP ~ WIFI默认开启 + GPS默认关闭 + 蓝牙默认关闭 + 旋转屏幕关闭
How to add two factor authentication for WordPress websites
CPT 102_ LEC 16
弄懂了采矿业与碳中和的逻辑,就读懂了矿区无人驾驶的千亿市场
HOSATAPD_ CLI debug using WiFi whitelist
jdbc工具類的問題
Jetpack compose box control
6 best WordPress Image optimizer plug-ins to improve WordPress website performance
两部门联合印发《校外培训机构消防安全管理九项规定》
[resolved] how to fix another update in progress WordPress upgrade error
CPT 102_LEC 15
Write my Ini configuration file error
【长时间序列预测】Aotoformer 代码详解之[3]模型整体架构分析
Xampp is used under M1 chip, and the installation extension error
Explanation of spark common parameters
Arduino Uno接JQ8900-16p语音播报模块
CPT 102_ LEC 17
OpenJudge NOI 1.13 18:Tomorrow never knows?
If you understand the logic of mining and carbon neutrality, you will understand the 100 billion market of driverless mining areas