当前位置:网站首页>Flink 数据偶尔数据积压导致checkpoint失败
Flink 数据偶尔数据积压导致checkpoint失败
2022-06-30 08:27:00 【//承续缘_纪录片】
一、问题描述
Flink 程序测试成功,运行一段时间后,数据偶尔数据积压导致checkpoint失败
二、问题原因
可能原因,内存磁盘等承载能力小,数据写入延迟
三、解决方式(暂时)
1.增加checkpoint创建时长,减少写入次数(.enableCheckpointing)
2.增加允许失败次数(.setTolerableCheckpointFailureNumber)
env.setStateBackend(new FsStateBackend(FSSTATEBACKEND));
env.enableCheckpointing(10000);// 每 ** ms 开始一次 checkpoint
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);// 设置模式为精确一次
env.getCheckpointConfig().setCheckpointTimeout(100000);// Checkpoint 必须在** ms内完成,否则就会被抛弃
env.getCheckpointConfig().setMaxConcurrentCheckpoints(2);// 同一时间只允许一个 checkpoint 进行
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(3000);// 确认 checkpoints 之间的时间会进行 ** ms
env.getCheckpointConfig().setTolerableCheckpointFailureNumber(5);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.of(10,TimeUnit.SECONDS)));//重启策略:重启3次,间隔10s



边栏推荐
- Redis design and Implementation (II) | database (deletion strategy & expiration elimination strategy)
- 一次cpu 跌底排查
- [untitled]
- Experiment 3 remote control
- Unity basic lighting model
- Opencv4.2.0+vs2015 configuration
- 自制GIF动态图-gifcam
- TP5 set direct download file
- Emoji icons supported by markdown
- How to handle the expired data of redis and what are the elimination mechanisms?
猜你喜欢

Redis设计与实现(一)| 数据结构 & 对象

Redis设计与实现(四)| 主从复制

F12 packet capture is used for the whole process analysis of postman interface test

【NVMe2.0b 14-6】Format NVM、Keep Alive、Lockdown command

How to handle the expired data of redis and what are the elimination mechanisms?

Graffiti Wi Fi & ble SoC development slide strip

codeforces每日5题(均1700)-第三天

Redis设计与实现(七)| 发布 & 订阅
![[nvme2.0b 14-8] set features (Part 2)](/img/fe/67fd4f935237f9aa835e132e696b98.png)
[nvme2.0b 14-8] set features (Part 2)

Sword finger offer II 076 The kth largest number in the array (use heap to solve TOPK problem)
随机推荐
【NVMe2.0b 14】NVMe Admin Command Set
Redis设计与实现(四)| 主从复制
Redis design and Implementation (V) | sentinel sentry
[JUC series] overview of fork/join framework
领域驱动下cloud项目中单个服务的示例
Redis design and Implementation (II) | database (deletion strategy & expiration elimination strategy)
Sword finger offer II 075 Array relative sort (custom sort, count sort)
C# Console. Writeline() function output format
Want to ask, how to choose securities companies for stock speculation? Is it safe to open an account online?
Gilbert Strang's course notes on linear algebra - Lesson 4
This point in JS
自制GIF动态图-gifcam
[flower carving experience] 13 build the platformio ide development environment of esp32c3
Dart tips
Common tools installation, configuration, compilation, link, etc
Redis design and Implementation (VI) | cluster (sharding)
Experiment 2 LED button PWM 2021/11/22
【NVMe2.0b 14-6】Format NVM、Keep Alive、Lockdown command
Be careful of this hole in transmittable thread local
微信公众号第三方平台开发,零基础入门。想学我教你啊