当前位置:网站首页>Flink 数据偶尔数据积压导致checkpoint失败
Flink 数据偶尔数据积压导致checkpoint失败
2022-06-30 08:27:00 【//承续缘_纪录片】
一、问题描述
Flink 程序测试成功,运行一段时间后,数据偶尔数据积压导致checkpoint失败
二、问题原因
可能原因,内存磁盘等承载能力小,数据写入延迟
三、解决方式(暂时)
1.增加checkpoint创建时长,减少写入次数(.enableCheckpointing)
2.增加允许失败次数(.setTolerableCheckpointFailureNumber)
env.setStateBackend(new FsStateBackend(FSSTATEBACKEND));
env.enableCheckpointing(10000);// 每 ** ms 开始一次 checkpoint
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);// 设置模式为精确一次
env.getCheckpointConfig().setCheckpointTimeout(100000);// Checkpoint 必须在** ms内完成,否则就会被抛弃
env.getCheckpointConfig().setMaxConcurrentCheckpoints(2);// 同一时间只允许一个 checkpoint 进行
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(3000);// 确认 checkpoints 之间的时间会进行 ** ms
env.getCheckpointConfig().setTolerableCheckpointFailureNumber(5);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.of(10,TimeUnit.SECONDS)));//重启策略:重启3次,间隔10s



边栏推荐
- Cesium learning notes (I)
- MIME类型大全
- 微信公众号第三方平台开发,零基础入门。想学我教你啊
- Dlib database face
- Is it difficult to jump job ByteDance? With these skills, you can easily pass
- 1162 Postfix Expression
- Build a docker image of Henkel database from 0
- Cesium learning notes (IV) visual image & Terrain
- Redis设计与实现(八)| 事务
- Transformer architecture understanding
猜你喜欢

Axure制作菜单栏效果

What management improvements can CRM bring to enterprises

【NVMe2.0b 14】NVMe Admin Command Set

Build a docker image of Henkel database from 0
![[flower carving experience] 12 build the Arduino development environment of esp32c3](/img/76/a66e6d5c62d25067841b47eb01b718.jpg)
[flower carving experience] 12 build the Arduino development environment of esp32c3

Wechat official account third-party platform development, zero foundation entry. I want to teach you

Redis设计与实现(八)| 事务
![[nvme2.0b 14 - 5] commande de téléchargement / commande du logiciel](/img/98/bb250030f3e31e71d48e3be3ea4698.png)
[nvme2.0b 14 - 5] commande de téléchargement / commande du logiciel

Redis design and Implementation (VII) | publish & subscribe

微信小程序使用vant weapp报错
随机推荐
Opencv video
【NVMe2.0b 14-6】Format NVM、Keep Alive、Lockdown command
1163 Dijkstra Sequence
领域驱动下cloud项目中单个服务的示例
Build a docker image of Henkel database from 0
End-to-end 3D Point Cloud Instance Segmentation without Detection
微信小程序使用vant weapp报错
Introduction to opencv (I): image reading and display
Wechat official account third-party platform development, zero foundation entry. I want to teach you
[flower carving experience] 12 build the Arduino development environment of esp32c3
el-input 限制只能输数字
Redis设计与实现(八)| 事务
2021-04-29
微信公众号第三方平台开发,零基础入门。想学我教你啊
Gilbert Strang's course notes on linear algebra - Lesson 4
Redis设计与实现(七)| 发布 & 订阅
【NVMe2.0b 14】NVMe Admin Command Set
Does the oscilloscope probe affect the measurement of capacitive load?
PHP API to obtain QR code and combine to generate pictures
layer. Open processing method when the passed value is an array or the value is too long