当前位置:网站首页>Flink 数据偶尔数据积压导致checkpoint失败
Flink 数据偶尔数据积压导致checkpoint失败
2022-06-30 08:27:00 【//承续缘_纪录片】
一、问题描述
Flink 程序测试成功,运行一段时间后,数据偶尔数据积压导致checkpoint失败
二、问题原因
可能原因,内存磁盘等承载能力小,数据写入延迟
三、解决方式(暂时)
1.增加checkpoint创建时长,减少写入次数(.enableCheckpointing)
2.增加允许失败次数(.setTolerableCheckpointFailureNumber)
env.setStateBackend(new FsStateBackend(FSSTATEBACKEND));
env.enableCheckpointing(10000);// 每 ** ms 开始一次 checkpoint
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);// 设置模式为精确一次
env.getCheckpointConfig().setCheckpointTimeout(100000);// Checkpoint 必须在** ms内完成,否则就会被抛弃
env.getCheckpointConfig().setMaxConcurrentCheckpoints(2);// 同一时间只允许一个 checkpoint 进行
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(3000);// 确认 checkpoints 之间的时间会进行 ** ms
env.getCheckpointConfig().setTolerableCheckpointFailureNumber(5);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.of(10,TimeUnit.SECONDS)));//重启策略:重启3次,间隔10s



边栏推荐
- 【NVMe2.0b 14-7】Set Features(上篇)
- Tidb v6.0.0 (DMR): initial test of cache table - tidb Book rush
- php api获取二维码、组合生成图片
- A troubleshooting of CPU bottom falling
- Using typera+picgo to realize automatic uploading of markdown document pictures
- Redis design and Implementation (IV) | master-slave replication
- Common tools installation, configuration, compilation, link, etc
- Gilbert Strang's course notes on linear algebra - Lesson 2
- 【NVMe2.0b 14-1】Abort、Asynchronous Event Request、Capacity Management command
- Transformer architecture understanding
猜你喜欢

【NVMe2.0b 14-5】Firmware Download/Commit command

【kotlin 协程】万字协程 一篇完成kotlin 协程进阶

Sword finger offer II 076 The kth largest number in the array (use heap to solve TOPK problem)

Tidb 6.0: making Tso more efficient tidb Book rush

How to handle the expired data of redis and what are the elimination mechanisms?

Cesium learning notes (IV) visual image & Terrain

Axure制作菜单栏效果

Vulfocus entry target
![[flower carving experience] 13 build the platformio ide development environment of esp32c3](/img/32/2c30afe77bf82774479a671ff16898.jpg)
[flower carving experience] 13 build the platformio ide development environment of esp32c3

Redis design and Implementation (VI) | cluster (sharding)
随机推荐
Transformer architecture understanding
Wechat official account third-party platform development, zero foundation entry. I want to teach you
2021-04-29
Redis设计与实现(一)| 数据结构 & 对象
Unit Test
Map,String,Json之間轉換
【NVMe2.0b 14-1】Abort、Asynchronous Event Request、Capacity Management command
Redis design and Implementation (III) | interaction between server and client (event IO model)
Is it difficult to jump job ByteDance? With these skills, you can easily pass
Development technology sharing of Jingtan NFT digital collection system
【NVMe2.0b 14-4】Directive Send/Receive command
【NVMe2.0b 14-7】Set Features(上篇)
【NVMe2.0b 14-6】Format NVM、Keep Alive、Lockdown command
Sword finger offer II 074 Merge interval (sort, array)
电流探头的干扰源电流谱测试
vite项目require语法兼容问题解决require is not defined
An example of a single service in a cloud project driven by a domain
2021-02-22
Unity简单shader
【NVMe2.0b 14-3】Doorbell Buffer Config command、Device Self-test command