当前位置:网站首页>Occasionally, Flink data is overstocked, resulting in checkpoint failure
Occasionally, Flink data is overstocked, resulting in checkpoint failure
2022-06-30 08:37:00 【//Continuous margin_ documentary】
One 、 Problem description
Flink Program test succeeded , After running for a while , Data occasionally leads to data backlog checkpoint Failure
Two 、 Question why
Possible causes , Memory and disk have small carrying capacity , Data write delay
3、 ... and 、 Solution ( temporary )
1. increase checkpoint Creation time , Reduce writes (.enableCheckpointing)
2. Increase the number of failures allowed (.setTolerableCheckpointFailureNumber)
env.setStateBackend(new FsStateBackend(FSSTATEBACKEND));
env.enableCheckpointing(10000);// Every time ** ms Start once checkpoint
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);// Set the mode to precise once
env.getCheckpointConfig().setCheckpointTimeout(100000);// Checkpoint Must be in ** ms Finish in , Otherwise, they will be abandoned
env.getCheckpointConfig().setMaxConcurrentCheckpoints(2);// Only one is allowed at a time checkpoint Conduct
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(3000);// confirm checkpoints The time between will be ** ms
env.getCheckpointConfig().setTolerableCheckpointFailureNumber(5);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.of(10,TimeUnit.SECONDS)));// Restart strategy : restart 3 Time , interval 10s
边栏推荐
- VIM from dislike to dependence (21) -- cross file search
- [data analysis and display]
- 文件上传 upload 组件 on-success 事件,添加自定义参数
- 小心transmittable-thread-local的这个坑
- How can we get a satisfactory salary? These routines still need to be mastered
- Cesium learning notes (I)
- [nvme2.0b 14-8] set features (Part 2)
- Redis design and Implementation (II) | database (deletion strategy & expiration elimination strategy)
- 【NVMe2.0b 14-4】Directive Send/Receive command
- [nvme2.0b 14 - 5] commande de téléchargement / commande du logiciel
猜你喜欢
Flink Sql -- toAppendStream doesn‘t support consuming update and delete changes which
【NVMe2.0b 14-7】Set Features(上篇)
Redis设计与实现(七)| 发布 & 订阅
基于Svelte3.x桌面端UI组件库Svelte UI
微信公众号第三方平台开发,零基础入门。想学我教你啊
Gilbert Strang's course notes on linear algebra - Lesson 3
A troubleshooting of CPU bottom falling
【NVMe2.0b 14-5】Firmware Download/Commit command
Gilbert Strang's course notes on linear algebra - Lesson 2
Sword finger offer II 076 The kth largest number in the array (use heap to solve TOPK problem)
随机推荐
MIME type Encyclopedia
vite項目require語法兼容問題解决require is not defined
国债逆回购绝对安全吗 网上怎么开户
[nvme2.0b 14 - 5] commande de téléchargement / commande du logiciel
Axure make menu bar effect
MIME类型大全
Sword finger offer II 074 Merge interval (sort, array)
Axure制作菜单栏效果
Mmcv expanding CUDA operator beginner level chapter
127.0.0.1、0.0.0.0和localhost
File upload component on success event, add custom parameters
Cesium learning notes (IV) visual image & Terrain
示波器探头对测量电容负荷有影响吗?
Summary of common pytoch APIs
Common tools installation, configuration, compilation, link, etc
codeforces每日5题(均1700)-第三天
[untitled]
Leetcode47. full arrangement II
Unity 基础光照模型
Flink SQL 自定义 Connector