The first day of Flink learning - what is batch and streaming computing?

• batch computing

  • Batch processing: process a certain amount of data, similar to moving bricks, 10 by 10
  • Scenario: offline data statistics, report analysis, etc. (1 trillion logs in the past year, analysis day, week, month, interface response delay status code)
  • Features: Batch calculation non-real-time, high latency, the result can be obtained after the calculation is completed
  • Framework: Hadoop MapReduce
• stream computing
  • Framework: Spark (macro), Flink
  • Scenarios: real-time monitoring, real-time risk control, etc.
  • Features: Streaming real-time, low latency, real-time get the latest results
  • Stream processing: process the continuous stream of data, similar to the water coming out of a faucet

• Distinguish (offline computing and real-time computing + streaming computing and batch computing)

  • Offline computing and real-time computing: the [delay] of data processing is different (one real-time and one non-real-time)
  • Streaming computing and batch computing: The [methods] of data processing are different (one stream and one batch)
  • Conclusion: Offline and batch are not equivalent, real-time and streaming are not equivalent, because they are not of the same dimension