[various * * question series] what are OLTP and OLAP?

OLTP and OLAP What is it? ？

OLTP On line transaction processing ：

        OLTP, Also called online transaction processing （Online Transaction Processing）, Represents a highly transactional system , It's generally a highly available online system , Mainly small transactions and small queries , When evaluating its system , It's generally seen that it's executed every second Transaction as well as Execute SQL The number of . In such a system , A single database processes Transaction Often more than a few hundred , Or thousands of them ,Select The execution amount of statements is thousands or even tens of thousands per second . Typical OLTP The system has an e-commerce system 、 Bank 、 Securities, etc , Such as the United States eBay Business database , It's a typical OLTP database . Business scenarios such as bank transfer and other real-time scenarios .

OLAP On line analytical processing ：

        OLAP, It's called OLAP (On-Line Analytical Processing), yes Data warehouse System (HBase、ClickHouse…) The main application of , Support complex statistical analysis operations on massive data , Persistent data is generally not modified , Data consistency requirements are not high , Focus on decision support , And provide intuitive and easy to understand query results , For example, the mall recommendation system , User portrait . Sometimes it's called DSS Decision support system , That's what we call data warehouse . In such a system , The execution amount of the statement is not the evaluation standard , Because the execution time of a statement may be very long , There's also a lot of data to read . therefore , In such a system , The evaluation standard is often the throughput of the disk subsystem （ bandwidth ）, If so, how much MB/s Of traffic . Business scenarios such as product recommendation , For example, common in e-commerce scenarios , Recommend products according to user behavior . After the user's behavior data is stored in the data warehouse , Do real-time calculations , Then send the recommendation results calculated by the algorithm model to the business end for display .

---

OLTP and OLAP comparative analysis ：

OLTP, On line transaction processing （Online Transaction Processing）

         Represents a highly transactional system , It's generally a highly available online system , Mainly small transactions and small queries , When evaluating its system , It's generally seen that it's executed every second Transaction as well as Execute SQL The number of . In such a system , A single database processes Transaction Often more than a few hundred , Or thousands of them ,Select The execution amount of statements is thousands or even tens of thousands per second . Typical OLTP The system has an e-commerce system 、 Bank 、 Securities, etc , Such as the United States eBay Business database , It's a typical OLTP database .OLTP The most common bottleneck in the system is CPU And disk subsystem .

• CPU Bottlenecks often appear in the total amount of logical reads and computational functions or processes , The total number of logical reads is equal to the logical reads of a single statement multiplied by the number of executions , If the execution speed of a single statement is very fast , But there are a lot of executions , that , It may also result in a large amount of logical reads . The method of design and optimization is to reduce the logical reading of a single statement , Or reduce their execution times . in addition , Some computational functions , Such as custom functions 、decode And so on , It will also consume a lot of CPU Time , Cause the load of the system to rise , Correct design method or optimization method , The calculation process needs to be avoided as much as possible , For example, saving the calculation results to the statistical table is a good method .
• The disk subsystem is OLTP Environment , Its carrying capacity generally depends on its IOPS processing capacity . Because in OLTP Environment , Disk physical reading is generally db file sequential read, That is, single reading , But it's read a lot . If it is frequently to the disk subsystem, it cannot carry its IOPS When , There will be big performance problems .

    OLTP The commonly used design and optimization methods are Cache Technology and B-tree Indexing technology ,Cache It determines that many statements do not need to obtain data from the disk subsystem , therefore ,Web cache And Oracle data buffer Yes OLTP System is very important . in addition , In terms of index use , The simpler the sentence, the better , In this way, the implementation plan is also stable , And be sure to use bound variables , Reduce statement parsing , Minimize table associations , Minimize distributed transactions , Basically do not use Partition Technology 、MV technology 、 Parallel technology and bitmap index . Because of the high concurrency , Batch updates should be submitted quickly in batches , To avoid blocking . 
OLTP The system is a data block that changes very frequently ,SQL System with very frequent statement submission . For data blocks , Try to keep data blocks in memory as much as possible , about SQL Come on , Use variable binding techniques as much as possible to achieve SQL reusing , Reduce physics I/O And repeated SQL analysis , So as to greatly improve the performance of the database .
    This affects performance in addition to binding variables , It may also be hot fast （hot block）. When a block is read by multiple users at the same time ,Oracle To maintain data consistency , Need to use Latch To serialize user actions . When a user gets latch after , Other users can only wait , The more users get this block of data , The more obvious the wait is . This is the problem of fast heating . This hot speed may be data blocks , It may also be rollback endblocks . For data blocks , It is usually caused by the uneven distribution of data in the database , If it is an indexed data block , Consider creating a reverse index to redistribute data , For rollback segment data blocks , You can add a few more rollback segments to avoid this kind of contention . 

OLAP, On line analytical processing （Online Analytical Processing） System

         Sometimes it's called DSS Decision support system , That's what we said Data warehouse . In such a system , The execution amount of the statement is not the evaluation standard , Because the execution time of a statement may be very long , There's also a lot of data to read . therefore , In such a system , The evaluation standard is often the throughput of the disk subsystem （ bandwidth ）, If so, how much MB/s Of traffic .
          The throughput of disk subsystem often depends on the number of disks , This is the time ,Cache Basically, it has no effect , The reading and writing types of the database are basically db file scattered read And direct path read/write. Try to use a large number of disks and a large bandwidth , Such as 4Gb Optical fiber interface .
stay OLAP In the system , Partition technology is often used 、 Parallel technology .
    Partition technology in OLAP The importance of the system is mainly reflected in the database management , For example, database loading , It can be realized by partition switching , Backup can be achieved by backing up partitioned table spaces , Delete data can be deleted by partition , As for the performance impact of partitioning , It can make the scanning of some large tables very fast （ Scan only a single partition ）. in addition , If partitions are combined with parallelism , It can also make the scanning of the whole table very fast . All in all , The main function of partition is the convenience of Management , It does not absolutely guarantee the improvement of query performance , Sometimes partitioning can lead to performance improvements , Sometimes it goes down .
    In addition to the combination of parallel technology and partition technology , stay Oracle 10g in , And RAC Combine to realize the simultaneous scanning of multiple nodes , The effect is also very good , Can put a task , Such as select Full scan table , Distribute evenly to multiple RAC On the nodes of .
    stay OLAP In the system , No binding required （BIND） Variable , Because the execution amount of the whole system is very small , Analysis time for execution time , You can ignore , And avoid the wrong execution plan . however OLAP Bitmap indexes can be used extensively in , Materialized view , For large transactions , Try to optimize the speed , There's no need to be like OLTP Request quick submission , Even deliberately slow down the speed of execution .
    The real use of bound variables is in OLTP In the system , This system usually has such characteristics , The number of concurrent users is very large , User requests are very intensive , And these requests SQL Most are reusable .
    about OLAP system , Most of the time, report jobs are running on the database , Execution is basically aggregated SQL operation , such as group by, Now , Set the optimizer mode to all_rows It's appropriate . For some website databases with more paging operations , Set to first_rows It will be better . But sometimes for OLAP System , When we have pagination again , We can consider in every SQL of use hint. Such as ：Select  a.* from table a;