Classmate, have you heard of mot?

This paper introduces openGauss Memory optimization table （Memory-Optimized Table,MOT） Characteristics and value of 、 Application scenarios and deployment .

MOT brief introduction

openGauss Introduced MOT Storage engine , It is a transactional row store , Optimized for multi-core and large memory servers .MOT yes openGauss The most advanced production level features of the database （Beta edition ）, It provides higher performance for transactional workloads .MOT Fully support ACID characteristic , And includes strict persistence and high availability support . Enterprises can be on mission critical 、 Performance sensitive online transaction processing （OLTP） Use in MOT, To achieve high performance 、 High throughput 、 Predictable low latency and high utilization of multi-core servers .MOT It is especially suitable for running on modern servers with multi-channel and multi-core processors , For example, based on Arm/ Huawei of Kunpeng processor TaiShan The server , And based on x86 Dell or similar server .

chart 1 openGauss Memory optimized storage engine

Such as chart 1 Shown ,openGauss Database memory optimization storage engine components （ The green part ） Responsible for managing the MOT And transaction .

MOT Create side-by-side with disk based normal tables .MOT The effective design of realizes almost complete SQL Cover , And support a complete set of database functions , Such as stored procedures and custom functions （ Restrictions see MOT SQL Coverage and limitations ）.

Through data and indexes completely stored in memory 、 Non uniform memory access awareness （NUMA-aware） Design 、 Algorithm for eliminating lock and latch contention and query native compilation ,MOT Provide faster and more efficient data access .

MOT Effective almost lock free design and highly tuned implementation , This enables it to achieve excellent near linear throughput expansion on multi-core servers , This is probably the best in the industry .

MOT Fully support ACID characteristic ：

• Atomicity （Atomicity）： Atomic transactions are a series of inseparable database operations . After the transaction is completed （ Submit or suspend... Separately ） after , These operations either all happen , Or it doesn't happen at all .
• Uniformity （Consistency）： When the business is over , The database is in a consistent state , Preserve data integrity .
• Isolation, （Isolation）： Transactions cannot interfere with each other .MOT Support different isolation levels of repeated read and read commit . In the next version ,MOT Serializable isolation will also be supported . For more information , Please see the MOT Isolation level .
• persistence （Durability）： Even if there are crashes and failures , Successfully completed （ Submit ） The transaction effect of is persistent .MOT Fully integrated openGauss Based on the WAL Logging of . Both synchronous and asynchronous logging options are supported .MOT It also supports synchronization + oriented NUMA Optimized group commit . For more information , Please see the MOT The concept of persistence .

MOT Application scenarios

MOT According to the characteristics of the load , Significantly accelerate overall application performance .MOT By improving the efficiency of data access and transaction execution , And by eliminating the lock and latch contention between concurrent transactions , Minimize redirection , This improves the performance of transaction processing .

MOT It's not just because it's in memory , Also because it is optimized around concurrent memory usage management . data storage 、 Access and processing algorithms are designed from scratch , To take advantage of the latest advanced technology of memory and high concurrent computing .

openGauss Allow applications to combine at will MOT And standard disk based tables . For enabling the most active that has proven to be a bottleneck 、 High contention and performance sensitive application tables , And tables that require predictable low latency access and high throughput ,MOT Particularly useful .

MOT Can be used in a variety of applications , for example ：

• High throughput transaction processing ： This is the use of MOT Main scenes of , Because it supports massive transactions , At the same time, the delay of a single transaction is required to be low . Examples of such applications are real-time decision systems 、 Payment system 、 Financial instrument transactions 、 Sports betting 、 Mobile games 、 Advertising, etc .
• Performance bottlenecks accelerate ： Tables with high contention can be used MOT benefit , Even if the table is a disk table . Due to lower latency 、 Less competition and locks and increased server throughput , Such table （ In addition to related tables and tables referenced together in queries and transactions ） The conversion of makes the performance significantly improved .
• Eliminate mid tier cache ： Cloud computing and mobile applications tend to have periodic or peak workloads . Besides , Many applications have 80% The above loads are read loads , Accompanied by frequent repeated queries . In order to meet the individual requirements of peak load , And reduce response latency to provide the best user experience , Applications often deploy an intermediate cache layer . Such additional layers increase the complexity and time of development , It also increases operating costs .MOT Provides a good alternative , Simplify application architecture through consistent high-performance data storage , Shorten development cycle , Reduce CAPEX and OPEX cost .
• Large scale stream data extraction ：MOT It can meet the needs of the cloud （ For mobile 、M2M And the Internet of things ）、 Transaction processing （Transactional Processing,TP）、 Analyze and process （Analytical Processing,AP） And machine learning （Machine Learning,ML） The extraction requirements of large-scale stream data .MOT Especially good at extracting large amounts of data from many different sources at the same time . These data can be processed later 、 transformation , And move in slower disk based tables . in addition ,MOT You can also find consistent 、 The latest data , To get real-time results . In the Internet of things and cloud computing applications with many real-time data streams , Usually there will be special data acquisition and processing . for example , One Apache Kafka Clusters can be used to extract 10 Ten thousand Events / Second data , Delay for 10ms. A periodic batch task will collect the collected data , And convert the format , Put it into a relational database for further analysis .MOT You can store the data stream directly in MOT Relationship table , Prepare for analysis and decision making , To support such scenarios （ At the same time, separate data processing layers are eliminated ）. This allows faster data collection and processing ,MOT Avoid costly layering and slow batch processing , Improved consistency , It increases the real-time performance of analyzing data , At the same time, it reduces the total cost of ownership （Total Cost of Ownership,TCO）.
• Reduce TCO： Improving resource utilization and eliminating the middle tier can save 30% To 90% Of TCO. Friendly business cases ：MemSQL、Azure.

Deploy MOT

The following sections describe the various required and optional settings , For optimal deployment .

MOT Server optimization ：x86

Usually , The database is bound by the following components ：

• CPU： Faster CPU Can speed up any CPU Bound database .
• disk ： High speed SSD/NVME Can accelerate any I/O Bind database .
• The Internet ： Faster networks can speed up anything SQL*Net Bind database .

In addition to the above , The following general server settings are used by default , It may significantly affect the performance of the database .

MOT Performance tuning is a key step in ensuring fast application functionality and data retrieval .MOT Support the latest hardware , Therefore, it is very important to adjust each system to achieve maximum throughput .

The following is used to optimize in Intel x86 Running on the server MOT Recommended configuration for . These settings are the best choice for high throughput workloads .

BIOS

• Hyper Threading Set to ON.

  It is strongly recommended to turn on hyper threading （HT=ON）.

  It is suggested that MOT Up operation OLTP Open hyperthread when workload . When using hyper threading , some OLTP The workload shows up to 40% The performance gain of .

Operating system environment settings

• NUMA

  Ban NUMA Balance , As shown below .MOT With extremely efficient NUMA-aware Memory management , Far more than the default method used by the operating system .

  echo 0 > /proc/sys/kernel/numa_balancing

• service

  Disable the following services ：

  service irqbalance stop           # MANADATORY 
  service sysmonitor stop           # OPTIONAL, performance  
  service rsyslog stop       # OPTIONAL, performance

• Tuning Services

  The following items are required .

  The server must be running throughput-performance The configuration file .

  [...]$ tuned-adm profile throughput-performance 

  
  throughput-performance Configuration files are widely applicable , It provides superior performance for a variety of common server workloads .

  Others are not suitable for openGauss and MOT The configuration of the server may affect MOT Overall performance of , Include ： Balanced configuration 、 Desktop configuration 、 Delay performance configuration 、 Network delay configuration 、 Network throughput configuration and energy saving configuration .

• System commands

  The following operating system settings are recommended for optimal performance .

  • stay /etc/sysctl.conf Add the following configuration to the file , And then execute sysctl -p command ：

    net.ipv4.ip_local_port_range = 9000 65535 
    kernel.sysrq = 1 
    kernel.panic_on_oops = 1 
    kernel.panic = 5 
    kernel.hung_task_timeout_secs = 3600 
    kernel.hung_task_panic = 1 
    vm.oom_dump_tasks = 1 
    kernel.softlockup_panic = 1 
    fs.file-max = 640000 
    kernel.msgmnb = 7000000 
    kernel.sched_min_granularity_ns = 10000000 
    kernel.sched_wakeup_granularity_ns = 15000000 
    kernel.numa_balancing=0 
    vm.max_map_count = 1048576 
    net.ipv4.tcp_max_tw_buckets = 10000 
    net.ipv4.tcp_tw_reuse = 1 
    net.ipv4.tcp_tw_recycle = 1 
    net.ipv4.tcp_keepalive_time = 30 
    net.ipv4.tcp_keepalive_probes = 9 
    net.ipv4.tcp_keepalive_intvl = 30 
    net.ipv4.tcp_retries2 = 80 
    kernel.sem = 250 6400000 1000 25600 
    net.core.wmem_max = 21299200 
    net.core.rmem_max = 21299200 
    net.core.wmem_default = 21299200 
    net.core.rmem_default = 21299200 
    #net.sctp.sctp_mem = 94500000 915000000 927000000 
    #net.sctp.sctp_rmem = 8192 250000 16777216 
    #net.sctp.sctp_wmem = 8192 250000 16777216 
    net.ipv4.tcp_rmem = 8192 250000 16777216 
    net.ipv4.tcp_wmem = 8192 250000 16777216 
    net.core.somaxconn = 65535 
    vm.min_free_kbytes = 26351629 
    net.core.netdev_max_backlog = 65535 
    net.ipv4.tcp_max_syn_backlog = 65535 
    #net.sctp.addip_enable = 0 
    net.ipv4.tcp_syncookies = 1 
    vm.overcommit_memory = 0 
    net.ipv4.tcp_retries1 = 5 
    net.ipv4.tcp_syn_retries = 5
    

  • Modify as follows /etc/security/limits.conf The corresponding part ：

    <user> soft nofile 100000 
    <user> hard nofile 100000

    Soft limit and hard limit settings can specify the number of files opened by a process at the same time . Soft limits can be changed by the processes running them , Until the hard limit is reached .

• disk /SSD

  Let's take the database synchronization submission mode as an example , This paper introduces how to ensure disk read-write performance, which is suitable for database synchronous submission mode .

  Run the disk as follows /SSD Performance testing ：

  [...]$ sync; dd if=/dev/zero of=testfile bs=1M count=1024; sync 
  1024+0 records in 
  1024+0 records out 
  1073741824 bytes (1.1 GB) copied, 1.36034 s, 789 MB/s 

  When the disk bandwidth is significantly lower than 789MB/s when , It may cause openGauss Performance bottleneck , Especially causing MOT Performance bottleneck .

The Internet

Need to use 10Gbps The above networks .

function iperf Command to verify ：

Server side: iperf -s 
Client side: iperf -c <IP>

rc.local： Network card tuning

The following optional settings have a significant impact on performance ：

1. take set_irq_affinity.sh · GitHub Under the set_irq_privacy.sh File copy to /var/scripts/ Under the table of contents .

2. Get into /etc/rc.d/rc.local, perform chmod command , Make sure that the boot Execute the following script ：

   'chmod +x /etc/rc.d/rc.local'  
   var/scripts/set_irq_affinity.sh -x all <DEVNAME> 
   ethtool -K <DEVNAME> gro off 
   ethtool -C <DEVNAME> adaptive-rx on adaptive-tx on 
   Replace <DEVNAME> with the network card, i.e. ens5f1

MOT Server optimization ： be based on Arm Huawei TaiShan2P/4P The server

The following is based on Arm/ Huawei based on Kunpeng architecture TaiShan 2280 v2 The server （2 road 128 nucleus ） and TaiShan 2480 v2 The server （4 road 256 nucleus ） Up operation MOT Recommended configuration for .

Unless otherwise stated , The following settings apply to both client and server machines .

BIOS

modify BIOS Related settings ：

1. choice  BIOS > Advanced > MISC Config. Set up  Support Smmu  by  Disabled.

2. choice  BIOS > Advanced > MISC Config. Set up  CPU Prefetching Configuration  by  Disabled.

   

3. choice  BIOS > Advanced > Memory Config. Set up  Die Interleaving  by  Disabled .

   

4. choice  BIOS > Advanced > Performance Config. Set up  Power Policy  by  Performance.

   

operating system ： Kernel and boot

• The following operating system kernel and boot parameters are usually determined by sysadmin To configure .

  Configure kernel parameters , As shown below .

  net.ipv4.ip_local_port_range = 9000 65535 
  kernel.sysrq = 1 
  kernel.panic_on_oops = 1 
  kernel.panic = 5 
  kernel.hung_task_timeout_secs = 3600 
  kernel.hung_task_panic = 1 
  vm.oom_dump_tasks = 1 
  kernel.softlockup_panic = 1 
  fs.file-max = 640000 
  kernel.msgmnb = 7000000 
  kernel.sched_min_granularity_ns = 10000000 
  kernel.sched_wakeup_granularity_ns = 15000000 
  kernel.numa_balancing=0 
  vm.max_map_count = 1048576 
  net.ipv4.tcp_max_tw_buckets = 10000 
  net.ipv4.tcp_tw_reuse = 1 
  net.ipv4.tcp_tw_recycle = 1 
  net.ipv4.tcp_keepalive_time = 30 
  net.ipv4.tcp_keepalive_probes = 9 
  net.ipv4.tcp_keepalive_intvl = 30 
  net.ipv4.tcp_retries2 = 80 
  kernel.sem = 32000 1024000000      500     32000 
  kernel.shmall = 52805669 
  kernel.shmmax = 18446744073692774399 
  sys.fs.file-max = 6536438 
  net.core.wmem_max = 21299200 
  net.core.rmem_max = 21299200 
  net.core.wmem_default = 21299200 
  net.core.rmem_default = 21299200 
  net.ipv4.tcp_rmem = 8192 250000 16777216 
  net.ipv4.tcp_wmem = 8192 250000 16777216 
  net.core.somaxconn = 65535 
  vm.min_free_kbytes = 5270325 
  net.core.netdev_max_backlog = 65535 
  net.ipv4.tcp_max_syn_backlog = 65535 
  net.ipv4.tcp_syncookies = 1 
  vm.overcommit_memory = 0 
  net.ipv4.tcp_retries1 = 5 
  net.ipv4.tcp_syn_retries = 5 
  ##NEW 
  kernel.sched_autogroup_enabled=0 
  kernel.sched_min_granularity_ns=2000000 
  kernel.sched_latency_ns=10000000 
  kernel.sched_wakeup_granularity_ns=5000000 
  kernel.sched_migration_cost_ns=500000 
  vm.dirty_background_bytes=33554432 
  kernel.shmmax=21474836480 
  net.ipv4.tcp_timestamps = 0 
  net.ipv6.conf.all.disable_ipv6=1 
  net.ipv6.conf.default.disable_ipv6=1 
  net.ipv4.tcp_keepalive_time=600 
  net.ipv4.tcp_keepalive_probes=3 
  kernel.core_uses_pid=1
  

• Tuning Services

  The following items are required .

  The server must be running throughput-performance The configuration file ：

  [...]$ tuned-adm profile throughput-performance
  

  throughput-performance Configuration files are widely applicable , It provides superior performance for a variety of common server workloads .

  Others are not suitable for openGauss and MOT The configuration of the server may affect MOT Overall performance of , Include ： Balanced configuration 、 Desktop configuration 、 Delay performance configuration 、 Network delay configuration 、 Network throughput configuration and energy saving configuration .

• Start tuning

  Add... To the kernel startup parameters iommu.passthrough=1.

  stay pass-through When running in mode , The adapter requires DMA Convert to memory , To improve performance .

MOT To configure

preset MOT Used to create work MOT. For best results , It is recommended that you customize it according to the specific requirements and preferences of your application MOT To configure （ stay mot.conf The document defines ）.

This file is read-only when the server starts . If you edit this file while the system is running , The server must be reloaded for the changes to take effect .

mot.conf File with the postgresql.conf The configuration file is in the same folder .

In the active / standby deployment mode , The number of active and standby nodes mot.conf The files need to be exactly the same , otherwise , Unclear system behavior .

read General principles , View and configure as needed mot.conf file .

  explain ：  Described above mot.conf Various settings in the file . In addition to the above , To understand specific MOT function （ Such as recovery ）, Refer to the relevant sections of this user manual . for example ,MOT recovery Illustrates the mot.conf Recovery of files , Including the impact of MOT Restore settings . Besides , Full instructions for recovery , see also “MOT management ” Chapter MOT recovery . Reference links are also provided in the relevant sections below .

Here's how mot.conf Various parts of the document , It contains settings and default values .

General principles

Here is the editor mot.conf General principles of the document .

•   Each setting item has a default value , As shown below

  # name = value

• You can accept spaces or leave them blank . 

• Add... In each line # No. can be annotated .

• The default value of each setting item will be displayed as a comment in the whole file .

• If the parameter has no comment and a new value is placed , Define new settings .

• Yes mot.conf File changes take effect only when the database server is started or reinstalled .

The representation of memory unit is as follows ：

• KB： kilobytes
• MB： Megabytes
• GB： Gigabyte
• TB： Terabytes

If no memory unit is specified , It is assumed to be byte .

Some memory units are postgresql.conf Medium max_process_memory The percentage of . for example ,20%.

The time unit is expressed as follows ：

• us： Microsecond
• ms： millisecond
• s： second
• min： minute
• h： Hours
• d： God

If no time unit is specified , It is assumed to be microseconds .

Redo log （MOT）

• enable_group_commit = false

  Whether to use group submission .

  This option is only available in openGauss When configured to use synchronous submission , That is, only if postgresql.conf Medium synchronization_commit Set to except off Any value other than is relevant .

  of WAL Redo log details , see also MOT logging ：WAL Redo log .

• group_commit_size = 16

• group_commit_timeout = 10 ms

  Only when MOT When the engine is configured to synchronize group commit logging , This option is only relevant . namely postgresql.conf Medium synchronization_commit Configure to true,mot.conf In the configuration file enable_group_commit Configure to true.

  When a set of transactions is recorded in WAL When redo logs , The values of the following setting items need to be determined ：

  group_commit_size： Number of transactions committed in a group . for example ,16 When in the same group 16 When their transactions were committed by the client , Aimed at 16 Each of the transactions , On disk WAL Write an entry in the redo log .

  group_commit_timeout： Timeout time , The unit is millisecond . for example ,10 It means that 10 In milliseconds , For the same set by the client application in the recent 10 Every transaction committed in milliseconds , On disk WAL Write an entry in the redo log .

  The commit group closes after reaching the configured number of transactions or after a timeout . After the group is closed , All transactions in a group wait for a group to finish executing , Then notify the client that each transaction has ended .

  For more information about synchronization group commit logging , see also MOT Log type .

checkpoint （MOT）

• checkpoint_dir =

  Specify the directory where checkpoint data is stored . The default location is at the top of each data node data In the folder .

• checkpoint_segsize = 16 MB

  The segment size used when specifying checkpoints . Perform checkpoints in segments . When a segment is full , It will be serialized to disk , And open a new segment for subsequent checkpoint data .

• checkpoint_workers = 3

  Specifies the number of worker threads to use during checkpointing .

recovery （MOT）

• checkpoint_recovery_workers = 3

  Specifies the number of worker threads to use during checkpoint data recovery . Every MOT The engine worker thread runs on its own core , By reading different tables into memory , Different tables can be processed in parallel . The default value is 3, This parameter can be set to the number of cores that can be processed . After recovery , Will stop and kill these threads .

Statistics （MOT）

• enable_stats = false

  Set periodic statistical printing information .

• print_stats_period = 10 minute

  Print summary time range .

• print_full_stats_period = 1 hours

  Set the time range for printing the full statistics report .

  The following settings are for each part of the periodic statistical report . If not configured , Suppress statistics report .

• enable_log_recovery_stats = false

  Log recovery statistics include recovery indicators of various redo logs .

• enable_db_session_stats = false

  Database session statistics contain transaction events , If submitted 、 Roll back, etc .

• enable_network_stats = false

  Network statistics include connections / Disconnection event .

• enable_log_stats = false

  Log statistics include redo log details .

• enable_memory_stats = false

  Memory statistics include memory layer details .

• enable_process_stats = false

  Process statistics include the memory and memory of the current process CPU Total consumption .

• enable_system_stats = false

  System statistics include the memory and memory of the whole system CPU Total consumption .

• enable_jit_stats = false

  JIT Statistics include information about JIT Query compilation and execution information .

Error log （MOT）

• log_level = INFO

  Set up MOT The log level of the message sent by the engine recorded in the error log of the database server . Valid values are PANIC、ERROR、WARN、INFO、TRACE、DEBUG、DIAG1、DIAG2.

• Log/COMPONENT/LOGGER=LOG_LEVEL

  Use the following syntax to set up a specific logger .

  for example , For the system components ThreadIdPool Logger configuration TRACE The level of logging , Please use the following syntax ：

  Log.System.ThreadIdPool.log_level=TRACE

  To configure the log level for all recorders under a component , Please use the following syntax ：

  Log.COMPONENT.log_level=LOG_LEVEL

  for example ：

  Log.System.log_level=DEBUG

Memory （MOT）

• enable_numa = true

  Specifies whether to use recognizable NUMA Of memory . When disabled , All affinity configurations will also be disabled .MOT The engine assumes that all available NUMA Nodes have memory . If the computer has some special configuration , Some of them NUMA Node has no memory , be MOT Engine initialization will therefore fail , Therefore, the database server startup will fail . In such computers , It is recommended to set this configuration value to false, To prevent startup failure and let MOT The engine is not using recognizable NUMA Normal operation in the case of memory allocation .

• affinity_mode = fill-physical-first

  Set up user sessions and internal MOT Thread affinity mode of task .

  When using thread pool , The user session will ignore this value , Because their affinity is controlled by the thread pool . But inside MOT The task still uses .

  Valid values are fill-socket-first、equal-per-socket、fill-physical-first、none.

  • Fill-socket-first Connect the thread to the core of the same slot , Until the slot is full , Then move to the next slot .
  • Equal-per-socket Make threads evenly distributed in all slots .
  • Fill-physical-first Connect the thread to the physical core in the same slot , Until all physical cores are exhausted , Then move to the next slot . When all physical cores are exhausted , The process starts again with the hyper threading core .
  • None Disable any affinity configuration , And let the system scheduler determine which core each thread is scheduled to run on .

• lazy_load_chunk_directory = true

  Set block directory mode , For memory block lookup .

  Lazy Mode sets the block directory to load partial directories on demand , This reduces the initial memory footprint （ About from 1GB Reduced to 1MB）. However , This can lead to slight performance loss and memory corruption in extreme cases . contrary , Use non-lazy The block directory will be additionally allocated 1GB Initial memory , Produce slightly higher performance , And make sure to avoid block directory errors during memory corruption .

• reserve_memory_mode = virtual

  Set the memory reservation mode （ The value is physical or virtual）.

  Whenever memory is allocated from the kernel , Will refer to this configuration value to determine whether the allocated memory is resident （physical） Still very resident （virtual）. This is mainly related to pre allocation , But it can also affect runtime allocation . about physical Reserved mode , Page errors occur on all pages spanned by forcing memory regions , Make the whole allocated memory area resident . To configure virtual Memory reservation can speed up memory allocation （ Especially during pre allocation ）, However, a page error may occur during the initial access （ This results in a slight performance impact ）, And more server errors when physical memory is not available . contrary , Physical memory allocation is slow , But follow-up visits are faster and guaranteed .

• store_memory_policy = compact

  Set the memory storage policy （ The value is compact or expanding）.

  When defined compact strategy , Unused memory is released back into the kernel , Until the memory limit is reached （ See below min_mot_memory）. stay expanding Strategy , Unused memory is stored in MOT In the engine , For subsequent reuse .compact Storage strategies can reduce MOT Engine memory usage , But occasionally it will lead to a slight decline in performance . Besides , In case of memory corruption , It can also cause memory unavailability . contrary ,expanding Mode takes up more memory , But it allocates memory faster , And it can better ensure that the memory can be reallocated after deallocation .

• chunk_alloc_policy = auto

  Set the block allocation policy of global memory .

  MOT Memory to 2MB Organized in units of blocks . Source NUMA The memory layout of nodes and each block will affect the table data NUMA Distribution between nodes , Therefore, it has a great impact on data access time . In specific NUMA When allocating blocks on nodes , Will refer to the allocation strategy .

  Available values include auto、local、page-interleaved、chunk-interleaved、native.

  • Auto The strategy selects the block allocation strategy according to the current hardware situation .
  • Local Strategies in their respective NUMA Allocate each data block on the node .
  • Page-interleaved Strategy from all NUMA Nodes are allocated by interleaved memory 4 A data block consisting of thousands of bytes of pages .
  • Chunk-interleaved The policy is based on round robin scheduling from all NUMA The node allocates data blocks .
  • Native The policy allocates blocks by calling the native system memory allocator .

• chunk_prealloc_worker_count = 8

  Set each NUMA The number of worker threads that the node participates in memory pre allocation .

• max_mot_global_memory = 80%

  Set up MOT The maximum limit of the engine's global memory .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  MOT Engine memory is divided into global memory （ long-term ） Memory , It is mainly used to store user data , As well as the local （ short-term ） Memory , Mainly used for user sessions , To meet local needs .

  Any attempt to allocate memory beyond this limit will be rejected , And report the error to the user . Please make sure max_mot_global_memory And max_mot_local_memory The sum does not exceed postgresql.conf Configured in max_process_memory.

• min_mot_global_memory = 0 MB

  Set up MOT Minimum limit of engine global memory .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  This value is used for memory pre allocation during startup , And make sure MOT The engine has minimal memory available during normal operation . When using compact When storing policies （ See above store_memory_policy）, This value specifies the lower limit , Memory that exceeds the lower limit will not be released back to the kernel , But stay in MOT In the engine for subsequent reuse .

• max_mot_local_memory = 15%

  Set up MOT Maximum limit of engine local memory .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  MOT Engine memory is divided into global memory （ long-term ） Memory , It is mainly used to store user data , As well as the local （ short-term ） Memory , Mainly used for user sessions , To meet local needs .

  Any attempt to allocate memory beyond this limit will be rejected , And report the error to the user . Please make sure max_mot_global_memory And max_mot_local_memory The sum does not exceed postgresql.conf Configured in max_process_memory.

• min_mot_local_memory = 0 MB

  Set up MOT Minimum limit of engine local memory .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  This value is used to pre allocate memory during startup , And make sure MOT The engine has minimal available memory during normal operation . When using compact When storing policies （ See above store_memory_policy）, This value specifies the lower limit , Memory that exceeds the lower limit will not be released back to the kernel , But stay in MOT In the engine for subsequent reuse .

• max_mot_session_memory = 0 MB

  Set up MOT Maximum memory limit for a single session in the engine .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  Usually ,MOT Sessions in the engine can allocate as much local memory as needed , As long as the local memory limit is not exceeded . To avoid a single session taking up too much memory , Thus rejecting the memory of other sessions , This configuration item limits the local memory allocation of small sessions （ Maximum 1022KB）.

  Please ensure that this configuration item does not affect the local memory allocation of large sessions .

  0 Indicates that the local allocation of each small session will not be limited , Unless by max_mot_local_memory Memory allocation limitations caused by local configuration .

• min_mot_session_memory = 0 MB

  Set up MOT Minimum memory reservation for a single session in the engine .

  Specify the percentage value and postgresql.conf in max_process_memory The total amount defined is related to .

  This value is used to pre allocate memory during session creation , And ensure that the session has a minimum amount of available memory to perform its normal operation .

• session_large_buffer_store_size = 0 MB

  Set the session's large buffer storage .

  When a user session executes a query that requires a lot of memory （ for example , Use many lines ）, Large buffer storage is used to increase the level of certainty available for such memory , And serve this memory request faster . For more than 1022KB Conversation , Any memory allocation is a large memory allocation . If the large buffer storage is not used or exhausted , Then these allocations will be considered as huge allocations provided directly from the kernel .

• session_large_buffer_store_max_object_size = 0 MB

  Sets the maximum object size in the session's large allocation buffer store .

  The large buffer stores objects of different sizes . This value is used to set the upper limit on the objects stored in the source buffer , And determining the objects divided into different sizes in the buffer storage .

  This size cannot exceed session_large_buffer_store_size Of 1/8. If exceeded , Adjust it to the maximum possible .

• session_max_huge_object_size = 1 GB

  Sets the maximum size of a single large memory allocation for a session .

  Huge allocations are provided directly from the kernel , Therefore, there is no guarantee of success .

  This value also applies to global （ Non conversational ） Memory allocation .

garbage collection （MOT）

• enable_gc = true

  Whether to use a garbage collector （Garbage Collector,GC）.

• reclaim_threshold = 512 KB

  Set the memory threshold for the garbage collector .

  Each session manages its own list of objects to be recycled , And perform its own garbage collection when the transaction is committed . This value determines the total memory threshold for objects waiting to be reclaimed , The threshold is exceeded , Session will trigger garbage collection .

• reclaim_batch_size = 8000

  Set the batch size of garbage collection .

  The garbage collector reclaims memory in batches from objects , In order to limit the number of objects recycled in a garbage collection delivery . The goal is to minimize the operation time of a single garbage collection delivery .

• high_reclaim_threshold = 8 MB

  Set the high memory threshold for garbage collection .

JIT（MOT）

• enable_mot_codegen = true

  Specifies whether to use... For plan queries JIT Query compilation and execution .

  JIT Query execution prepares for queries prepared in the planning phase JIT Compiled code . Whenever a prepared query is called , Will execute the generated JIT Compile function .JIT Compilation is usually done in LLVM In the form of . Not supported in native LLVM On the platform ,MOT Provides software based fallback （Tiny Virtual Machine,TVM）.

• force_mot_pseudo_codegen = false

  The current platform supports LLVM when , Whether to use TVM（ false LLVM）.

  Not supported in native LLVM On the platform ,MOT The automatic default is TVM.

  In native support LLVM On the platform , By default LLVM. This configuration item allows you to support LLVM On the platform TVM Conduct JIT Compile and execute .

• enable_mot_codegen_print = false

  Specify whether to be JIT The compiled query prints the issued LLVM/TVM IR Code .

• mot_codegen_limit = 100

  Limit allowed per user session JIT Number of queries .

Default MOT.conf file

The minimum settings and configuration assignments will postgresql.conf The document points to MOT.conf The location of the file ：

postgresql.conf  
mot_config_file = '/tmp/gauss/ MOT.conf'

Make sure max_process_memory The set value is sufficient to contain MOT Overall situation （ Data and index ） And local （ conversation ） Memory .

MOT.conf The default content of meets the needs of starting to use . The setting content can be optimized later .