当前位置：网站首页>PostgreSQL database Wal - resource manager rmgr

PostgreSQL database Wal - resource manager rmgr

2022-06-27 18:44:00 【Ink Sky Wheel】

XLog Logs are divided into multiple types of resource managers , Each resource manager only needs to be responsible for its own log processing （ Abstract out the operation function , Different logs implement different operation functions ）. The definition of the resource manager is as follows ：

typedef struct RmgrData {
 const char *rm_name;  Resource name 
 void (*rm_redo) (XLogReaderState *record);  Redo function 
 void (*rm_desc) (StringInfo buf, XLogReaderState *record);  Responsible for parsing the corresponding resource transaction log 
 const char *(*rm_identify) (uint8 info); //  analysis xlog In record Info Field 
 void (*rm_startup) (void); //  Initialization at startup 
 void (*rm_cleanup) (void); //  Clean up at the end 
 void (*rm_mask) (char *pagedata, BlockNumber blkno); //  Before checking the page for consistency , Whether some parts of the page need to be masked 
} RmgrData;
extern const RmgrData RmgrTable[];

Multiple types of resource managers are defined in src/include/access/rmgrlist.h in , Their functions and operation functions are as follows .

Operation function call process

rm_identify Interface and rm_desc The interface is used for pg_waldump Tool pair WAL Log reverse parsing （ Parse binary log records into strings ）.rm_startup、rm_redo、rm_mask and rm_cleanup be used for startup Process execution StartupXLOG Function XLOG Initialization resource manager in log playback RMGR、 The playback xlog、 Check the consistency of the page and clean it up , The execution process is as follows ：

void StartupXLOG(void){
 ...
 /* Initialize resource managers */ //  Initialize the resource manager RMGR
 for (rmid = 0; rmid <= RM_MAX_ID; rmid++) {
 if (RmgrTable[rmid].rm_startup != NULL)
 RmgrTable[rmid].rm_startup();
 }
 ...
 /* main redo apply loop */ //  Main playback logic 
 do { 
 ...
 /* Now apply the WAL record itself */ //  call rm_redo The playback xlog
 RmgrTable[record->xl_rmid].rm_redo(xlogreader); 
 /* After redo, check whether the backup pages associated with the WAL record are consistent with the existing pages. This check is done only if consistency check is enabled for this record. */ // redo  after , Check with  WAL  Record whether the associated backup page is consistent with the existing page . This check is only performed if consistency checking is enabled for this record .
 if ((record->xl_info & XLR_CHECK_CONSISTENCY) != 0)
 checkXLogConsistency(xlogreader); 
 ... 
 record = ReadRecord(xlogreader, LOG, false); /* Else, try to fetch the next WAL record */
 } while (record != NULL);


 /* Allow resource managers to do any required cleanup. */ //  Clean up 
 for (rmid = 0; rmid <= RM_MAX_ID; rmid++) {
 if (RmgrTable[rmid].rm_cleanup != NULL)
 RmgrTable[rmid].rm_cleanup();
 } 
}

1.rm_startup For different types xlog Resources call different initialization functions to initialize the resource manager RMGR.

2. according to xlog Medium xl_rmid Call the... Of different resources in the resource manager rm_redo Playback function for playback .

3. We know startup process recovery In the process , because wal replay BUG Or the physical data block of the standby database is abnormal wal replay The playback block is incorrect .PostgreSQL 10.0 Newly added wal_consistency_checking Parameters , It can be used to find such problems .wal_consistency_checking Parameters can be set to the following values ：all, heap, heap2, btree, hash, gin, gist, sequence, spgist, brin, and generic.

wal_consistency_checking (string)

This parameter is intended to be used to check for bugs in the WAL redo routines. When enabled, full-page images of any buffers modified in conjunction with the WAL record are added to the record. If the record is subsequently replayed, the system will first apply each record and then test whether the buffers modified by the record match the stored images. In certain cases (such as hint bits), minor variations are acceptable, and will be ignored. Any unexpected differences will result in a fatal error, terminating recovery. The default value of this setting is the empty string, which disables the feature. It can be set to all to check all records, or to a comma-separated list of resource managers to check only records originating from those resource managers. Currently, the supported resource managers are heap, heap2, btree, hash, gin, gist, sequence, spgist, brin, and generic. Only superusers can change this setting.

StartupXLOG --> checkXLogConsistency For different resource management ID, Execute the corresponding function , Its main logic is as follows ：

RmgrId rmid = XLogRecGetRmid(record); // record The type of XLogReaderState *
if (RmgrTable[rmid].rm_mask != NULL) {
 RmgrTable[rmid].rm_mask(replay_image_masked, blkno);
 RmgrTable[rmid].rm_mask(primary_image_masked, blkno);
}

4. For different resource management ID, Call its corresponding function rm_cleanup Clean up resources

From the above process, we can see that the playback process is wal_consistency_check, Depending on xlog info Whether to set XLR_CHECK_CONSISTENCY, The setting process is in XLogRecordAssemble Function , If wal_consistency_checking The array corresponds to the resource ID Set to true（wal_consistency_checking Array in GUC Module assign_wal_consistency_checking Function according to wal_consistency_checking Parameter setting ）, Corresponding xlog info Set XLR_CHECK_CONSISTENCY sign .

static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi){


 /* Enforce consistency checks for this record if user is looking for it. Do this before at the beginning of this routine to give the possibility for callers of XLogInsert() to pass XLR_CHECK_CONSISTENCY directly for a record.*/ //  If the user is looking for this record , Then the consistency check is enforced . Do this before this routine starts , In order to make  XLogInsert()  The caller of can directly pass  XLR_CHECK_CONSISTENCY  To get records 
 if (wal_consistency_checking[rmid])
 info |= XLR_CHECK_CONSISTENCY;


}