Mysql database recovery case sharing

The case shared this time is about the recovery of stored data , On storage RAID The crash caused the storage to fail to start . Store internal common 6 More than virtual machines , among LINUX virtual machine 3 Important data for customers . The engineer preliminarily analyzed that the storage structure is that all physical disks are in one storage pool , The storage pool is divided into several LUN,LUN1 yes vmfs volume , Three stations LINUX The virtual machine is also in this . 1、 restructuring RAID During the reorganization process, Ben RAID5 defect 2 Block plate （ The hot spare disk replaces the first drop disk after the drop , After that, another disk dropped to make RAID5 In a degraded state . Finally, the disc was scratched when the third disc was dropped RAID collapse ）, The data of the lost disk cannot be obtained directly through verification , Therefore, you can only use full disks of the same size 0 Reorganize the image （ This method can only be used in emergency situations , Because it depends on empty images RAID The file system structure will be seriously damaged , It is equivalent to that each strip will be missing two blocks of data ）. 2、 extract LUN Analyze the storage structure , Gets the of the storage partition MAP block . Find MAP After the block is parsed, each LUN Data block pointer for , Write a data extraction program to extract LUN debris . After extraction, the fragments are spliced , Complete composition LUN. export LUN All virtual machines in the , Try to start . After exporting the virtual machine, try to start , Same as expected , The operating system is damaged and the virtual machine cannot be started . 3、 Extract the files in the virtual machine When the virtual machine fails to start, you can only take the second place , Extract the files in the virtual machine . Test after taking out the file , Found that most of the files were corrupted , Only a few small files can be opened . After communicating with the customer, I learned that there are... In the virtual machine MYSQL database , Because of the particularity of the underlying storage of the database , Data can be extracted by scanning data pages . After this virtual machine is found, the virtual machine is found to enable snapshot , When both the parent disk and the snapshot file are damaged, the normal merge operation cannot be completed , Using independent research and development of North Asia VMFS The snapshot merging program performs snapshot merging . 4、 obtain MYSQL Data page and analyze according to MYSQL Data page features scan and export data pages （innodb The engine can use this scheme ,myisam Because no “ Data pages ” The concept is therefore not available ）, Analyze the system table to obtain the information of each user table , According to the... Of each table ID Split the data page . 5、 Extract table structure Because the database has been used for a long time , The table structure has also changed many times , In addition, some data is lost after the system table is damaged , There is a lot of resistance in the record extraction process . First, get the table structure of each table in the original version of the database ： The parent disk before the snapshot was merged because it was written earlier , Use the first drop disk for verification to obtain the complete data of this file , Then extract the table structure of each table in the database , After that, the client provided the latest version of the database table creation script . Extract record ： Two groups of different table structures are used to extract data records and import them into the recovery environment MYSQL In the database , Then eliminate the garbled data caused by the change of table structure in each table , Finally, the two sets of data are exported as .sql file . 6、 Data recovery results Because the database table structures of the two versions are different , So I contacted the application engineer of the client for debugging . Import the platform after debugging , verified , The data can be recovered successfully .