Synchronization mechanism of dual namenodes

Preface ：Hadoop2.0 Based on JournalNode The double NameNode Metadata synchronization mechanism

Log nodes （JournalNode） The concept and function of

JournalNode yes HA colony （ High availability cluster ,HighAvailabilityCluster） in Active NameNode（ANN, Activity name node ） and StandBy NameNode（SNN, Alternate name node ） Edit logs between Edits and HDFS The transmission medium for metadata sharing .

The main functions are ： For two NameNode Sharing data between , sync .

The process

1. Active Namenode Go to Journal Nodes Write in EditLog data

2. StandBy Namenode Again from Journal Nodes Read from EditLog Data synchronization

Log manager （QJM） The concept and working principle of

Concept ：QJM It consists of multiple log nodes JournalNodes（JN） Composed of JournalNode colony , Mainly used for storage EditLog, Generally, an odd number of JournalNodes, Need at least 3 individual , Every JournalNode Save the same EditLog copy .

be based on QJM Data synchronization mechanism of shared storage ：

1. When Active NameNode Make any changes to the namespace , It needs to be told JournalNodes Most of the clusters （ More than half ） Of JournalNodes process

2. and Standby NameNode Be responsible for observing JournalNodes The change of , Have the ability to learn from JournalNodes Read from Active NameNode Messages sent , Keep monitoring EditLog The change of , And update its internal namespace , Keep monitoring EditLog The change of , And update its internal namespace

3. once Active NameNode Encounters an error ,Standby NameNode Need to have been from JournalNodes Read all in EditLog journal , Ensure that before the failover occurs NameNodes The state of has been fully synchronized , then ,Standby NameNode Switch to Active state