DataNode的启动流程

LD is tigger forever，CG are not brothers forever， throw the pot and shine.
Efficient work is better than attitude。All right, hide it。Advantages should be hidden.
talk is selected, show others the code,Keep progress，make a better result.

目录

概述

hadoop datanode 数据服务的源码解析

正文：

数据服务的源码
1.初始化DataXceiverServer
2.初始化HTTP服务
3.初始化DN的RPC服务端
4.DN向NN（NameNode ）注册
5.向NN发送心跳

如下源码：
storage = new DataStorage();
// global DN settings
registerMXBean();
// 初始化 DataXceiver
initDataXceiver();
// 启动 HttpServer
startInfoServer();

pauseMonitor = new JvmPauseMonitor();
pauseMonitor.init(getConf());
pauseMonitor.start();
// BlockPoolTokenSecretManager is required to create ipc server.
this.blockPoolTokenSecretManager = new BlockPoolTokenSecretManager();
// Login is done by now. Set the DN user name.
dnUserName = UserGroupInformation.getCurrentUser().getUserName();
LOG.info(“dnUserName = {}”, dnUserName);
LOG.info(“supergroup = {}”, supergroup);
// 初始化 RPC 服务
initIpcServer();
metrics = DataNodeMetrics.create(getConf(), getDisplayName());
peerMetrics = dnConf.peerStatsEnabled ?
DataNodePeerMetrics.create(getDisplayName(), getConf()) : null;
metrics.getJvmMetrics().setPauseMonitor(pauseMonitor);
ecWorker = new ErasureCodingWorker(getConf(), this);
blockRecoveryWorker = new BlockRecoveryWorker(this);
// 创建 BlockPoolManager
blockPoolManager = new BlockPoolManager(this);
// 心跳管理
blockPoolManager.refreshNamenodes(getConf());
// Create the ReadaheadPool from the DataNode context so we can
// exit without having to explicitly shutdown its thread pool.
readaheadPool = ReadaheadPool.getInstance();
saslClient = new SaslDataTransferClient(dnConf.getConf(),
dnConf.saslPropsResolver, dnConf.trustedChannelResolver);
saslServer = new SaslDataTransferServer(dnConf, blockPoolTokenSecretManager);
startMetricsLogger();
if (dnConf.diskStatsEnabled) {
diskMetrics = new DataNodeDiskMetrics(this,
dnConf.outliersReportIntervalMs);
}
}

2.1 初始化 DataXceiverServer
点击 initDataXceiver
private void initDataXceiver() throws IOException {
// dataXceiverServer 是一个服务，DN 用来接收客户端和其他 DN 发送过来的数据服务
this.dataXceiverServer = new Daemon(threadGroup, xserver);
this.threadGroup.setDaemon(true); // auto destroy when empty
…

2.2 初始化 HTTP 服务

点击 startInfoServer();

private void startInfoServer()
throws IOException {
// SecureDataNodeStarter will bind the privileged port to the channel if
// the DN is started by JSVC, pass it along.
ServerSocketChannel httpServerChannel = secureResources != null ?
secureResources.getHttpServerChannel() : null;
httpServer = new DatanodeHttpServer(getConf(), this, httpServerChannel);
httpServer.start();
if (httpServer.getHttpAddress() != null) {
infoPort = httpServer.getHttpAddress().getPort();
}
if (httpServer.getHttpsAddress() != null) {
infoSecurePort = httpServer.getHttpsAddress().getPort();
}
}

DatanodeHttpServer.java
public DatanodeHttpServer(final Configuration conf,
final DataNode datanode,
final ServerSocketChannel externalHttpChannel)
throws IOException {
… …
HttpServer2.Builder builder = new HttpServer2.Builder()
.setName(“datanode”)
.setConf(confForInfoServer)
.setACL(new AccessControlList(conf.get(DFS_ADMIN, " ")))
.hostName(getHostnameForSpnegoPrincipal(confForInfoServer))
.addEndpoint(URI.create(“http://localhost:” + proxyPort))
.setFindPort(true);
.

2.3 初始化 DN 的 RPC 服务端
private void initIpcServer() throws IOException {
InetSocketAddress ipcAddr = NetUtils.createSocketAddr(
getConf().getTrimmed(DFS_DATANODE_IPC_ADDRESS_KEY));
… …
ipcServer = new RPC.Builder(getConf())
.setProtocol(ClientDatanodeProtocolPB.class)
.setInstance(service)
.setBindAddress(ipcAddr.getHostName())
.setPort(ipcAddr.getPort())
.setNumHandlers(
getConf().getInt(DFS_DATANODE_HANDLER_COUNT_KEY,
DFS_DATANODE_HANDLER_COUNT_DEFAULT)).setVerbose(false)
.setSecretManager(blockPoolTokenSecretManager).build();
… …

设计思路

相关代码如下：

完整版下载： hadoopDataNode解析

小结：

主要讲述了自己的一些体会，里面有许多不足，请大家指正~

参考资料和推荐阅读

深度开源: link

欢迎阅读，各位老铁，如果对你有帮助，点个赞加个关注呗！~