tornado之多进程服务

tornado多进程开启

在前文"tornado之hello world"中简单介绍过tornado的基本使用。tornado默认启动的是单进程，启动方式如下：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    # application.listen(8888)
    server = HTTPServer(application)
    server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

如果要开启多进程，可进行如下修改：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    server = HTTPServer(application)
    server.bind(8888)
    server.start(2)
    tornado.ioloop.IOLoop.instance().start()

启动后生成了3个进程(注意关闭debug模式)。start函数中参数说明如下：

If num_processes is ``None`` or <= 0, we detect the number of cores
available on this machine and fork that number of child
processes. If num_processes is given and > 1, we fork that
specific number of sub-processes.

意思是，当参数小于等于０时，则根据当前机器的cpu核数来创建子进程，大于１时直接根据指定参数创建子进程。
看一下bind函数的说明：

def bind(self, port, address=None, family=socket.AF_UNSPEC, backlog=128,
         reuse_port=False):
    """Binds this server to the given port on the given address.

    To start the server, call `start`. If you want to run this server
    in a single process, you can call `listen` as a shortcut to the
    sequence of `bind` and `start` calls.

    Address may be either an IP address or hostname.  If it's a hostname,
    the server will listen on all IP addresses associated with the
    name.  Address may be an empty string or None to listen on all
    available interfaces.  Family may be set to either `socket.AF_INET`
    or `socket.AF_INET6` to restrict to IPv4 or IPv6 addresses, otherwise
    both will be used if available.

    The ``backlog`` argument has the same meaning as for
    `socket.listen <socket.socket.listen>`. The ``reuse_port`` argument
    has the same meaning as for `.bind_sockets`.

    This method may be called multiple times prior to `start` to listen
    on multiple ports or interfaces.

    .. versionchanged:: 4.4
       Added the ``reuse_port`` argument.
    """

也就是说，当start函数的参数为１时，listen函数可以代替bind和start。

多进程开启原理

对启动代码进行分析：
先创建一个HTTPServer实例。然后在bind函数中，调用了bind_sockets方法来创建sockets。接着在start方法中，调用process.fork_processes(num_processes)方法来创建子进程，并将sockets通过add_sockets方法添加到server实例中。
也就是说，这一段代码的实质如下：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    sockets = tornado.netutil.bind_sockets(8888)
    tornado.process.fork_processes(2)
    server = HTTPServer(application)
    server.add_sockets(sockets)
    tornado.ioloop.IOLoop.instance().start()

继续分析fork_process函数，其主要过程如下：

def start_child(i):
    pid = os.fork()
    if pid == 0:
        # child process
        _reseed_random()
        global _task_id
        _task_id = i
        return i
    else:
        children[pid] = i
        return None

for i in range(num_processes):
    id = start_child(i)
    if id is not None:
        return id
num_restarts = 0
while children:
    try:
        pid, status = os.wait()
    except OSError as e:
        if errno_from_exception(e) == errno.EINTR:
            continue
        raise
    if pid not in children:
        continue
    id = children.pop(pid)
    if os.WIFSIGNALED(status):
        gen_log.warning("child %d (pid %d) killed by signal %d, restarting",
                        id, pid, os.WTERMSIG(status))
    elif os.WEXITSTATUS(status) != 0:
        gen_log.warning("child %d (pid %d) exited with status %d, restarting",
                        id, pid, os.WEXITSTATUS(status))
    else:
        gen_log.info("child %d (pid %d) exited normally", id, pid)
        continue
    num_restarts += 1
    if num_restarts > max_restarts:
        raise RuntimeError("Too many child restarts, giving up")
    new_id = start_child(id)
    if new_id is not None:
        return new_id
# All child processes exited cleanly, so exit the master process
# instead of just returning to right after the call to
# fork_processes (which will probably just start up another IOLoop
# unless the caller checks the return value).
sys.exit(0)

从这个源码可以看出，子进程的开启实际上是调用的linux系统提供的fork功能。当子进程存在时，父进程通过一个while循环来处理子进程的终止信号，尝试重启已经停止的子进程，直到超过默认的最大重启次数100。