8205663: Tool to stop a TCPServer (or HTTPServer) in a graceful manner (when the all currently running work is done). Thisscript works either for forked servers or single-process servers.

Added 3 years ago by Taha Jahangir

Love

2 Add your love!

Tags

Discussion

Graceful Stopper

Tool to stop a TCPServer (or HTTPServer) in a graceful manner (when the all currently running work is done). This script works either for forked servers or single-process servers.

The graceful-stop process can be triggered by sending SIGHUP to server (in a forked environment, sending SIGHUP to master server, propagates to all children). Server immediately releases the listening socket (so another server can be launched on that port) and wait for running requests to finish. The server is also stops if stop-timeout is reached (defaults to 100 seconds, can be changed or disabled).

Usage

  1. single-process GracefulHTTPServer using listen:

    server = GracefulHTTPServer() server.listen(8888) IOLoop.instance().start()

  2. multi-process GracefulHTTPServer using start:

    server = GracefulHTTPServer() server.bind(8888) server.start(0) # Forks multiple sub-processes IOLoop.instance().start()

  3. advanced multi-process GracefulHTTPServer using add_sockets

    sockets = tornado.netutil.bind_sockets(8888) server = GracefulHTTPServer() # should be before fork tornado.process.fork_processes(0) server.add_sockets(sockets) IOLoop.instance().start()

  4. advanced usage of GracefulServerStopper for any TCPServer

    stopper = GracefulServerStopper() # should be called before fork, (server # can be passed inline as keyword arg) server = HTTPServer() server.bind(8888) server.start(0) stopper.server = server # can be called before or after fork stopper.setup_worker_handlers() # should be called after fork IOLoop.instance().start()

Customizing stop behaviour

Detecting the WORK_DONE state is not so easy, the simplest way is to watch IOLoop queue to be empty, but there may be resident handlers in IOLoop queue, so io-loop is always not-empty. (periodic callbacks (e.g. monitoring tasks), persistent database connections, ...)

Behaviour of detecting WORK_DONE state is encapsulated and can be changed via setting work_done_detector attribute on stopper (can be passed by work_done_detector keyword argument to GracefulHTTPServer or GracefulServerStopper)

Also there is more advanced RequestInspectorWorkDoneDetector that inspects callbacks in IOLoop and detects the WORK_DONE state when there is not HTTPRequest is in progress. (Uses tornado-inspector)

File: graceful_http_server.py