Tornado coroutines simplify asynchronous code at the cost of a performance overhead incurred by wrapping the Stack Context to manage the lifecycle of the coroutine.

When performance is critical, and by critical I mean you've measured that this is something you need to do, well then you might consider removing coroutines in favor of callbacks.

Be warned, callbacks makes your code brittle and difficult to reason about.

Start with a coroutine

Consider this easy-to-follow coroutine:

from tornado import gen

@gen.coroutine
def greet():
    msg = "HELLO"
    msg = yield get_world(msg)
    msg = msg.lower()
    raise gen.Return(msg)

We can call this coroutine using yield from another coroutine:

@gen.coroutine
def caller():
    greeting = yield greet()

This is ideal - our code looks synchronous with all the async business neatly tucked away.

Convert to callback

If we're sure we can't afford the gen.coroutine, then we could refactor to callback-style code:

import sys
from tornado import gen

def greet():
    msg = "HELLO"

    return_future = gen.Future()
    get_world_future = get_world(msg)

    def get_world_done(f):
        try:
            result = f.result()
            result = result.lower()
        except Exception:
            return_future.set_exc_info(sys.exc_info())
        else:
            return_future.set_result(result)

    get_world_future.add_done_callback(get_world_done)

    return return_future

For every callback, you'll need to manually connect the chain by:

  1. Handling the Result
  2. Handling Exceptions

This is laborious and adds significant boilerplate; but is much quicker.

DRY up exception handling

Moving the exception handling to a decorator can make our lives a bit easier:

import sys
from functools import wraps
from tornado.gen import is_future

def fail_to(future):
    assert is_future(future), 'you forgot to pass a future'
    def decorator(f):
        @wraps(f)
        def new_f(*args, **kwargs):
            try:
                return f(*args, **kwargs)
            except Exception:
                future.set_exc_info(sys.exc_info())
        return new_f
    return decorator

Our greet function becomes:

from tornado import gen

def greet():
    msg = "HELLO"

    return_future = gen.Future()
    get_world_future = get_world(msg)

    @fail_to(return_future)
    def get_world_done(f):
        result = f.result()
        result = result.lower()
        return_future.set_result(result)

    get_world_future.add_done_callback(get_world_done)

    return return_future

Now our done callback can act on the result without worrying about exception management.

Cheers.