2012-10-24

Flask With Green Pony. Part #1 - Putting Damn Pony into the Cage.

So I figured to share some evening battle stories. Maybe it will help someone while fighting against their own demons and zombies via Google jails.

TL;DR - scroll to the bottom.

Right now I'm fighting Gunicorn - a green nifty WSGY server. It's not the best. But I like it. Buzz off.
The situation is this. I'm developing quite big flask app. The reasons that made me choose flask over others was that I'm for sure going to use SQLAlchemy and flask has some nifty plugins. For me Django without it's ORM seems bit pointless. So I figured to take something smaller and build on top of it.

One of my friends always says how he's used to start big and just remove stuff he don't need. As I saw him using snippets that was exactly what he did. Vim combo results in huge snippet and then a third of that is deleted. I have quite opposite plan of actions. Start small, but clean and then build up. Sure I see advantage in his approach that you don't need to remember/find that much stuff. Anyways. Flask.

Flask has some issues. Devs loves to put everything in single file. That pisses me off. I like hierarchical clean structures. Also Flask just adores global variables. I would suggest to find a room for those two [flask and global variables], but hey who I am to judge?

So the idea is that I have Flask app that is developed by my standards meaning that there is startup scripts, management commands, soon to be plug-in modules, clean settings for app. Yeah. More or less lighter Django with more litter in the code. But it's MINE. Buzz off.

So challenge is to run my fat-ass Flask app with gunicorn. What Flask docs has to offer?

http://flask.pocoo.org/docs/deploying/wsgi-standalone/#gunicorn

myproject:app? that's what?.. Method? setup.py has possibility to add global console scripts (terminal global scrips per say) that uses package:method declaration. But this does not work in my case if I give it main method. I wish... So what Google has to offer?

http://www.capsunlock.net/2011/04/running-flask-with-gunicorn.html
http://community.webfaction.com/questions/7229/nginxgunicornflask-setup
http://samrat.me/blog/2012/05/flask-nginx-gunicornon-a-vagrant-box/
http://wirtel.be/posts/en/2011/02/24/nginx_gunicorn_flask/

So it expects global variable once again. Great. But then again what's the point of it? I don't want to always run my app with $ gunicorn <stupid_miles_long_path>/main:app <crap-ton of properties>
I want to make it that I just could run MyApp within terminal and give few SIMPLE arguments and everything would be fine. So I want to implement gunicorn directly to my app. Most of the internet here is useless, all you can find is how to proxy with nginx to $ gunicorn main:app

Here is the golden link:

http://damianzaremba.co.uk/2012/08/running-a-wsgi-app-via-gunicorn-from-python/

It's almost right. Few fixes - main file is not main.wsgi in most projects, it's proper *.py and there is few problems. Last line of MyCustomApplication class.

If you look at util file https://github.com/benoitc/gunicorn/blob/master/gunicorn/util.py#L275 you'll find that it tries to extract app file from string (you're giving it string - what do you expect). If you're doing this directly from app - you have everything on your finger tips - why the hell you would want to get file from same folder try to parse it's name and then fail. Yep. It leads to error of booting workers. Something similar to -
https://github.com/benoitc/gunicorn/issues/338

If you're doing everything cleanly - it will simply wont find THE app. And it's easy to fix it. Instead of calling utg.import() simply import your global variable from whatever you placed it and return THE app.

Oh also I added ProxyFix. Not sure if I need it. In most google places it was added so I would assume it's a good thing.

So here is that custom extra class. I named it Server:

from gunicorn.app.base import Application

from werkzeug.contrib.fixers import ProxyFix

from MyApp import app


class Server(Application):
    '''
    Custom Gunicorn Application
    '''

    def __init__(self, options={}):
        '''__init__ method

        Load the base config and assign some core attributes.
        '''
        self.usage = None
        self.callable = None
        self.options = options
        self.do_load_config()

    def init(self, *args):
        '''init method

        Takes our custom options from self.options and creates a config
        dict which specifies custom settings.
        '''
        cfg = {}
        for k, v in self.options.items():
            if k.lower() in self.cfg.settings and v is not None:
                cfg[k.lower()] = v
        return cfg

    def load(self):
        '''load method

        Imports our application and returns it to be run.
        '''
        app.wsgi_app = ProxyFix(app.wsgi_app)
        return app

And this is how my part of main:main looked with flask running:

def main():
    startup()

    app.secret_key = settings.SECRET
    app.run(
        host=settings.ARGS['--host'],
        port=int(settings.ARGS['--port']),
        debug=settings.DEBUG,
    )

And this is how it looks now:

def main():
    startup()

    app.secret_key = settings.SECRET
    Server({
        'bind': '%s:%s' % (settings.ARGS['--host'], settings.ARGS['--port']),
        'debug': settings.DEBUG,
        'worker_class': 'gevent',
        'workers': numCPUs() * 2 + 1,
    }).run()

Sure main has right now access to numCPUs() method and sure Server is imported. numCPUs() method can be found in previously mentioned link - http://www.capsunlock.net/2011/04/running-flask-with-gunicorn.html
Also my app uses docopt, so I just removed __doc__ string from examples here.

I still right now have some issues - it seems gunicorn ignores my debug option. Demonizer suicides. But that's for another evening.

Over and out.