Dispatching TurboGears via FCGI

This confirmed to run on Mac OS X 10.4.7 under Turbogears 0.9.9 and 1.1a (so, no reason not to run under the 1.0b release).

Information was drawn from the Turbogears trac wiki, which shows how to use Nginx to proxy to TG, and this wiki's FastCGI Example page, the latter detailing the PHP/FCGI process.

This is for Nginx/FCGI/Turbogears.

Conventions

Substitute throughout with the values relevant to your own set-up:

${HOST} = localhost - (or whatever you choose)

${PORT} = 8080 - (or whatever you choose)

${NGINX} = /usr/local/nginx - location of nginx installation

${PROJECTBASE} /opt/projects/wiki20 - location of Turbogears project

${PROJECTNAME} wiki20

Getting the required files

Two files are required to be created: ${NGINX}/scripts/fcgi.py and ${NGINX}/scripts/${PROJECTNAME}.fcgi.

To create ${NGINX}/scripts/fcgi.py ...

mkdir ${NGINX}/scripts

curl -o ${NGINX}/scripts/fcgi.py http://www.saddi.com/software/py-lib/py-lib/fcgi.py

To create ${NGINX}/scripts/${PROJECTNAME}.fcgi ...

Copy and paste the following to ${NGINX}/scripts/${PROJECTNAME}.fcgi. Edit the file, navigate to the "USER EDIT SECTION" and replace each instance of ${PROJECTBASE} and ${PROJECTNAME} with the corresponding values for your project.

   1 #!/usr/bin/python
   2 #
   3 # File name: project.fcgi
   4 #
   5 # This module provides the glue for running TurboGears applications behind 
   6 # FastCGI-enabled web servers. The code in this module depends on the fastcgi
   7 # module downloadable from here:
   8 #
   9 # http://www.saddi.com/software/py-lib/py-lib/fcgi.py
  10 #
  11 # NOTE: The fcgi.py file needs to be placed in a location that is on the
  12 # system path, such as the same the directory as the tg_fastcgi.py file
  13 # or in the base directory of the TG app code.
  14 #
  15 # To configure this module, please edit the three variables in the "USER EDIT
  16 # SECTION" before starting the TG application.  Also remember to edit the
  17 # top of this file with the correct Python installation information.
  18 
  19 import cherrypy
  20 import sys
  21 import os
  22 from os.path import *
  23 import pkg_resources
  24 import turbogears
  25 
  26 pkg_resources.require("TurboGears")
  27 
  28 # -- START USER EDIT SECTION
  29 # -- Users must edit this section -- 
  30 code_dir = '${PROJECTBASE}'    # (Required) The base directory of the TG app code.
  31 root_class_name = '${PROJECTNAME}.controllers.Root'    # (Required) The fully qualified Root class name.
  32 project_module_name = '${PROJECTNAME}.config'          # (Required) The config module name.
  33 log_dir = ''                                           # (Optional) The log directory. Default = code_dir.
  34 # -- END USER EDIT SECTION
  35 
  36 class VirtualPathFilter(object):
  37     def on_start_resource(self):
  38         if not cherrypy.config.get('virtual_path_filter.on', False):
  39             return
  40         prefix = cherrypy.config.get('virtual_path_filter.prefix', '')
  41         if not prefix:
  42             return
  43 
  44         path = cherrypy.request.object_path
  45         if path == prefix:
  46             path = '/'
  47         elif path.startswith(prefix):
  48             path = path[len(prefix):]
  49         else:
  50             raise cherrypy.NotFound(path)
  51         cherrypy.request.object_path = path
  52 
  53 
  54 def tg_init():
  55     """ Checks for the required data and initializes the application. """
  56 
  57     global code_dir
  58     global root_class_name
  59     global log_dir
  60     global project_module_name
  61     last_mark = 0
  62 
  63     # Input checks
  64     if not code_dir or not isdir(code_dir):
  65         raise ValueError("""The code directory setting is missing.
  66                             The fastcgi code will be unable to find
  67                             the TG code without this setting.""")
  68 
  69     if not root_class_name:
  70         raise ValueError("""The fully qualified root class name must
  71                             be provided.""")
  72 
  73     last_mark = root_class_name.rfind('.')
  74 
  75     if (last_mark < 1) or (last_mark + 1) == len(root_class_name):
  76         raise ValueError("""The user-defined class name is invalid.
  77                             Please make sure to include a fully
  78                             qualified class name for the root_class
  79                             value (e.g. wiki20.controllers.Root).""")
  80 
  81     sys.path.append(code_dir)
  82 
  83     # Change the directory so the TG log file will not be written to the
  84     # web app root.
  85     if log_dir and isdir(log_dir):
  86         os.chdir(log_dir)
  87     else:
  88         os.chdir(code_dir)
  89         log_dir = code_dir
  90 
  91     sys.stdout = open(join(log_dir, 'stdout.log'),'a')
  92     sys.stderr = open(join(log_dir, 'stderr.log'),'a')
  93 
  94     if exists(join(code_dir, "setup.py")):
  95         turbogears.update_config(configfile=join(code_dir, "dev.cfg"),modulename=project_module_name)
  96     else:
  97         turbogears.update_config(configfile=join(code_dir, "prod.cfg"),modulename=project_module_name)
  98 
  99     # Set environment to production to disable auto-reload and
 100     # add virutal path information.
 101     cherrypy.config.update({
 102         'global': {'server.environment': 'production'},
 103             '/' : { 'virtual_path_filter.on' : True,
 104             'virtual_path_filter.prefix' : '/bel.fcgi' }
 105                 })
 106 
 107     # Parse out the root class information for Cherrypy Root class.
 108     package_name = root_class_name[:last_mark]
 109     class_name = root_class_name[last_mark+1:]
 110     exec('from %s import %s as Root' % (package_name, class_name))
 111     Root._cp_filters = [VirtualPathFilter()]
 112     cherrypy.root = Root()
 113 
 114 # Main section -
 115 # Initialize the application, then start the server.
 116 tg_init()
 117 
 118 from fcgi import WSGIServer
 119 cherrypy.server.start(initOnly=True, serverClass=None)
 120 
 121 from cherrypy._cpwsgi import wsgiApp
 122 WSGIServer(application=wsgiApp).run()

Adjust the TurboGears configuration

Edit the ${PROJECTBASE}/dev.cfg or ${PROJECTBASE}/prod.cfg file (whichever you are using), uncomment the server.socket_port assignment and change ${PORT} to a value of your choice (make sure nothing else is running on that port, Tomcat defaults to 8080, as does Jetty. Save yourself some time and check first with a telnet localhost 8080, you should see Connection refused).

The relevant lines in prod/dev.cfg are:

# Some server parameters that you may want to tweak
server.socket_port=${PORT}
server.socket_host=${HOST}

Spawning a FastCGI TurboGears process

The lighttpd "spawn-fcgi" script is useful: download, compile and install lighttpd. Then (replacing ${HOST} and ${PORT} values appropriately), execute the following:

/usr/local/bin/spawn-fcgi -a ${HOST} -p ${PORT} -u nobody -f ${NGINX}/scripts/${PROJECTNAME}.fcgi

Nginx configuration

Save the following into ${NGINX}/conf/fastcgi_params

#fastcgi.conf
fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx;

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name; 

Add the following to the server section of the ${NGINX}/conf/nginx.conf configuration file, changing ${HOST} and ${PORT} as appropriate:

    # static files
    location ~ ^/(images|javascript|js|css|flash|media|static)/  {
        root ${PROJECTBASE}/${PROJECTNAME}/static;
    }
    location = /favicon.ico  {
        root ${PROJECTBASE}/${PROJECTNAME}/static/images;
    }
    # pass all requests to FastCGI TG server listening on ${HOST}:${PORT}  
    #  
    location / {  
        fastcgi_pass   ${HOST}:${PORT};
        fastcgi_index  index;  
        fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;  
        include        conf/fastcgi_params;  
    }  

Starting nginx

Start nginx with ${NGINX}/sbin/nginx. Point your browser to http://${HOST}:${PORT}/, your Turboears project should be serving via FastCGI. If so ... congratulations.

Performance test software

Basic but usefully free http://www.hpl.hp.com/research/linux/httperf/

Note

[I left the IP address as 0.0.0.0 because it worked for me, whereas 127.0.0.1 did not. If you're experiencing difficulties connecting to 0.0.0.0:8080, these are both alternative options; localhost:8080, 127.0.0.1:8080.]

Good luck.


NginxTurboGearsFCGI (last edited 2007-12-30 18:59:13 by Emiller)