Linux Picture The Other LAMP
Part 1 of an intro to using mod_python
- using Linux Apache Mysql Python (plus potentially Postgresql)

Anthony Barker

April 21, 2004

Testing out the latest LAMP framework Linux Apache Mysql Python w/ mod_python

This tutorial assumes you have some knowledge of html and web programming. If you don't there is tons of information available on the web. In particular I recommend Phil Greenspun's book.


I am going to assume that you are developing on a windows machine and will deploy to a linux server.

Why Python?

Python is the ideal programming language for new programmers. It is easy and quick to learn, the interpreter makes playing with your code easy and it comes with an extensive library so you can reuse other people's code and develop more quickly. This makes it ideal for developing prototypes and rapid development. Websites such as google, yahoo mail, yahoo groups and others were at some point in their lives mostly python.

The downsides? Because python is dynamically typed for larger applications unit testing is important. Not as many ISPs support mod_python as PHP, mod_perl and ASPs yet.

mod_python is a module that allows python to run embedded directly into the apache server allowing for good performance. The current version includes support for sessions, psp templates, and cgi support.

Scaling

How fast is mod_python? Recent benchmarks that I have made put it (20-50x) faster than cgi or zope and mildly faster than mod_php. If you need to make your application truly scalable you should investigate caching servers such as mod_cache in apache or HTTP server acceleration using squid server. Generate much of your content as possible as static pages and try to minimize the session information held in memory.
If bandwidth is the constraint have a look at mod_gzip or the gzip example in mod_python. You will see 50% to 20% bandwidth savings. Most browsers support this - unfortunately most spiders don't yet support gzip.


How to run code?

The original method of running code on web servers was through the CGI model. In CGI each time the program is run a separate process is spawned and its standard output is redirected to the browser. Because python is interpreted there is significant overhead of launching the interpreter. Every CGI request causes a process to be created and the Python interpreter loaded and initialized, which results in unnecessary CPU and I/O activity. mod_python allows for the script to stay resident in memory thereby reducing overhead.
One advantage of CGI is that memory leaks don't tend to build. Memory is cleaned up after each time the script is run. Another advantage is that cgi is available under every platform and web server. Luckily, mod_python includes a compatibility library for cgi. The form handling is similar, but session handling is quite different. I will be focusing on pure mod_python.

GETTING STARTED


Installation You are going to want to download and install Python, Apache 2, mod_python 3.1 and MySQL and some optional tools.


Python

Grab the latest version of python from the python website. I recommend that you add the directory to your path. eg c:\python23; It includes some great documentation in windows help format. Typically Linux includes python. If you have an older version that came with your linux distro you should upgrade. With older versions of redhat you had to ensure you didn't stomp on the original version of python that was installed. However, since up2date for redhat is discontinued you shouldn't have to worry about that any more.

Learning python:

Check out the tutorials from the python website. One of my favorite presentations was done by by David Beazley or here as one page http://www.ferg.org/python_slides/handouts.html

The documentation on python.org also includes the library reference, which is good to have. Another good way to get help on the libraries is from the Python interpreter. If you run the python interpreter and type help(), you will be in the help menu. Then, type a module name and you will get documentation on that module. A wide array of dead tree material is available.


Apache2

Get version 2.0.48 or better. During the install it will prompt you for where it should be located. If you are on windows I recommend that you install right off your drive letter so you don't need to type so much if you are accessing it from the command prompt. c:\apache is good. You will see that apache installs a monitoring tool and a link to the httpd.conf file where configuration settings are stored. The website is typically in c:\apache\Apache2\htdocs. This location can be changed by editing your httpd.conf file and changing the DocumentRoot parameter.

For linux visit apache.org and download the latest tar file or rpm and do the configure;make;make install; dance.

mod_python
Get mod_python from the modpython.org website. Edit your httpd.conf file and update the AddHandler parameter
On you local machine edit your c:\apache\Apache2\conf\httpd.conf file and add
LoadModule python_module libexec/mod_python.so
<Directory "c:/apache/Apache2/htdocs/test">
        AddHandler mod_python .py
        PythonHandler mptest
        PythonDebug On
    </Directory>
Where test is the directory you will add your application and mptest.py will be the python file initial app that runs.

MySQL
Download and install mysql as well as mysql command center.
Also install the mysqldb driver for python
Mysqldb is a dbi 2.0 compliant driver for mysql. DBI in a way comparable to PERL dbi, or the PHP pear db, gives you a standardized way of accessing databases. If you are unfamiliar with sql visit an excellent sql tutorial http://sqlzoo.net/ or http://philip.greenspun.com/sql/
.

Before you can start using it you will need to add your Windows machine to the list of hosts allowed to access the server. This does require you to login to the server. Launch a Terminal window and, at the command line, type "mysql" before pressing enter. This will launch the MySQL Monitor where you can perform SQL operations on the backend databases. In this case we need to issue a GRANT statement to add a new user. We will give full privileges to a user called "root" from any machine ('%') to the all databases. The command is:


GRANT ALL ON *.* TO 'root'@'%';
CREATE DATABASE zoo;
CREATE TABLE `animals` (
`entry_id` int(11) NOT NULL auto_increment,
`name` varchar(250) NOT NULL default '',
`category` varchar(250) NOT NULL default '',
PRIMARY KEY (`entry_id`)
) TYPE=MyISAM; 

or using the command center



Other Useful stuff:

Cheetah

Cheetah is an alternative templating system (different from the .PSP that comes with mod_python). I recommend that you install it to see which templating system you prefer. It is a little more mature and has more documentation than psps.

Text Editor / IDE

You will probably need a text editor. If you are using windows editplus or textpad are ok. In linux I use emacs or vi. For a full list of editors visit the python website.

Firefox

Firefox browser with Live HTTP headers installed so you can see exactly what is happening on the server and client. Firefox includes a DOM browser and Javascript debugger.

SQL Object
http://www.sqlobject.org/

SQL object is worth checking out -SQLObject is an object-relational mapper. It allows you to translate RDBMS table rows into Python objects, and manipulate those objects to transparently manipulate the database.

It allows you to use something like
>>> p = animal.new(name="Giraffe", category="mammal")
to create a new animal instead of
>>> db.execute(("INSERT into animals (name, category) values %s, %s), "Giraffe", "mammal")

db_rows
http://opensource.theopalgroup.com/

Often it is handy to get a handle on returned data by field name - not by an integer. PHP for example allows you to do this with MySQL. db_rows return a query row as a tuple of column values.db_rows is light-weight and fast, adding 8% overhead to your queries.

so a query that looks like this:
>>> cursor.execute('SELECT name, category FROM zoo;')
>>> results = cursor.fetchall()
>>> results
[ ("Giraffe","mammal"), ("Dog","mammal") ]

becomes:
>>> cursor.execute('SELECT name, category FROM zoo;')
# Make a class to store the resulting rows
>>> R = IMetaRow(cursor.description)
# Build the rows from the row class and each tuple returned from the cursor
>>> results = [ R(row) for row in cursor.fetchall() ]
>>> print results[1].fields.name, results[1]['category']



Test
A) TEST mod_python
Start Apache Server service
First test your mod_python installation to ensure it is working properly
create the directory test in your htdocs directory
And create a file with the following text.

from mod_python import apache

    def handler(req):
        req.write("Hello World!")
        return apache.OK


Create a mysql database
Create that example mysql database using the command centre or mysql command line.

C) Test MySQL connection
Start mysql from the menu and then go to the command prompt in windows or shell in linux type "python"

C:\temp>python
Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>> db = MySQLdb.connect(host="localhost", user="joe", passwd="secret", db="zoo")
>>> cursor = db.cursor()
>>> cursor.execute("SELECT * FROM animals")
>>> data = cursor.fetchall()
>>> data
1) import the MySQLdb module
2) connect to the server and database, customize the parameters such as user, passwd and db as you require
3) create a cursor. This is actually a dumby cursor as mysql won't support cursors until version 5.x
4) execute SQL statement
5) Close connection
Have a look at this tutorial for more on mysqldb http://images.devshed.com/Server_Side/Python/PythonMySQL/PythonMySQL.pdf

Also check out python dbi documentation
http://www.python.org/peps/pep-0249.html

Getting started with dbi - the standard python api for databases check out this article:
http://www.linuxjournal.com/article.php?sid=2605

General mysqldb stuff
http://www.kitebird.com/articles/pydbapi.html

Next Your First Form
Creating records Data
Updating Data
Viewing Data
Logging in the user
Getting and setting session data