Web Development with Python

Authors:	Robert Forkel
Date:	September 12, 2007
Location:	MPDL

In this seminar we give an introduction to Web Application development, with a focus on the TurboGears framework - a web application stack for the python language.

Contents

Introduction

We'll talk about

the Web (without depicting it as a cloud),
Web Applications,
Python (not really),
TurboGears (but you should really try it out).

What's the Web?

The web is ...

... the HTTP-speaking part of the internet.

(which mostly speaks HTML, too.)

Ok, so what's HTTP?

The Hypertext Transport Protocol

client (or user agent) requests a response from a server:

GET /index.html HTTP/1.1
host: dev.livingreviews.org

HTTP/1.1 200 OK
Date: Sat, 01 Sep 2007 14:35:29 GMT
Server: Apache
Last-Modified: Tue, 03 Apr 2007 08:00:52 GMT
ETag: "22c05a-1c89-bc8ed500"
Accept-Ranges: bytes
Content-Length: 7305
Content-Type: text/html; charset=UTF-8

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML ...
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
...

Note: No matter how many layers of abstraction a web applicatio is built on, HTTP is what goes over the wire.

HTTP is standardized in a RFC Unfortunately it shares the fate of (all?) other web standards: many applications only follow the letter but not the spirit of the law. (We'll come to the spirit later, it's called REST.)

Statelessness

Everybody knows:

HTTP is stateless

Why is this good?

easy to implement server side protocol.

Why is it bad?

if you need state, the protocol does not help, alas ...

Is it true?

Well, no: cookies store state on the client!

Example of non-stateless protocol: ftp, e.g. the server has a notion of a current working directory. A tell-tale sign is the LOGOUT (QUIT) command, which is absent in HTTP.

URLs

Uniform Resource Locators:

<scheme>://<host><path>?<searchQuery>

The HTTP variety:

http(s)://<host>.<domain><path>?key1=value1&key2=value2#<fragment>

Notes:

The exact format of the query string is not specified! The "&-separated key=value pairs" notation is just a convention.
The path must start with a "/".

MIME Types

The Content-Types of the Internet:

Multipurpose Internet Mail Extensions Media Types

They specify how to interpret the body.

Top level media types (the stuff before the slash):

text
image
audio
video
application

For details:

http://www.isi.edu/in-notes/rfc2046.txt

The Spirit of HTTP

What is REST?

Representational State Transfer.
HTTP done right:
- HTTP method tells what to do,
- request path specifies the resource to act on,
- use GET when server state does not change.

REST is the web app architecture for friends of Leibniz:

HTTP is the best possible protocol for the web.

Do I need to know HTTP?

Short answer: YES!

Longer answer:

debugging
- which characters did the browser send?
testing
- write your own user agent to test a web app
compliance
- GET is supposed to be side-effect free!

What's a webb app?

A web application is

server side software ...
... that speaks HTTP ...
- to browsers
- and other user agents.

What's the difference between a web service and a web application?

None that I'm aware of.

Web App History

In the following we take a look at the history of web apps:

HTTP server
CGI
Web App Frameworks

HTTP server

The HTTP server

basically an HTTP interface to a remote filesystem,
interprets HTTP request headers,
responds by sending (HTML) files as response body,
one way communication (ignore PUT, POST and DELETE methods).

Programming Interface:

just the HTML in the pages, sent to the client.

CGI

The Common Gateway Interface: An API for web applications provided by HTTP servers:

Web server calls external program to process request,
Request header available as environment variables,
Request body available from standard input,
Program writes (HTTP) response to standard output.

Web App Frameworks

Modern web application frameworks at least provide

a scaffolding in which to plug request handlers,
request handlers with request and response objects;

but typically also

configurable URL mapping,
session handling,
support for response body creation (templating),
support for persistent storage,
a scheduler.

URL Mapping

URL mapping: The mechanism to get from request path to response.

Web server:

request path is a filesystem path, response is the file content.
CGI:

request path is a filesystem path, response created by the script located at this path.
Object publishing:

request path is a path in the web application's object hierarchy.
Configurable:

mapping request path to response handler is completely flexible.

Templating

Three options to get from programming language to target format:

Put code in the target format: psp, php, asp, kid,
- in the best case preserving the validity of the format.
Write target format from code: quichote,
- degrades target form to simple strings.
Separate transformation language: xslt,
- fine if you data happens to be xml, otherwise you need another conversion to create data in the correct format.

Web Dev and Python

A myriad of web application frameworks:

ZOPE, Django, TurboGears, WebWare, web.py, pylons, ...

Because it's so easy to build one in python!

TurboGears

TurboGears is a web app framework subscribing to the Model View Controller paradigm:

Object Relational Mapper provides native python objects to manipulate data in RDBMS.
the templating system accepts native python objects for view (i.e. reponse body) creation.
controllers are the request handlers, i.e. python objects which are published.

TurboGears embraces a best-of-breed philosophy of using/integrating external components.

URL Mapping in TG

path components are used to determine the object to handle the request,
additional path components are passed as positional parameters,
parameters from the query string are passed as keyword parameters,
form parameters of type file are passed as file objects.

Anatomy of a TG App

new-app/
|-- README.txt
|-- dev.cfg
|-- new_app
|   |-- config
|   |   `-- app.cfg
|   |-- controllers.py
|   |-- json.py
|   |-- model.py
|   |-- static
|   |   |-- css
|   |   |   `-- style.css
|   |   |-- images
|   |   |   |-- favicon.ico
|   |   `-- javascript
|   |-- templates
|   |   ...
|   `-- tests
|       ...
|-- new_app.egg-info
|   ...
|-- sample-prod.cfg
|-- setup.py
`-- start-new_app.py

TurboGears provides a place for

your configuration,
static content,
your model,
controllers,
templates

and mechanisms to access all the above conveniently.

The Request Cycle

In the easiest case, request handling for the programmer means:

receiving input parameters in an exposed method/function,
doing stuff,
returning a dictionary with data for the view.

What TG does behind the scenes:

starting a db transaction,
object traversal,
invoking the controller,
commiting the db transaction,
invoking the appriate renderer,
delivering the rendered output,

Where's the HTTP?

Request headers:

inspect the request object.

Response creation:

set response status code by:
```
response.status = 200
```
create response headers by assigning to response.headers, e.g.:
```
response.headers['Content-Type'] = 'text/plain'
```
create reponse body by returning the body as string from the controller method.

Ajax

var getData = function () {
  var req = getXMLHttpRequest();
  req.open("GET", '/data');
  var d = sendXMLHttpRequest(req);
  d.addCallback(evalJSONRequest);
  d.addCallback(showData);
  return false;
}

var showData = function(data) {
  document.getElementById('id').innerHTML
  = data
}

@expose(template=package.templates.view.kid)
@expose(format='json')
def data(self, *args, **kwargs):
  return dict(args=args, kwargs=kwargs)

Client side is supported via mochikit - a javascript library that comes with TG.
Server side is supported via TurboJson - a converter to serialize python objects as json.
Controllers can be used for multiple view formats using multiple @expose decorators. Compare RenderedAsHtml and RenderedAsJson.

tg-admin

quickstart a new application: tg-admin quickstart
administrate the database: tg-admin sql
add commands yourself: tg-admin mycommand
- to run tests,
- to initialize the database with data,
- etc.

Easy Development

devlop with sqlite, deploy with postgres.
autoreload of packages while developing.
distribute/deploy apps as eggs.
the templates are valid xhtml, thus can be looked at in a browser.

Tools

You want to have FireFox (2.x) and the following extensions:

web developer toolbar
firebug
live HTTP headers

Summary

TG does a lot for you
but still lets you get down to raw HTTP easy enough.

Web Development with Python

MPDL September 12, 2007

Web Development with Python

Introduction

What's the Web?

Ok, so what's HTTP?

Statelessness

URLs

MIME Types

The Spirit of HTTP

Do I need to know HTTP?

What's a webb app?

Web App History

HTTP server

CGI

Web App Frameworks

URL Mapping

Templating

Web Dev and Python

TurboGears

URL Mapping in TG

Anatomy of a TG App

The Request Cycle

Where's the HTTP?

Ajax

tg-admin

Easy Development

Tools

Summary