Merry Christmas

Heading off on computer free Christmas holidays tomorrow. Back to my homeland of New Zealand and looking forward to crayfish and beef on a spit for Christmas dinner. So just wanted to post a greeting now. Thanks to all the people who have read my blog, the members of the open source community who have helped me and have provided feedback. To those who celebrate Christmas, have a great one with family and friends.

Recharged after OSDC

After attending OSDC last week, I came back energised and wanting to get some more open source stuff done.

So apart from getting Roundup to work with WSGI, I have been busy integrating a newly released version of informixdb into the Zope Informix DA and SQLObject. Informix is our development database at work and for any *ix Python DBI access we have been using mxODCBC as the previous release of informixdb had suspect threading support. The new version can be compiled with or without thread support and uses distutils to do the compile. The use of distutils alone makes it worthwhile upgrading as anyone who has tried to get the old informixdb to link has a horror story to tell. It also supports the newer data types of IDS 9 and 10.

The Zope Informix DA work is done and is currently being tested prior to release.

The experience of creating an Informix data connection for SQLObject has been interesting. I was surprised to discover that SQLObject creates insert and update SQL statements as complete strings including the data values rather than using placeholders. This created a problem for me with TEXT BLOBS as the informixdb did not translate from a string but required a buffer. The various solutions I tried can be viewed here. As none of the solutions were really perfect, I decided to request an enhancement to informixdb so it would do the string to BLOB translation automatically and the Open Source community delivered yet again. In less than 24 hours, Carsten Haese had provided a patch that did exactly what I wanted.

It had been my intention to get all these changes tested and checked in prior to going on Christmas holidays but as usual my free time as been limited with other important things like family. So looks like it will have to wait until January.

OSDC 2005 - My WSGI Talk Errata

I have updated the copy of the paper to include more info on using paste.deploy pipelines. This info was presented at the conference but wasn't part of my submitted paper due to time constraints.

OSDC 2005 - My WSGI talk

Due to lack of an Internet connection during the conference (my issue not OSDC's), I have not had a chance to post anything about the conference until now. On Monday 5 December I gave my talk on WSGI. Not really sure how it went across as no one asked any questions at the end or really talked to me about it during the rest of the conference. Of course maybe it was so good, everyone now fully understands WSGI, but that is very unlikely. Anyway the paper can be found here and the only photographic evidence of my talk can be seen here.

In the closing keynote "Tools for Freedom", Jeff Waugh made an interesting observation about how a user of our software doesn't really give a damm about the technical aspects, they just want it to make their life easier. That got me thinking about how to improve the uptake of WSGI. As much as TurboGears can use WSGI, most developers will just run the standard CherryPY server. Maybe when they start thinking about production deployment, then WSGI may be used as the "making life easier" solution. So for the next couple of months I hope to focus on making some existing python apps WSGI enabled so the users of these apps will find deployment easier. So during breaks in the conference, I wrote an adaptor to run Roundup as a WSGI app. Fixed the last of the problems on the plane to Sydney this morning and once I have fully tested it and tried it at work for one of our existing Roundup tracker sites, I will look at checking it in to the Roundup CVS.

Not sure, about the next Python app, any suggestions?

Deploying ViewCVS with WSGI

In my continuing expermentation with Python Paste I came across the cgiapp module. The module provides a class that acts as a proxy to an existing CGI script, which means you can take an existing CGI script and run it using a WSGI enabled server. Rather than testing it's functionality with a simple cgi script, decided to see if I could run ViewCVS with it. To get it to run was very simple.

Create a app factory wrapper script called viewcvs_wsgi.py

from paste import cgiapp

def app_factory(global_config, **local_config):
 
 viewcvs_app = cgiapp.CGIApplication(global_config, script, path=None, include_os_environ=True, query_string=None)
 return viewcvs_app

Create a Paste Deploy configuration file called viewcvs.ini

[app:main]
paste.app_factory = viewcvs_wsgi:app_factory
script = "/usr/local/viewcvs-0.9.4/cgi/viewcvs.cgi"

[server:main]
use = egg:PasteScript#wsgiutils
host = 192.168.1.1
port = 8082

Start the WSGIUtils webserver using paster

paster server viewcvs.ini

Point a browser to the URL and browse the CVS repository.

Apart from images not displaying everything else worked as normal. Now why would I want to do this? Well it means I can now run viewcvs under IIS as an ISAPI extension using my isapi_wsgi adaptor. But thats a future blog entry.

SyPy Meetup Last Night

Last night was the last meeting of the year for the Sydney Python group. Andy Todd did a test run of his OSDC presentation on PythonCard. Apart from having to cope with using a Windows laptop instead of his beloved iBook, it was very interesting introduction in how to use PythonCard. I gave a presentation on Python Paste and hopefully, a few more know now what it is. The discussion on Paste.Deploy prompted Graham Dumpleton to give an impromptu presentation on an extension for mod_python he is creating called lamia that as one of it's features uses Python as it's configuration language.

Confirming that Python programmers are happy to look at other languages Alan Green gave us a look at Groovy, the new Java scripting language.

As with all the meetups this year, lots of other interesting conversations and looking forward to next year. Thanks to Alan Green for organising all this years meetings.

Deploying a WSGI app with Python Paste

As part of my research of the current state of WSGI for my paper for OSDC2005 I revisited PythonPaste. Ian has added lots more functionality since I last looked and I wanted to use Python Paste as the "glue" for a demo of creating a web app from WSGI middleware components. As with most stuff Ian does, there was enough documentation to get me going or so I thought. In reality, it took me a lot longer to get things working than I originally planned. This is no reflection on Ian's docs, but more on me not "getting it", and trying to make the solution complex. So what follows is a little tutorial on how to deploy a very simple WSGI app using Paste.Deploy and Paste.Script under Ubuntu 5.10 Linux and Python 2.4. Also this tutorial only uses a very small subset of Paste functionality.

0. If you have not installed any Python packages using easy_install before, you will need to download it and install it following the instructions at this site.

1. Install paste, paste.deploy and paste.script using easy install

sudo easy_install http://cheeseshop.python.org/packages/2.4/P/Paste/Paste-0.3-py2.4.egg
sudo easy_install http://cheeseshop.python.org/packages/2.4/P/PasteDeploy/PasteDeploy-0.3-py2.4.egg
sudo easy_install http://cheeseshop.python.org/packages/2.4/P/PasteScript/PasteScript-0.3-py2.4.egg

Also install WSGIUtils so we have a WSGI server to run our WSGI app under.

sudo easy_install http://pylons.groovie.org/files/WSGIUtils-0.6-py2.4.egg

2. Create a simple WSGI app in a file called hello_wsgi.py

def application(environ, start_response):
    """Simple WSGI application"""
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n']

def app_factory(global_config, **local_config):
    """This function wraps our simple WSGI app so it
    can be used with paste.deploy"""
    return application

3. Create a paste deploy configuration file called config.ini

[app:main]
paste.app_factory = hello_wsgi:app_factory

[server:main]
use = egg:PasteScript#wsgiutils
host = localhost
port = 80

The first section app:main is where we define our application, or more simply tell paste.deploy where the code for our WSGI app can be found, in our case in the file hello_wsgi.py and by calling function app_factory.

The second section server:main tells paste.script what WSGI server to use. In this case we will be using the simple WSGI server that comes with WSGIUtils.

4. To serve the app we use the paster script that was installed as part of Paste.Script. So enter the following at the commandline.

export PYTHONPATH=.   # Ensure our hello_wsgi.py module can be imported by Python
sudo paster server ./config.ini

On the console, a message will be displayed similar to:

Starting server in PID 9582.

5. Enter the URL for the web app in a browser and hopefully you see hello world displayed.

6. Replace the boring hello world app with a more useful WSGI app you are now inspired to create.

OSDC Proposal Outline

It would appear that you need a login to view my proposal. So here is a copy.

WSGI - Gateway or Glue?

Short Description: Present a review of where WSGI currently is and describe/show how it is being used to create a framework-neutral set of components.

Stream: Python

30 minute Paper Presentation in English


Abstract
The Web Service Gateway Interface is an API defined in PEP 333 that allows web servers to talk to Python web applications. At the time of OSDC 2005, the PEP will be 2 years old and the presentation will review the state of WSGI implementations for the various web servers including Apache, IIS, Twisted etc. It will also show how the API is used to implement a web server interface and create an application. In the last 6 months, the main WSGI community focus has been on creating web applications by composing them from components that communicate via WSGI. The remainder of the presentation will discuss this and show what can be done.

OSDC Proposal Accepted

"Your proposal has now been reviewed, and I am pleased to be able to inform you that it has been accepted for inclusion in the conference program" so it is now time for me to gather together all the ideas that have flowing around in my head and create a coherent paper from them. The proposal description can be found at http://osdc2005.cgpublisher.com/proposals/24

After looking at the other accepted proposals, I know this is going to be an even better OSDC than last year. Also as mentioned in a previous post there is a much better balance in the "P" scripting languages than last year ;-)

So if you want to spend an interesting 3 days in Melbourne in early December, why not goto the Open Source Developers Conference

Argh, having to learn java again

After managing to keep myself employed for almost the last decade without having to program in Java, a project at work means that the end is nigh. Pulled the "Teach yourself Java in 21 days" out of storage and noted that it was published in 1996. So Java and I have kept our distance for a long time. Also means I need to get myself up to speed with what has happened to Java over the period. In the end, I just need to learn enough to embed Jython 2.1 within an existing Java front-end so I can access it's API, well that's the theory anyway. Have spent the weekend consuming lot's of articles from the web and playing in eclipse. Have lot's of respect for what "real" Java programmers are doing out there, but hard to get back into writing lot's of code to achieve something. Python has spoilt me.

OSDC Proposal Submitted

Due to the "massive" response from my readers (thanks Richard) I went with WSGI and submitted a proposal. As of last night, it looks like they have 55 proposals of a wide and interesting range of topics. It would also appear there are more Python than Perl proposals, but I had my rose coloured glasses on, so I may be mistaken.

Goodbye Windows, Welcome Linux to my home

At work I use Windows XP as my workstation because that's what our clients use to run our frontend. Most of our development work is done on servers running Linux and truthfully the most used program on my XP box is ssh. But at home, I have always run a Microsoft OS. Of late, I have been questioning why. The home machine is used for paying the bills via the Internet, reading email, browsing the web, updating this blog and the odd bit of wordprocessing. In the past, it was also used to play games, but the XBox has taken over that role. Based on the last couple of months, the home machine seems to spend most of it's time downloading service packs or the lastest anti-virius update. So I decided to give Linux a chance to prove that it could do my home tasks. Using a "Live" CD version of Ubuntu 5.04, I have survived two weeks without having to see a Microsoft logo. Another interesting thing I have discovered is thanks to GMail etc, that I haven't needed any local hard disk storage.

Database design for newbies

Found this whitepaper on database design by Ian Bicking & Brian Moloney. A good explanation of Dr Codd's rules but in a language a newbie could understand.

How fast a year goes by, OSDC decisions again...

It doesn't seem that long ago that I was considering presenting a paper at OSDC 2004. But I did, and here we are a year later in the same situation. I really enjoyed last year's conference, both giving my presentation, meeting many interesting people, and deep discussions over many beers each night. I want to submit a proposal for a paper but still haven't decided what I want to talk about. Currently I have narrowed it down to 3 topics:

• WSGI - Present a review of where it currently is and describe/show how it is being used to create a framework-neutral set of components.
  
• A Smart Client using Python - Microsoft is pushing 2 types of user interface for an n-tier application, a web client and what they refer to as a "smart" client. In M$'s case, it is written using WinForms and other .NET technologies. I want to produce something similar using Python. Have been playing with both wxPython & PyXPCOM/XULRunner before deciding on the UI toolkit to use.
  
• ActiveGrid - This is an open source framework for building Web 2.0 using LAMP. Think it's more a candiate for a lighting talk.

Will have to make up my mind soon, as proposals have to be in by Friday 19 August.