IronPython 1.0 RC1 released

It was announced on the IronPython mailinglist today, that the first release candiate of IronPython 1.0 is available for download. IronPython is an implementation of Python written in C# and designed to run on an implementation of CLI like Microsoft .NET 2.0 CLR or Mono 1.1.16.1+. The goal of the IronPython team is for 1.0 is to be compatible with CPython 2.4. But they have included some features from CPython 2.5 that are enabled with the -X:Python25 commandline switch. These include PEP 308: Conditional Expressions and PEP 343: 'with' statement.

As well as many bugfixes, the other positives for me are:

• In some cases code execution is faster, and it launches 2.5 times faster.
• Since I do alot of my IronPython work with Mono, I can now finally exit using Control-D

If you are in Sydney this Thursday 27 July, why not come to the Sydney Python Users Group meeting and hear me talk about my IronPython experiences.

Overriding IronPython's built-in modules

For those who haven't used IronPython, batteries are not included. After installing IronPython, you need to copy, symbolic link or update sys.path with the CPython standard libs if you want to do more than script .NET/Mono classes. As the various beta's of IronPython have been released, some of the standard library modules are being coded in C# and included in the IronPython Assembly. The socket module is one of these. But as of Beta 9, the socket module does not implement the getaddrinfo function. Many Python network modules expect this function and I was having trouble with a script I was trying to port because of it. But I knew that Seo Sanghyeon had written a IronPython socket module prior to it being included as a built-in and it had getaddrinfo. Since the calls to socket were in other third party python modules, I couldn't just rename socket.py and import it the new name. I needed to override the built-in but how? Then I remembered a post by Fuzzyman about including Python code in an assembly. So by using the code in the top of my script I was able to use the python version of socket.

import imp
import sys

moduleName = 'socket'
moduleSource = './Lib.ip/socket.py'
newModule = imp.new_module(moduleName)
execfile(moduleSource, newModule.__dict__)
sys.modules[moduleName] = newModule

import socket

Of course, there may be a better way.

SyPy Meetup Reminder Thursday 27 July

The Sydney Python group is having its first meeting for the year on Thursday July 27.

Usual time and new place:

Thursday, July 27, 2006 (6:30 PM - 8:30 PM)

The "new" University of Sydney School of IT Building.

Thanks to Bob Kummerfeld for arranging this.

The venue is approx 1 km from both Central and Redfern stations.

Use the entrance from the University side, not the Cleveland St side. If you come from City Rd, enter the Seymour Centre forecourt and follow the curve of the new building down to the foyer entrance.

http://www.cs.usyd.edu.au/~dasymond/index.cgi?p=Map

Please reply to this message (or click the appropriate radio button on http://upcoming.org/event/89388) if you will be coming.

Talks:

Graham Dumpeton on what is coming in the next major version of mod_python (3.3). This version of mod_python should represent a significant improvement over previous versions in certain areas with ramifications on stability. New features have also been added which make mod_python a bit more flexible than it is now and more useable in the way that Apache modules should be able to be used. Result is that mod_python can truly be used for more than just a jumping off point for stuff like WSGI and all those Python web frameworks that
keep popping up every day.

I will be giving a talk on my experiences in using IronPython with .NET and Mono.

The talks will be 15-20 minutes in length with plenty of time for questions.

See you there.

IronPython and the moving API

Last week the IronPython team released IronPython .0 beta 9. This should be the last beta before the release candiate.  Also  they  say they have finally locked down  the Hosting API. This has an impact on what I have been working on, which involves hosting the interpreter within an ASP.NET handler. Thankfully the final Hosting API has returned to a more pythonic API rather than imho, the ugly API of beta 8.

With each beta release, there seem to be just enough changes in how to do things that writing an article on IronPython that will work with later beta's is a challenge. Even the IronPython team have been having trouble keeping their tutorials that ship with the beta in sync.

For example, on the IronPython mailinglist, Michael Foord pointed out a link to an article on IronPython. It is well written and has some interesting code examples that could attract a .NET programmer to give IronPython a go. I am not sure when the article was written but at least one code example doesn't work as a python list is no longer automatically converted to an array when passed as a argument to a .NET class. So to get this example to work for beta 9:

import clr
clr.AddReference("System.Windows.Forms")
from System.Windows.Forms import *

class MyForm(Form):
    def __init__(self):
        Form.__init__(self)
        
        Button1 = Button()
        Button1.Top = 10
        Button1.Left = 10
        Button1.Text = "One"
        
        Button2 = Button()
        Button2.Top = 50
        Button2.Left = 10
        Button2.Text = "Two"

        ctrls = [Button1, Button2]
        self.Controls.AddRange(ctrls)
        
f = MyForm()
Application.Run(f)

You need to modify the line:

ctrls = [Button1, Button2]

to:

ctrls = System.Array[System.Windows.Forms.Control]( (Button1, Button2) )

You will also need to add

import System

after the

import clr

line.

The author of the article provides the above syntax as an alternative solution, but as of beta 9 it's the only solution. I can understand why these changes have needed to happen so late in the beta programme, but it's a shame we miss a chance to bring more programmers under the spell of Python because the code didn't work.

What does the World Cup Football and OSDC 2006 have in common?

While watching the second half of today's final as Italy and France played towards a penalty shootout, I made the most of my time and submitted my paper proposal for OSDC 2006. If it is accepted I will be speaking about using Agile Languages in this case, IronPython and Boo with Mono.

Alan Green has submitted one, now just need Andy Todd to submit one, and the gang of 3 could be all speaking at OSDC again.

Another weblog to maintain

To not pollute this blog's Atom feed with too much detail on my CLI adventures, I have created a new blog.

Now if blogger had category filters, I wouldn't have to do this. Of course, I could stop mucking around and get a domain name, find a web host and do my own.

CLI - a journey of discovery

My open source interest has been focused on CLI for the last month, not the Command Line Interface but the Common Language Infrastructure. Thanks to mono there are tools to develop cross platform, cross language applications. As I am discovering the reality is a little different. So expect a few entries in this blog documenting the challenges, disappointments and victories as my journey of discovery continues. Maybe it's a good topic for OSDC 2006 paper.

Today I am an Australian

After living in Australia for the last 7 years, marrying an Australian, and creating a new little Australian, last night with 80 others, I attended my Australian citizenship ceremony. So another phase in my life begins.

OSDC 2006 Call for Papers

The Open Source Developers Conference is happening again this December in Melbourne, Australia. The last two conferences were excellent. Of course what makes a great conference is the people that attend and the content. The Call for Papers is now on.

I hope to attend this year but not sure if I will submit a paper proposal.

SQLAlchemy, more than an ORM

I remember seeing references to Michael Bayers' SQLAlchemy and thinking yet another Python ORM to look at sometime in the future. I have used SQLObject for various hobby projects, but since I deal with legacy databases (some database schema's just do not map well to an ORM) at the day job, I have tended to use SQL and the Python DBI for my database access work. Part of my job is to evaluate solutions to make my team more productive, and in reviewing where we allocate our time during development, I noted that a good portion is ensuring our SQL works across multiple RDBMS's. So I have been looking for Python solutions that abstract the creation of SQL against different RDBMS's. On investigating SQLAlchemy I was surprised to see that it you can use it to do exactly what I wanted. Also one of goals of Ian Bickings' SQL-API is to offer this functionality, but what follows is a discussion of how SQLAlchemy could help.

First connect to a database

from sqlalchemy import *
engine = create_engine("sqlite://filename=mydb",echo=True)

The option echo is set so we can view the generated SQL.

Now describe the tables using using table metadata objects

entries = Table('entries', engine,
        Column('entry_id', Integer, primary_key = True),
        Column('title', String(255), nullable = True),
        Column('content', String(), nullable = True),
        Column('status', String(10), nullable = False),
        Column('creation_date', DateTime()),
        Column('publish_date', DateTime())
    )

comments = Table('comments', engine,
        Column('comment_id', Integer, primary_key = True),
        Column('entry_id', Integer, ForeignKey("entries")),
        Column('comment', String(255), nullable = False),
        Column('submitter', String(60), nullable = False),
        Column('submit_date', DateTime())
    )

# Create the tables
entries.create()
comments.create()

Now load some data.

entries.insert().execute(entry_id=1,
            title="My first blog entry",
            content="A blogging I will go.....",
            status="published",
            creation_date="2006-04-17 08:15:30",
            publish_date="2006-04-17 08:25:00")

entries.insert().execute(entry_id=2,
            title="My second blog entry",
            content="Another day, another blog entry....",
            status="draft",
            creation_date="2006-04-18 09:35:30")

comments.insert().execute(comment_id=1,
            entry_id=1,
            comment="The entry needs more substance.",
            submitter="joe.blogger@home.net",
            submit_date="2006-04-17 09:05:00")

comments.insert().execute(comment_id=2,
            entry_id=1,
            comment="I disagree with the first comment.",
            submitter="jane.doe@rip.net",
            submit_date="2006-04-17 09:05:00")

Get some data with a simple select with where clause and order by

cursor = entries.select(entries.c.status=='published',
                order_by=[entries.c.publish_date]).execute()

rows = cursor.fetchall()
for row in rows:
    # Get column data by index
    title = row[1]
    # by column name
    content = row['content']
    # by column accessor
    status = row.status

Now a simple join

cursor = comments.select(comments.c.entry_id==entries.c.entry_id).execute()

which generates the following SQL:

SELECT comments.comment_id, comments.entry_id, comments.comment, 
comments.submitter, comments.submit_date
FROM comments, entries
WHERE comments.entry_id = entries.entry_id

Of course, this quick overview only scratches the surface of what you can do with SQLAlchemy. It also has support for outer joins, subqueries, unions, updates and deletes. See here for more in depth documentation.

Pylons - another great tool for my programming toolkit

In a previous post I mentioned that I had been looking at Pylons, a WSGI enabled web framework and had a project that would be a good fit. Finally have found some time to start work on it and my experiences with Pylons so far have been good. I picked Pylons for a number of reasons:

• WSGI enabled
• Uses Paste
• Not aligned with any particular ORM
• Support of different template packages via Buffet
  
• Reasonable documentation to get you started
  

I originally had concerns about learning yet another template syntax - Myghty, Pylons default template package, but have found it easy enough to learn and use. So one of the reasons I choose Pylons - support of different template packages really has no substance. But some of the other things I have discovered while using Pylons certainly should be added to the reasons list. Both of them are re-implementations of Rails functionality.

• Routes - makes it easy to create nice and concise URL's like http://timesheets.com/user/fred-smith/edit. No more ugly http://timesheets.com/user-edit.html?name=fred-smith
  
• WebHelpers - functions that simplify web development with template languages by providing common view patterns in re-usable modules. In addition to the Rails webhelpers, there helpers for HTML generation and pagination for collections and ORMs.

And if you are not interested in using Pylons, both these packages can be used standalone.

As I progress with my project, I am sure there will be more posts about my Pylons experiences.

Getting ISAPI-WSGI from Subversion Repository

Had an anonymous email asking how to check-out isapi-wsgi from subversion. The following command will do it:

svn co http://svn.isapi-wsgi.python-hosting.com/trunk

Hex Dump Tools written in Python

In reviewing my blog access stats, it would appear that most used search engine keyword combination that sends visitors to my site are "hex" and "dump". So to not disappoint these 20% of visitors, I have tracked down a number of hex dumping tools written in my favorite programming language.

READBIN by Tony Dycks is a Text Console-based program which reads a single Input File specified on the command line one character at a time and prints out a formatted hex "dump" representation of the files contents 16 characters per display line. A prompt for continuation is issued after displaying 20 lines (320 characters of information). An entry of "X" or "x" followed by the key terminates the program execution. Any other entry followed by continues the display of the formatted hex and character information. A "." character is used for any non-displayable hex character

Hex Dumper by Sébastien Keim is a function which will display to stdout, the classic 3 column hex dump of a string passed to it.

Hexdump by Ned Batchelder prints a 3 column hex dump to stdout of a list of files or stdin.

What's been happening in the world of WSGI

Good to see some activity in the development of WSGI component based microframeworks.

I have been following Ben Bangert and James Gardners' work on Pylons which is based on Myghty, with a custom Resolver, full Paste and WSGI integration. Haven't had a play with it yet, but have an idea for a project where Pylons may be the perfect fit.

Also Julian Krause has just released RhubarbTart, a light object publishing web framework built on WSGI and Paste. It's object publishing model is similar to CherryPy. Since it built on Paste, RhubardTart makes the most of Paste's selection of WSGI middleware components to provide the other functionality expected of a web framework. So if have been using CherryPy because of it's ease in exposing object methods, but want to customise the component stack you use for a web project, have a look at RhubarbTart.

Then again, if you want to create your own WSGI/Paste framework, Ian Bickings has created a great tutorial to get you started. If you have a problem with understanding what WSGI/Paste is all about, read the tutorial, you will be enlightened.

And even Guido this week has nice things to say about WSGI - 'Maybe the current crop of Python web frameworks (as well as Rails BTW) have it all wrong. Maybe the WSGI folks are the only ones who are "getting" it.'

One of the joys of programming in Python is "batteries included" and the fact I get to pick which batteries to use to help me solve the programming task at hand. With WSGI I get the freedom to pick what components I want use in the HTTP request/response area for a web project, and get to focus on solving the actual programming problem or challenge.