Blog of Note?

Today this blog is Bloggers "Blog of Note". Well that explains the abnormal traffic statistics.

mod_wsgi Performance

Graham Dumpleton has published his benchmarks for mod_wsgi.

mod_wsgi - simple and fast apache wsgi serving

There are a many ways to serve a WSGI application via Apache with FastCGI appearing to be the favorite. If you have to support many diverse operating systems running different versions of Apache, serving a WSGI application via CGI is a guaranteed but not fast deployment method. You may be wondering why I haven't suggested the WSGI mod_python adaptor as a solution. I am sure it is a workable alternative but it requires mod_python and I have had lots of issues getting it to compile for some platforms and to work with a specific Python interpreter. Now there is a new solution.

Graham Dumpleton has been working on an Apache module called mod_wsgi. Quoting from his website:

"The mod_wsgi adapter is an Apache module that provides a WSGI compliant interface for hosting Python based web applications within Apache. The adapter is written completely in C code against the Apache C runtime and for hosting WSGI applications within Apache provides significantly better performance than using existing Python based WSGI adapters for mod_python. The module can be compiled for and used with either Apache 1.3, 2.0 or 2.2."

I have been lucky enough to get early access to the code and try it with my WSGI applications. I able to compile, configure and have mod_wsgi running a simple WSGI application under Apache 2.0.55 and Python 2.4.3 in under 5 minutes, Certainly a much better experience than I have had with mod_python. Since I have been testing it on my "gutless" laptop rather listing the results of my benchmarking, I will leave Graham to publish his more comprehensive benchmarking results. But I will make some observations. It appears to run simple WSGI applications at least twice as fast as mod_python and 15-20 times faster than using CGI. I was also able to run a number of my WSGI applications that use either Pylons or combinations of WSGI middleware with no problems. At yesterdays SyPy meeting, Graham showed it running Django. The official release will hopefully be in March.

SyPy Social Meetup

Last night was the second meeting of the Sydney Python Users Group for 2007 and we had a good turnout.

There was a wide range of topics discussed including:

• Unit Testing
• Software Deployment
• Distributed Version Control
• Apache and WSGI - more about this in another post
  

So if you are in the Sydney and are interested these types of topics, why not join us the first Thursday of each month. Details of the next meeting can be found at:

http://groups.google.com/group/sydneypython/web/sypy-home

Hope to see you in March.

LCA 2007 My Day 3

Yesterday was the last day of linux conf au. As usual it was the mad rush from work to get to my first session, at least this time I wasn't late. The session was Robert McQueen's "Telepathy - Unifying IM, Voice and Video Communications". This was very interesting and after seeing the video demo using a Nokia N800, apart from wanting one, I will be keeping an eye of this project.

The next was Andre Pang's "Erlang and Concurrency" talk. Andre is an excellent and entertaining speaker. This talk investigated everyone's dislike or fear of threading, and then explained the advantages of Erlang's message passing distributed model. Every year I try to learn a new language, maybe it will be Erlang this year or maybe I should just look at Stackless Python.

After lunch, it was Rob Collins, Erik de Castro Lopo, and Peter Miller's "Do it our way" presentation. After attending their tutorial on Thursday, I was looking forward to this talk to hear about the other testing patterns they feel we should be using. Learnt a little bit more and looking forward to the release of the paper. They also intend to create a website to allow sharing of these testing patterns.

The last session I attended was Valerie Henson's "Choosing and Tuning Linux File Systems". Picked up a couple of interesting things, but the main take home of the talk was "use ext3 unless you know you need something else".

The day for me ended with the closing ceremony. Two of the conference attendee's went home with a olpc after being selected based on their submissions for a olpc software project proposal. Others who submitted a proposal were told not too cry as it was likely that a olpc was coming there way as well. I intend to have a play with the olpc emulation soon.

Thanks to the 7 team for a great conference and to all those who took the time to discuss OSS stuff with me.

IronPython and Traceback line numbers in Mono (workaround)

One of the reasons why I need to have access to a Windows dev environment when doing my IronPython/Mono development is the lack of line numbers in IronPython tracebacks under Mono. But via Seo Sanghyeon comes a workround. He has discovered that that Mono can't display line numbers from AssemblyBuilder built in memory, but only from Assembly loaded from disk. So you need run mono in debug mode and get IronPython to compile and save the assemblies to disk. So if we have a buggy script like:

a = 1
b = 0
c = a/b

Prior to the workround the traceback would have been:

Traceback (most recent call last):
  File foo, line unknown, in Initialize
ZeroDivisionError: Division by zero

But if I start IronPython using:

mono --debug ipy.exe -X:SaveAssemblies foo.py

you get a more helpful traceback

Traceback (most recent call last):
  File /u1/ip/IPCE/IPCE-svn/Lib/site.py, line 33, in Initialize
  File /u1/ip/IPCE/IPCE-svn/Lib/fepy/__init__.py, line 7, in install
  File /u1/ip/IPCE/IPCE-svn/Lib/fepy/__init__.py, line 11, in install_option
  File /u1/ip/IPCE/IPCE-svn/Lib/fepy/ast.py, line 17, in Initialize
SystemError: Object reference not set to an instance of an object
Traceback (most recent call last):
  File /tmp/foo.py, line 3, in Initialize
ZeroDivisionError: Division by zero

Thanks again Sangyheon.

LCA 2007 My Day 2

Today was day 4 of linux conf au, and my second day of attendance.

My first session today was Peter Miller, Erik de Castro Lopo, and Robert Collins tutorial "The Hao of testing". Since I have been to work before heading to the conference, I arrived at the tutorial a couple of minutes late to discover there had been homework. So a mad rush to get connected, gmp downloaded and built. Then I needed to write a unit test for C program, yuk, I was beginning to question my decision to attend. But once I started listening to what they had to say, I got over my initial horror and enjoyed the talk for it's lessions. And the last part of the tutorial was actually unit testing part of a python application. Looking forward to their session tomorrow where they will present a paper on what was discussed today and alot more.

After lunch I went to Malcolm Tredinnick's "Building Websites with Django" tutorial. Since I already have used Django, I attended this tutorial to see how non users picked up the framework and maybe learning some news things. Malcolm warned us that he was presenting something that normally took 3 hours but would try to condense it into the 1.5 hour timeslot. I feel he spent too much time explaining where Django came from and what people had done with it, rather than showing how to use the framework. There was no actual "Wow" factor until half way thru the session, but by then a third of the attendee's had left for other sessions. The last half of the tutorial was much better and I did learn a couple of new things. The slides and the example app for the tutorial can be found here.

The last session for my day was Ralph Giles "Screencasting Howto" tutorial. The aim of the tutorial was for all of us to have created a screencast and have it peer reviewed. Only a few of us managed to achieve this due to issues with getting the various tools working but there a good interaction between the attendee's and I learnt alot. One of the tools we used was Istanbul, a desktop session recorder for the Free Desktop which is written in Python and uses GStreamer. It's a little buggy and many of us didn't have much success until we dropped the recording frame rate to 1 fps.

Many of the sessions have been videoed by the LCA team and can be accessed from the Programme web page.

It was also the conference open day but I didn't get to see much as it was well attended (read as overcrowded), but this can only be good for OSS community.

LCA 2007 My Day 1

Today was day 3 of linux conf au, but it was the first day I attended and only got there just before 2pm. As I wanted to hear Michael Carden's talk on Digital Preservation, I headed to the lecture room where it was scheduled. As I was one session early, I listened to Rafael Espindola's talk - The ARM backend of LLVM.

Then it was Michael's talk - Digital Preservation - The National Archives of Australia, Open Standards and Open Source. He is a very entertaining speaker and gave a great overview of what they have developed at NAA. It was interesting to discover the actual physical setup of the NAA digital archive is 3 separate systems:

• Quarantine - where the digital objects submitted are virius scanned, then stored for 28 days and scanned again.
• Preservation - where the digital objects media type is identified, then object is preserved as the original is base 64 encoded with metadata and as an open document format with metadata. This is all done by Xena.
  
• Digitial Archive - actually 2 systems, one running Windows Server OS and the other running Linux. Each system is connected to a RAID arrays from different vendors.
  

The most interesting thing about the physical setup is there is no network connection to the outside world and each system is protected by an "Air Gap Firewall". In other words, all data transfer between the system is via a data carrying device (USB Portable Disk Drive). I know it sounds strange but the reasons Michael gave were valid.

The final session I attended was Stewart Smith's - eat my data: how everybody gets file IO wrong
Again a very entertaining speaker, and there was alot of attendee participation. The gist of the talk was never assume the data you have just written has made it to the platter. The various issues were explained and solutions given. Maybe I should start using the XFS filesystem.

So looking forward to tomorrow.

Need to find time for LCA 2007

Linux conf 2007 is on this week and I have paid to attend. The only problem will be finding time to go, otherwise it will be the most expensive t-shirt I have ever purchased. The urge to attend LCA was induced while attending OSDC 2006 and meeting a number of interesting people who are speaking at LCA. So hopefully will be able to attend:

Digital Preservation - The National Archives of Australia, Open Standards and Open Source

Eat my data: how everybody gets file IO wrong


The Hao of testing


Concurrency and Erlang


Do it our way

My OSDC 2006 Paper

At this years OSDC, I did a presentation titled Development of Mono Applications with Agile Languages. The agile languages I discussed were Boo and IronPython. I have presented at the last three OSDC's and this years topic generated the most feedback and post presentation discussion for me. The paper published in the proceedings can be downloaded from here.

ISAPI WSGI SVN has moved

Due to heavy ticket spam on the old SVN site http://isapi-wsgi.python-hosting.com, it would appear that the isapi-wsgi site has been taken off-line. A couple of months ago I started mirroring the SVN at http://isapi-wsgi.googlecode.com/svn/trunk/ so access to the source code is still available. Thank goodness for backups.

Is Linux ready for Inflight Entertainment?

Yesterday I flew to Malaysia on a MAS 747. It is a 8+ hour flight, but the plane has the individual movie/game consoles in the back of the seat headrest to allow me to catch up on all the movies I normally would never get to see. The only problem was that every 40-50 minutes, my console crashed. Not with the "blue screen of death", but with a Tux image and a crash trace. A few minutes later it would re-boot and the movie would continue. As a geek I do find reading a crash trace entertaining but for non-geeks, maybe not the best introduction to OSS.

More Blogger Beta Woes

Now the compose mode in browser editor has disappeared when creating and editting posts. You only understand how useful something is when it is taken away.

Also the Atom feed generation seems to continue to randomly decide I have modified posts that I haven't.

The joys of beta software.

OSDC 2006 Starts on Wednesday

Flying down to Melbourne tomorrow for OSDC 2006. The program is looking very good and I am sure the topics will be mind expanding. My talk on Mono, Boo and IronPython is midday on the second day of the conference. Based on my experience at the last two OSDC conferences, I have to agree with Andy Todd's statement:

"I expect OSDC this year like last will be full of smart people sharing great ideas, prompting me to make promises to myself to look into things that I won’t have time to keep."

So once I am back from the conference, I need to do the posts I have promised some readers of this blog:

The Status of IronPython and Eggs

IronPython and WSGI

Then I need to find the time for all the other things OSDC will have got me excited about.