Python Meetup in London tonight. See you there!

Python has a module for everything.

Looks like I've been outed.

Loads of stuff in the blogsphere about the Google App Engine, unsurprisingly, some of it even worth reading.

• Ruby fanboys throw their toys out of the pram. You will be assimilated. Mwahahahahaha!
• Python is the new BASIC
• Tomato tracking. "It’s going to be the world’s greatest web application." ;-)
• Django on Google App Engine in 13 simple steps. Nice getting started tutorial.
• Experimenting with Google App Engine. "The lack of SQL is actually refreshing." ... "I am impressed. The App Engine team has done a fantastic job, and I think they have already changed the way I do hobby projects."

So, it's looking good, though I'm still concerned about lock-in. It's not just the Datastore API which locks you in - that's probably fairly east to emulate - but also the Authentication and Authorization API, which looks much trickier.

Update: As Ade says, you certainly don't have to lock yourself in to GAE's Users API.

The Python blogsphere has been exploding with the news of the Google App Engine today. And fascinating it is too - free hosting for Python web apps, Django included. With the might of Google behind it, this will push Python big style. Take that, Rails hype!

I'm a bit uneasy about the datastore, though. Not only does this gut any existing Django app, it also any means that any app that you write that persists data (read - non-toy) will be bound to the GAE API. What's wrong with a real database?

Naturally, I've signed up already. ;-)

Some of our log files are huge - I have a 10 GB file on my HDD right now. These files are unwieldy to say the least, and you usually have a good idea as to the time in which a particular problem occurred, so it's often handy to be able to chop out a specified time range. I keep forgetting how to do this, and re-inventing the process. So, here for my own benefit at a couple of methods.

The hard way

From the bash shell:

wc -l your.log

This will count the lines in your file.

grep -n 12:00: your.log | head -n 1

This will give you the line number of the 1st line containing "12:00:" - in this example, I want log entries starting at midday, so this is the first line that I want.

grep -n 12:10: your.log | tail -n 1

This will give you the line number of the 1st line containing "12:10:" - in this example, I want log entries up to ten past twelve, so this is the last line that I want.

tail -n $[total-first] your.log | head -n $[total-last] > your_focused.log

Replace first, last and total with the values you got above, and you'll end up with a file containing only the time range that you wanted. (If you only want to look at the file once, you can just pipe into less or whatever rather than piping into an output file.)

The easy way

python -c "import itertools, sys; sys.stdout.writelines(itertools.takewhile(lambda item: not '12:10:' in item, itertools.dropwhile(lambda item: not '12:00:' in item, open('your.log'))))" > your_focused.log

Same thing, only this will read through the file just once.

Now, I'm fully expecting someone to come and tell me the real easy way. ;-)

Thanks to Jay for pointing out this rant, via reddit.

Now, I'm sorry they guy can't get a job, but if this is his attitude, I can't say I'm surprised. He's burned his bridges with ThoughtWorks, for a start. ;-)

Disclaimer - I work for ThoughtWorks. But I can say that with perfect honestly that his impression of TW doesn't mesh with mine. Thing is, he seems to think that what TW sells is platform expertise. It's not. We primarily sell smart people. (Oh, and me, too.) A smart person will pick up a new platform quickly. If that's not possible with Rails, that that's Rail's problem - not that I believe that to be the case. Yes, you do need a cadre of people with platform experience for a project to succeed, but not the whole team. (Perhaps he'd have better luck job hunting if he didn't restrict himself exclusively to Rails?)

He says that TW leaves bad code bases behind. Again, in my limited experience, that's not true at all. TWers seem to care deeply about what they do. Anyone else out there who doesn't work for TW who has any experiences they'd care to hare?

Also, his dismissal of TDD and agile practices such as pair programming suggests to me that he's not really tried them. We at TW have a lot of experience in these areas, and it all works just fine for us. If he claims to have improved productivity by dropping them, then he's clearly not comparing like-for-like. Done properly, I fervently believe that these practices improve productivity in the long term (if not in the short term).

I do wish we used more Django and less Rails, though. ;-)

I so need this as a tee-shirt.

And what timing!

This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.



This means you're free to copy and share these comics (but not to sell them). More details.

Busy week - XTC today and London Python tomorrow.

I wasted a good couple of hours today trying to get cx_Oracle running on my Mac. I didn't have any luck - Oracle's Mac client is PPC only. You can still run it in an Intel Mac due to the magic of Rosetta, but you don't seem to be able to compile against it, so these instructions don't work for me.

Anyone got a binary I can snarf?

What if powerful languages and idioms only work for small teams? (via Small teams and big jobs) is interesting. We are running a really large Agile project here at GU; what, 60 odd people? Mention any of the road bumps you hit, and people tell you your team's too large.

That's no good. The team, and the project, are the size they need to be. Splitting them arbitrarily might be possible, but it would carry huge costs of its own. The challenge is to make agile work with a big team. And on the whole, I think we are.

Interestingly, despite the fact that we've made agile methodologies scale up, some people here are still nervous about whether agile languages can do the same. Plus ca change...

So, Annabel was complaining that she couldn't see a log of just her Subversion revisions along with details of the files that have changed from the command line. Shock horror - something you can do from Tortoise that you can't easily do from the shell!

(If you don't care about seeing a list of the files that have been changed, you can just do svn log | grep -A2 sbrunning | less, but adding the -v option turns the output to gibberish.)

Clearly I wasn't going to to let that stand, so I knocked up a very quick script. Then proceeded to over-engineer the snot out of it. (In my own time I must add.)

You can filter by author or by checkin message, with regexps. You can also reverse the output and see the latest checkin last, which is handy if you are only want to see the most recent checkins and don't want to pipe the output to less.

Get it here: svnlog (syntax highlighed - loving that Django TextMate theme) or svnlog (text). Requires Python 2.5. Works on Mac & Linux, probably on Windows too if you give it a .py suffix.

Returning None is Evil. I agree. I've always subscribed to the Samurai Principle myself. If you allow nulls to float around you inevitably end up with NPEs. These can be hard to diagnose properly; not only do you not know if null is actually invalid, but if it is, it's often non-trivial to work out where it came from.

Fuzzyman has a new mocking library for Python, which he presents in Mocking, Patching, Stubbing: all that Stuff.

Michael takes issue with Martin Fowler's Mocks Aren't Stubs; specifically where he defines mocks as objects pre-programmed with expectations which form a specification of the calls they are expected to receive. Michael's mocks are not pre-programmed with expectations - his expectations are defined after the test execution.

Now, to me, this is a trivial distinction - the important difference between a stub and a mock is in the existence of expectations. Whether the expectations are defined before or after the text execution is not crucial - it's still a mock.

It does matter in terms or usability, of course. It feels more natural to Michael to define his expectations after the test object is exercised, along with his assertions. For me, I have to say that defining the mock object's behaviors all in one place makes sense, so both the expected method calls and any return values should defined together - and the return values have to be pre-defined. We are using EasyMock here at GU (along with the Hamcrest constraints library) and I like it just fine. But that's more a matter of taste than anything else.

ThoughtWorks have given me the go-ahead to organise another Python meetup at the London office. Thanks, guys!

Details here - London Python meetup, Wednesday, December the 5th.

Julian and I have been asked to put together some unit and functional test coverage figures. We've not actually been given any time in which to do it, though, so I wouldn't hold your breath.

Looks like some work was put into using Emma to provide these figures, but it's not finished.

Over in the Python world, there's been some discussion over on c.l.py about Ned Batchelder's coverage.py, which looks like a fairly nifty module.

OTOH, I think you have to be a bit careful with metrics such as those provided by these coverage tools. Ned himself points out many of the problems with taking your coverage figures at face value (though Emma's block-level approach fixes some of them).

I'd add one more issue - gaming. People can start to see good coverage figures as an end in and of themselves. If the coverage isn't at, oh, say 80%, the tests must be crap. If they are at 100%, they must be great. It's not just that the figures can be meaningless, they can actually lead people astray.

Now, for an approach to coverage testing that actually makes sure you are testing that your code does the right thing, check out Jester & Pester.

I've never been too intimidated by driving Subversion from the command line, so I've always maintained my branches by hand, keeping note of what had been merged in the check-in messages. But today Graham Tackley recommended I take a look at svnmerge. I must say, I didn't know what I was missing.

It keeps track of merges for you, ensuring that you'll never try to merge the same revision more than once, or miss any out by accident. Makes it easy to block revisions that you don't want to merge, too. Give it a try.

Don't forget - you'll need a decent merge tool, too

Java may not be getting much Apple love, at least not for the moment, but the better language sure is. ;-)

Also Leopard has built-in Subversion, including Python bindings, and ssh key management too. Sweet.

Just as soon as they get Java up to spec, looks like I'll be jumping.

Update: And perhaps Java on Leopard isn't looking so bad after all.

Jython 2.2.1 is out. It's fabulous to see Jython really making strides again. I used Jython 2.1 for years, and it worked just fine, but somehow if people don't see an OSS project making regular releases they somehow assume that they should steer clear, regardless of the quality of the existing software.

And I must say that while the missing new Python features were never a show stopper, it'll be nice to have them.

London Python went well last night - more here.

I've arranged a London Python meetup tomorrow evening. See some of you there!

I'll be at Jez's London Java meetup this evening. I've not been to one for ages - not since September - so I'm really looking forward to it.

Provided, that is, that they won't talk about bloody Groovy all night. I've still yet to be convinced that Groovy has anything that Jython didn't have nearly ten years ago. Apart from closures and inelegant syntax, that is. ;-)

A new venue this time - it's at The Crown Tavern, EC1R 0EG.

Update: Seems Jython was around exactly ten years ago.

Don't forget the London Python meetup tonight. The bloody football's on again (does it never end?) so it'll be heaving. But then, so will everywhere else.

Update: Right, I'm off.

Oooh, exciting, Michael Sparks (of Kamaelia fame) is thinking about organising a UK PyCamp. If the dates are right, I'm there.

I've arranged another Python meetup for Wednesday the 21st of February at The Porterhouse, Covent Garden. See London Python Announce for details. Err, not that there are any more details than I've just given you. ;-)

We had around thirty people last time, so fingers crossed for another good one.

Right; I'm off to the London Python meetup. See you there.

Sam recently set up the London 2.0 Community Portal. It occurred to me that we London Pythonistas could really do with a site of our own, and since Menno had already set up Planet London Python, with hosting and domain provided by Remi at Web Faction, so I thought I'd ask if it could be hosted there.

So, here's London Python Announce, the London Python event portal.

By the way, I'd seen the screencast of Web Faction's control panel before - but everything looks good in a demo. ;-) Now I've tried the real thing, and it's almost too easy. El Presidente watched over my shoulder, and he thinks that even he could set it up.

Right, I'm off to London 2.0, via a spot of CD and book shopping. I'll see a few of you there, no doubt.

Time to select a venue for next week's London Python meetup on Wednesday (the 15th). I was thinking of trying The Porterhouse in Covent Garden for a change. The place is vast, so there should be no problem getting a table or two so long as I get there reasonably early, and it's pretty central. Worth giving it a go, I reckon.

All welcome, from Python gurus to the merely curious. Please leave a comment if you are thinking of coming.

BTW, to be reminded of this and any other London Python meetup, try this:

If you are going to be doing a lot of merging working with Subversion (or, I'd imagine, CVS), you are going to need good a good merge tool. And believe me, you are going to be doing a lot of merging.

Subversion itself doesn't provide a merge tool - and nor should it. It's a cross-platform command line driven tool, and besides, I suspect that most developers have their own favourite - or need to find one quickly.

If you are working on Windows, you'll probably have Tortoise installed, 'cos it's really good for simple stuff. Tortoise does come with a merge tool, but it's horrid, really nasty. Aside from just how damn ugly the thing is, it's also almost totally incomprehensible. All those colours - what do they mean?

But that's OK. There are alternatives, and they are easy enough to integrate into Tortoise. The first thing I came across was WinMerge. WinMerge is very good for showing diffs, but it doesn't (as yet) do three way merging - so that's out. Another, more powerful alternative is KDiff3. By the time I came across this, I'd already made my happy move to the Mac, so I've not used it myself, but my Windows-victim colleagues report that it works fine.

So the trick is to use the tool with diff in the name for merging, and the tool with merge in the name for diffing. ;-)

Tortoise also provides tools for making and applying patches, but they seem to use fully-qualified path names, so you can't apply patches from one machine to another unless it happens to keep everything in the same place. So, it's best to use Subversion to make your patches with svn patch > whatever.patch to make your patches (or svn st | awk '/^\s?[MAD]/ { print $NF } ' | xargs svn diff > whatever.patch if you've made the same externals mistake that I did), and to use unxutils' patch to apply them.

In the Mac, there's a natural solution - FileMerge, part of the XCode suite. It's very nice to use, if not perfect. (A few more keyboard shortcuts would be nice, for example, Apple, Just 'cos it's GUIfied and beautiful doesn't mean I should be reaching for the mouse all the time.) Integrating it with command line Subversion would be non-trivial but for Bruno De Fraine's lovely Using FileMerge as a diff command for Subversion - he's done all the hard work for you. Thanks for that, Bruno.

Oh, SubClipse also provides a merging tool, but I've never really used it. It's nice to have SubClipse around - the fact that it decorates all your files showing you their status against the repository is nice, and I often use it to add files - quicker than an svd add if you are in the IDE already. But IDE/version control integration is far less crucial than it used to be in the bad old VSS days when you couldn't edit a file without checking it out, so I do most of my Subversion tasks from the command line.

I'd be interested to know what all you *nix types use - especially anything console based.

Have I missed anything cool?

There were some people who weren't able to make it to last week's restrained and dignified London Python meetup, 'cos it hadn't been arranged sufficiently far in advance, so I'm planning the next one now. It'll be on Wednesday the 15th of November.

I'll announce a venue a bit nearer the time. The Stage Door is a very nice pub, but it's a bit small, and sometimes very noisy, so I'm looking for somewhere a but more suitable for a techie meetup. Any suggestions? (No, not the bloody Bank of England.)

If it were to be a good distance from anywhere that's open 'till the small hours selling cocktails and containing femme fatale redheads, that would be an advantage.

Please add a comment if you think you might be coming.

A big thank you to Menno Smits and Remi Delon for setting up and hosting Planet London Python! Now we'll have even more to natter about at the next London Python meetup.

I've sent Menno a bunch of URLs - at the moment, it seems to be mostly my blatherings up there, and no body wants that.

Update: Blimey, that was quick. Menno had the new feeds up in seconds flat.

Further update: There's a new URL for this: http://londonpython.org.uk/.

One thing that came up yesterday was that people still aren't feeling comfortable with os.walk(). Which is a shame - I love it. I mentioned that I'd used it only that day, in a nice little script that locates malformed XML in a directory tree, and Simon suggested that I post it. So, here it is:

for path, dirs, files in os.walk(os.getcwd()):
    for xml in [os.path.abspath(os.path.join(path, filename)) for filename in files if fnmatch.fnmatch(filename, '*.xml')]:
        try:
            ElementTree.parse(xml)
        except (SyntaxError, ExpatError):
            print xml, "\tBADLY FORMED!"

Syntax highlighted version, with imports: find_dodgy_xml.py.

Notice that it takes more code to import ElementTree than it does to do the actual work! It'll be nice what we can rely on version 2.5 being available, but that's a while away.

Hmm, actually, this is such a common pattern that it's probably worth a helper function:

def locate(pattern, root=os.getcwd()):
    for path, dirs, files in os.walk(root):
        for filename in [os.path.abspath(os.path.join(path, filename)) for filename in files if fnmatch.fnmatch(filename, pattern)]:
            yield filename

Syntax highlighted version: locate.py.

This simplifies the main loop to:

for xml in locate("*.xml"):
    try:
        ElementTree.parse(xml)
    except (SyntaxError, ExpatError):
        print xml, "\tBADLY FORMED!"

Worth a cookbook recipe, or is it too simple?

A good Python booze-up last night. No Steve Holden, but a very good crowd, including Simon Willison, Tim Couper, Remi Delon, Chris Miles, Fuzzyman with a couple of his colleagues, Menno Smits, Ben Sanders, Ryan someone-or-other and loads of other people, some of whose names I might even remember at some point. Topics of conversation included XML libraries, Steve Irwin, DVD piracy (theoretical and applied), the Python job market, JavaScript, os.walk and the path module, some Web 2.0 malarkey that I only caught the tail end of and didn't understand a word of, the Python, Ruby and Perl communities compared and contrasted, PyCon and EuroPython, and embarrassing drinking events - I'm sure that more will come back to me later.

As per usual, Chris made us all go to Cubana after the pub closed for several jugs of daiquiris and mojitos, and some very odd shooters.

We also bumped into a girl who was a dead ringer for the lovely Geraldine Somerville. I was convinced it was actually her, but alcohol may have been a factor there.

I felt fine this morning, strangely enough, though I did sleep through my alarm. I'm just too old for this two in the morning business these days.

Photos forthcoming, when Chris recovers from his hangover and emails them to me.

If you were there, please identify yourself!

I picked up the 2nd edition of Python in a Nutshell by Alex Martelli, Python's very own Umberto Eco. (I donated my 1st edition to my Padawan, Dan.) As expected, there's not that much changed - a few new string methods there, a new module or two there.

But one thing I was hoping for was some ElementTree coverage. The 2nd edition only covers Python 2.4 rather than 2.5, and ElementTree wasn't yet part of the standard library at that point, but ElementTree has been a hugely popular library for a while, so it's a pity it wasn't included. It's my XML API of choice, and chatting to Simon and others at last night's Python booze-up, I'm far from alone. (Tim prefers 4suite, though. To each his own, I suppose.)

Still, all in all, a must-have book. I'm not sure I'd say an upgrade is essential, though, if you have the 1st edition.

Right, I'm off to the London Python Meetup. It's at The Stage Door, but Cubana is nearby, Chris Miles is going, and he's such a bad infuence. So, wish me luck...

There's a discussion going on at the moment concerning refactoring dynamic languages. For whatever reason, its protagonists don't seem to be aware of Phil Dawes' excellent Bicycle Repair Man.

I'd love to give Bicycle Repair Man a go; it looks fab when Phil demos it. but there's one hurdle to get over first - Emacs. And it's a big one.

I've given Emacs a go a couple of times (most recently Aquamacs), and it seems to make sense to me. It's not totally horrid like that nasty vi thing that just beeps at me all the bloody time. No, it's just that the learning curve is steep, and I have real work to do, work that I can accomplish far more easily at the moment using jEdit.

There are loads of tasks that I can achieve really easily using jEdit that I wouldn't even know where to start looking for in Emacs - line sorting, search and replace across a filtered set of files throughout a directory sub-tree, opening files in archives or FTP repositories, HTMlifying, XML re-indenting. I know Emacs can do all this stuff, but I really don't know where to look for it all, and I need to get stuff done.

There are a couple of other jEdit features that I really like and would miss, too. The File System Browser (which I keep docked and open at all times) is a fabulous tool. I like a mouse driven interface for navigating around the file system, and jEdit's is a very powerful one. Also, the combination of the search bar and the hypersearch panel (which I leave on by default and dock at the bottom respectively) is really powerful too. Are there Emacs analogs of these tools?

Oh yes, and I like buffer tabs too.

And before Andy comes in and starts accusing me of being a weak-minded GUI lover, I'm not. I'm getting on fine with bash on the Mac. I'm starting to use awk and sed to do stuff on the command line that are totally impossible via a GUI. (Oh, and thanks for the help on that, Andy.)

So, should I stick with what I know? After all, it's not like I'm using Notepad here - jEdit's very good. Or should I take the pain and try to switch to Emacs? I'd be able to use Bicycle Repair Man!

Tap, tap. Is this thing on?

Now, blackmail is an ugly word, but I can't think of any other word for it - Simon Willison tells me that he'll come to tomorrow evening's London Python Meetup (at The Stage Door near Waterloo) if, and only if, I start blogging again. So, I don't really have a lot of choice, do I? Let's see if I can manage a post or two a day...

It would be beneath me to get my revenge by mentioning that at one point last night (at Sam Newman's London 2.0 meetup) Simon referred to Jack Davenport as "the hot one from Pirates of the Caribbean", don't you think? ;-)

Anyway, do try and get along tomorrow evening, all you London based Pythonistas. I think there will be a good crowd, including Steve Holden, Simon Willison, Tim Couper, Remi Delon and many other equally nice but not quite so famous chaps. (Yes, probably all chaps.) So, get practising.

I'm taking the opportunity of all the waiting around time to write some great Python stuff. I'm away from the office for at least two weeks, which is far too great a strech of time to be away from our subversion repository for me to feel that I'll be able to merge stuff back into the trunk if I were to make any big changes, so I can't work on my main project - but I have roadkill with me, and a locker to leave it in while I'm in court, so I have a little side project on the go.

We are finding that we have to write huge swathes of Java code for each new business entity (policy, claim, broker, that kind of thing) that we want to make available for our users - domain objects implementing soft coded rules (in Jython!), data access objects, model view controller classes and the JSPs to go with them, web services and clients, plus interfaces and unit tests for them all, and a ton of config files - it just goes on and on. It's at least couple of days work just to get the basics in place, and it's the worst kind of work - repetitive and boring without being easy.

But it occured to be that 90% of this stuff could be automatically generated from the information in the Hibernate mapping document that we write for each of these entities - more given the fact that Hibernate helpfully allows you to add your own meta tags to your mappings, so I can add stuff to the mappings specifying which fields get shown in which places, for example.

I've not done any serious work in Python for a while - it's been all Java. So it comes as a surprise, once again, how powerful Python is. And how much fun! Nice to be reminded.

Aside from Python itself, there's a wierd kind of synergistic effect you get when you combine various well (Pythonically) designed modules. For example - I'm using classses inheriting from a new-style-class variation on Alex Martellini's Bunch class to store data about the entities in question, and their properties, and I'm using ElementTree to parse the mapping file. (I'm using ElementTree for its elegance here, not for its performance. The mapping files are tiny.) The fact that ElementTree spits out a node's attributes as a dictionary made the data just fall into my objects, so my parser has an obscenely small amount of code considering how much it does, it's still very readable.

Then I use Cheetah to generate the Java code, and again, the fact that Cheetah can be driven by my objects as they are means that the only code that I have to add to them is for derived values (which I'm implementing as properties) - no dross, only real domain logic.

I've pretty much finished the Python code now, I think. :-( Just the rest of the templates to build.

Sam is organising London 2.0rc3, the latest of the combined Python/Django/TurboGears/CherryPy/Ruby/Rails/Java/Spring/Hibernate/the million other things you need to build a Java web app these days/Groovy meetup for Monday the 13th of March. He's not mentioned a venue, but the last few have been at The Old Bank of England, Fleet Street, so I shouldn't be too surprised if this one is there also. Keep an eye on his blog for confirmation.

I'll try to get along, but I can't guarantee anything this time. I'm on Jury Service at the time, and I have little idea what to expect.

London 2.0 rc2 is on for tonight. I hope to see a bunch of you there.

We didn't get a room, I'm afraid. It's all my fault - Sam didn't book a room because he though I would have done so. I didn't book a room 'cos I had been thinking of another venue, so when Sam's post mentioned The Bank of England, I assumed that he'd booked the room. Sigh. Sorry about that. Piss up in a brewery, anyone?

Anyway, I'm hoping for a bunch of nerds of many colours - Python, Ruby, Java, Django, Turbogears, Rails, you know the kind of thing.

What I'm not expecting is many women. One or two - Katherine and perhaps Sarah - but London tech meetups suffer from the same malaise as the IT industry in general; a woeful gender imbalance. I'm not sure what, if anything, can be done about it, but Sarah is having a go by organising the Geek Girl Dinners. Do get along if the rules allow - and pass on the news to the nerdy women of your acqainance.

This Month's London Python meetup on Tuesday will once again be just a part of a pan-nerd event, I'm glad to say - London 2.0rc2. And don't forget - the multi-talented Python celebrity Steve Holden is hoping to get along, as is Django co-author Simon Willison, and Python's answer to Max Clifford, Dr. Tim Couper, so don't miss it.

As last time, it's at The Old Bank of England, Fleet Street, any time after work. I'll see some of you there, I trust. I hope I get home a little earlier this time...

Back at work, and straight back into the thick of it. Meetings all day for the past two days.

Anyway, it'll be less frenetic tomorrow, I hope. Still got 2000 Bloglines entries and 500 email threads to get through. In the meantime, a Happy New Year to all of you, and yes, the Python meet is still on for the 10th. ;-)

I was off yesterday. I was feeling a little under the weather. Can't think why. Perhaps I had a dirty glass on Monday night.

I met a lot of nice, interesting people, some for the first time, many not - more on that later.

The night ended with Chris, Edward and I at the Roadhouse. What a flea pit! I was home at four...

Right - I'm off to The Old Bank of England, Fleet Street, for London 2.0rc1. See you all there.

Don't forget, London based nerds of whatever stripe, Monday night is London 2.0rc1. I hope to see a whole bunch of you there.

All IT systems have roughly equal hassle rates. Should you improve the quality of your software (by means of thorough automated testing and so forth), other components of the system (such as the RDBMS, or the hardware) will take up the slack by failing more frequently.

We have a special guest for January's London Python Meetup; c.l.py regular Steve Holden, perhaps best known for the terribly good Python Web Programming.

The date will be Tuesday the 10th of January (see, a Tuesday - I do listen!), and the venue will be announced nearer the time, though it will certainly be somewhere in central London. It will depend in part upon whether we'll be teaming up with other groups à la London 2.0 or going it alone. I've not yet heard back from Sam and Jez on that. I'll update nearer the time.

In the meantime, make a note in your diaries. And as ever, please leave a comment if you plan on coming.

There goes our crowd for London 2.0.

Update: Just to be clear - London 2.0 isn't off by any means. It's just that I'm concerned that many of our potential attendees will go the the Backstage meetup.

This is getting out of hand. December the 12th now appears to be a combined Python/Django/TurboGears/Ruby/Rails/Java/Spring/Hibernate/the million other things you need to build a Java web app these days/Perl/Catalyst/Maypole/Groovy/Grails Christmas party, and Jez is trying to get the Geek Girls on board too. Though that may be just wishful thinking on his part. ;-)

So, we really need another name. Jez suggested London 2.0, which is just so zeitgeist. Me, I'm more London 0.9, but I'll go with it...

Let's just hope the Perl guys don't have a punch up.

Speaking of the London Web Framework Night - I never did write that up, did I?

OK, so, first up, Catalyst. What can I say? Catalyst may or may not be a great web framework, but since I didn't understand a word of the presentation, I'm not in any position to judge. It seemed mainly to consist of a list of CPAN modules that are either part of Catalyst, or can be plugged into it. No code, no working system, no screenshots, no inkling of what's in it for me. Nothing. Awkward, opaque, and unfriendly to all but insiders - much like I imagine Perl to be. (It must be said, the Perl mongers are aware of the problem - see this post, and this one. Message for Simon Wistow re your "Show Leon's Catalyst based web debugger. I'm willing to bet that it would be largely impossible in either of the other two" comment - I give you Ajaxy Exception Catching.)

Next up came Simon's Django presentation. I may have put The Fear into Simon about presenting to so many people, but perhaps I did him a favour - despite his admitted inexperience as a presenter, it was a cracking show. Compelling, funny, enthusiastic, and giving a very good idea as to what Django does, and how it does it. Using it will be pretty much a no-brainer when it comes to putting together a CMS style site. Whether it's the right platform for database driven sites like the kind of thing that I do for my day job is another matter. There's a clear front end/back end division with Django, with editors using its spectacular "magic" admin interface, and users mainly viewing content - though you do get community features like commenting pretty much for free. Perfect for a lot of sites, but would it suit enterprise database apps like banks and insurance houses need - and I write?

Simon demonstrated both the front and back ends of Django using lawrence.com - one of the sites for which Django was developed in the first place. Though the God of Demos made an apprearance at one point giving Simon an SQL exception, he also had a number of "oooh"s, and outright applause at least twice.

Last up was Matt Biddulph giving us a flavour of Rails. Struggling manfully through a stinking cold, Matt gave us the phiosophy of Rails in a very punchy manner. It looks very much a case of "do it our way", but that's often the right approach. Not enough code on show to tempt me away from a Python platform, though. ;-)

Without a doubt, the highlight here was Matt's demo of BBC Programme Catalogue. I have no words for how cool this project is. None.

I missed out on the booze up afterwards - I was feeling a bit fragile after several heavy sessions on the trot - so I'm sorry if I missed any of you. It was a good night nevertheless - a big thank you to Dean Wilson for organising it all.

Me? I'm looking at TurboGears. ;-) I like the concept of tying together best-of-breed components. Getting it running on my Mac was trivial, and the 20 minute Wiki is a superb demo. As soon as I locate some of that copious free time of mine, I'll try throwing together a simple site or two to see how it hangs together.

Not content with a Python/Django/Ruby/Rails meetup, Sam, Jez and I have teamed up to throw a Python/Django/Ruby/Rails/Java Christmas party this year. (For "party", read "bunch of nerdy blokes, many with beards, standing around drinking beer and chatting about computers".) It's at The Old Bank of England, Fleet Street on the evening of December the 12th.

Do leave a comment if you fancy coming - we have a room booked, and but we can change it to a bigger one if we need to. I anticipate a lot of interest, what with combining the Java crowd with the dynamic language people, and possible extra interest due to last week's London Web Frameworks Night.

Anyone want to demo TurboGears? ;-)

Update: Thanks to Dave and Dave, it's now a Python/Django/TurboGears/Ruby/Rails/Java/Perl/Catalyst/Maypole Christmas party. That has to be every web nerd in London, right?

Son of Update - This Time it's Personal: See London 2.0rc1.

Not that I'll be listening to Aerial much tonight - I'm off to the London Python/Django/Ruby/Rails meetup, with Alex, my Technical Director. I hope to see some of you there. No Sam, though; he's malingering. Or whatever.

It's going to be a busy month for nerdy stuff.

Firstly, there's Jez's London Java meetup on Thursday. Always a good night.

Then, next Monday, we have the London Python/Django/Ruby/Rails meetup. It's looking like we might get a good turnout for that - including my Technical Director! Gulp - I'd better be on my best behavior.

Then again, naaah.

On the 17th, there's the London Web Frameworks Night. Looks fascinating - demos of many of the happening web app frameworks back to back. Given the noise and drunkeness I don't tend to take much detail in during the Python/Django/Ruby/Rails meetups, so it'll be interesting to see more formal demos. Let's hope I'm quick enough to book a seat...

And lastly, there's the Einstein vs. Newton debate at the Royal Society on the 23rd. A different kind of nerdy, but nerdy nevertheless. Not to be missed.

Update: If you want to go to the London Web Frameworks Night, sign up now.

One for your diaries, London based nerds; The next London Python/Django/Ruby/Rails meetup will be on Monday 7th of November at The Old Bank of England, Fleet Street. Sam has booked us a back room, so there might be some chance of hearing any demos that might go on this time. Until I get pissed and start singing, that is.

If you fancy coming, do leave a comment. People like to be sure that they won't be the only ones there. ;-)

We've not automated our functional tests in the past - I know, I know - but we are really trying to pick up our game in our currenty project, so manual functional testing is another of our bad habits that we are consigning to the dustbin of history. With a bit of a steer from Sam, I looked at a number of functional test tools, but Selenium really stood out from the rest. It's just powerful enough, it's really simple, and it runs in your browser, so you can test browser compatability. Take a look at the demos to see how simple it all is.

In essence, your test script is just an HTML table, each line of which is either an action or an assertion. There's even a tool to record your actions to give you a hand building regression tests.

Now I just need to work out how to integrate it all with Cruise Control. Anyone done this?

Oh yes, and I need to find a way of testing our web services, too. Is there no end to the array of tools that you need to build a web application these days?

Don't forget the London Python/Django/Ruby/Rails/Kamaelia meetup this evening...

There's rather an interesting thread going on over on c.l.py about private attributes and Python. It gave me the opportunity to post a link to one of my favourite c.l.py posts of all time, the Martellibot's wonderful essay decrying the "principle of least privilege". It should, as Steven D'Aprano says, be required reading for all developers, regardless of platform.

Also in the thread, the effbot comes to my defense on the subject of the rationale behind Python's name mangling feature, and links to a facinating thread from depths of Python's history in which the feature is originally discussed, and which contains this from Guido:"Python is an experiment in how much freedom programmers need. Too much freedom and nobody can read another's code; too little and expressiveness is endangered". Poetry.

BTW, do any Java-heads out there know whether or not cglib allows you to bypass private and protected?

I've not organised a London Python Meetup since April's, partly due to meetup.com's demise, partly due to my personal finances (I spent £100 last time!), but mostly due to pure sloth. Last night, though, at Sam's Django vs. Rails bash, considerable interest was shown in Python meetups.

So, Sam and I have decided to combine the two. On October the 10th, please come along to Smith's of Smithfield for nerdy chat about Python, Django, Rails, or anything at all really. Say, sixish 'till late.

Last night's Django vs. Rails thing was great fun. Funny thing - almost no one who came along had actually made use of either package. We were all there to hear from the experts, but there don't seem to be any of those just yet.

Well, except for Simon Willison, that is. Simon (1st page blogging for 'Simon', damn him!) is one of the three guys who built Django. Sadly, he hadn't brought a demo along either. :-( I did confirm that Django was named after the great Django Reinhart, though.

There were plenty of other interesting people around, some I knew already (Hi, Edward) and some I didn't (Hi, Torchbox people), and most of who's names I'd forgotten within seconds of having been being introduced. I'd blame the beer, but that would be lying, 'cos I'm always like that.

Tonight, to Jez's Java meetup with El Presidente and Tulna. Should be in interesting...

If you are coming along, do leave a comment...

Only on comp.lang.python would this post have recieved helpful responses. ;-)

I've been getting on fairly well with Roadkill, my nice new Powerbook, but it's been a bit of a learning curve. The pretty Mac front end stuff is a piece of cake on the whole, with one major exception - undo, cut, copy and paste are Apple-z, x, c, and v respectivly, instead of Ctrl-z, x, c, and v as they are on Windows. I've been using those shortcuts for so long now that they are embedded in muscle memory. I've hit the wrong shortcut before my brain (such as it is) has had a chance to overrule them. Getting over that is going to take a while.

Getting some of the command line driven stuff - Ant, Subversion and Tomcat - working was a bit more of a challenge. Easy once you know how, but you keep being tripped up by things that I didn't know about.

I had installed Java 1.5, but the installer didn't tell me where it had been put, so I had to go and look for it. I found it in /System/Library/Frameworks/JavaVM.framework/Versions/1.5/ in the end.

There was already a version of Ant installed in /Developer/Java/Ant, probably as part of the XCode tools install, but it was the wrong version, and in any case I wasn't able to add 3rd party tasks to it, so I needed to use my own version.

Both of these problems involved setting environment variables, and I gather that it's recommended that you don't muck about with global environment variables, so I set up some shell scripts to start and stop tomcat and to run ant scripts with the environment variables set correctly. I was even smart enough to put the shebang line at the top (though finding the bloody '#' key took a while) and to mark them as executable wwith chmod. But I still couldn't run the bloody scripts. The shell was claiming that tey wen't there, though I could see them with ls.

It turns out that your current working directory isn't on your path by default. Arrrgh! That one took me hours, literally. Prefixing the script names with ./ does the trick there.

I still can't resolve any of the other boxes on my network by name, only by IP address. Irritating, but no show stopper. I can always add them to /etc/hosts, but for the moment I'm just using the IP addresses.

Still, I managed to get past all that, and my current web application is building and running on Roadkill happily enough now, after fixing a couple of minor Windows-isms in our build file. ;-)

Next on my list; installing PostgreSQL, and getting my app to work with it, and working out this whole multiple Python versions on one machine thing, and working out where the insert and delete keys are...

One other thing - what do you, uh, we Mac users use to read Usenet? Entourage seems to have trouble getting at Gmane - this fix, uh, doesn't.

Pop along to next week's London Java Meetup on Tuesday the 6th. I'll be there - and I'm bringing Tulna and El Presidente with me this time. Perhaps Steve, too.

This month's topic is Spring, but as usual, off topic nerdyness of any kind seems to be welcome. For instance, I happen to know that Tulna (nerd in denial) will be interrogating Darren about shooting RAW, since she has a shiny new Nikon D70s.

I've dropped the ball a little recently with the Python meetups, but I'll probably organise one for October. I can't afford one this month - I spent over £100 last time!

First post from roadkill!

Kevin Barnes' Freedom languages (via Smalltalk Tidbits, Industry Rants) is very interesting. Read it. ;-)

I seem to have left my iPod playing when I got home last night, 'cos it was dead this morning. I hate commuting without music. (It's happy enough now though - it was just the battery.)

Besides which, being the nerd I am, I had to knock this up to make sure that my metadata is still correct:

import win32com.client
 
def main():
    iTunes = win32com.client.gencache.EnsureDispatch("iTunes.Application")
    
    quiet_tracks_playlist = iTunes.LibrarySource.Playlists.ItemByName('Reduce Played Count')
    for track in quiet_tracks_playlist.Tracks:
        print 'reducing played count for', track.Artist, '-', track.Album, '-', track.Name, 'from', track.PlayedCount,
        track.PlayedCount -= 1       
        print 'to', track.PlayedCount
 
if __name__ == '__main__':
    main()

How sad is that?

(I suppose that I'll be learning how to script iTunes all over again once I've got my new Mac.)

Why do I care about re-setting the played count? 'Cos I want this to be right:

import datetime
import time
import win32com.client
 
def main():
    iTunes = win32com.client.gencache.EnsureDispatch("iTunes.Application")
 
    total_time = datetime.timedelta(0)
 
    for track in iTunes.LibraryPlaylist.Tracks:
        try:
            track_time = time.strptime(track.Time, '%M:%S')
        except ValueError:
            track_time = time.strptime(track.Time, '%H:%M:%S')
        track_length = datetime.timedelta(hours=track_time.tm_hour,
                                          minutes=track_time.tm_min,
                                          seconds=track_time.tm_sec)
        track_time = track_length * track.PlayedCount
        total_time += track_time
                                                 
    print 'Total iPod time:', total_time
 
if __name__ == '__main__':
    main()
    raw_input('Hit enter to continue...')

And how sad is that?

Ever since Python 2.4.0, the Python for Windows binaries have been build with MSVC7, as opposed to version 6. The main reason for this, IIRC, was that VC6 was no longer available to buy, and was increasingly difficult to get hold of. (See the bottom of the python-dev Summary for 2003-05-01 through 2003-05-15 for more.) Microsoft sweetened the deal by giving free copies of VC7 to several key Python developers, and pointing out that free (as in beer) versions of the compilers are available for download.

All C coded Python extensions distributed as Windows binaries need to be compiled by the same version of VC as core Python, so this required all extension builders to get VC7. But then, most probably already had it, and given that VC6 was no longer available at all, this probably made things better rather than worse on the whole. It was the right decision.

Update: Phillip J. Eby correctly points out that an extension builder doesn't need VC7 - only the core developers need that. An extension builder can use the free Microsoft compiler, GCC, or MinGW.

On the other hand, being Microsoft, there was bound to be a price to pay - and it turns out to be a big one. Software compiled with VC7 requires a dll, msvcr71.dll, at runtime. Owners of a full version of VC7 are free to distribute this, but no one else is.

This isn't a problem for the standard Python distribution, since the core Python developers have donated copies of VC7. It's not a problem for Python C extension builders and users, either, because the extensions can be compiled using the free compiler, and use the version of msvcr71.dll distributed with the standard Python distribution at runtime.

So, why is msvcr71.dll a problem? Py2exe, that's why. It's no longer possible to use py2exe to build a working stand-alone executable from a Python script and distribute it to people who don't have Python already installed. The exe built by py2exe needs msvcr71.dll to run, and you can't legally distribute that unless you own VC7.

Was this a deliberate trap? Probably not. I'm not that paranoid about The Great Satan, err, I mean Microsoft. I doubt anyone realised in advance that there'd be a problem. But it does underline the need to avoid proprietary software in general, and Microsoft software in particular, if any reminder were needed.

In fact, there's at least one other problem that I've come across caused by the move to VC7. There are Python bindings available for Subversion, (which rocks, by the way) and you need these to run Trac. But the Python Subversions bindings aren't available for Python version 2.4, and won't be in the foreseeable future; Apache is still compiled using VC6, so Subversion is built using it too.

I don't really have much in the way of a suggestion as to a solution to this, I'm afraid. It would, of course, be lovely if Python for Windows was built with a free compiler such as GCC or MinGW, but there would be a huge amount of effort involved in making the move, for which I'm in no position to volunteer.

Having read Raymond Chen's Using script to query information from Internet Explorer windows, well, how could I resist?

import win32com.client
shell = win32com.client.gencache.EnsureDispatch("Shell.Application")
 
windows = shell.Windows()
for window in (windows.Item(index) for index in range(1, windows.Count+1)):
    if window:
        print window.LocationName, '=', window.LocationURL

Quite pointless, but there you are.

There are other fun things you can do with the shell interfact, too, like opening exporer windows pointing to the directory of your choice - shell.Explore(r'C:\projects\morph') - or opening up control panel items - shell.ControlPanelItem('timedate.cpl') - but they are mostly pretty pointless too. I can think of uses for some things, I suppose - shell.IsServiceRunning('Tomcat5') might come in handy, along with ServiceStop and ServiceStart.

You can find some of this stuff documented in MSDN - Shell Object and IShellDispatch2 Object.

Mum is looking to do a spot of Instant Messaging with her Amida Buddhism chums. (My, look at all those blogs! All down to Mum's example.)

I've been looking at IRC recently, and it's really nice for techies. I've been using Bersirc as a client and Freenode as a server, and it's dead good. I've had some really useful help with Spring on #spring, and #python is cool too. (Say "Hi" if you are around there - I'm on as "small_values".)

But is it good for non-techies? If not, what should I recommend to her instead? Trillian? Miranda? Using what protocol? ICQ? (Anyone mentioning MSN can bugger off. These people are Buddhists, and need to keep their souls pure.)

I'm off to CenterParcs with the kids, and my Mum, sister, brother in law and nieces next week, so Small Values will be even quieter than usual.

Blogging has been sporadic at best recently 'cos I've been so busy, mainly with a new Java web-app that we've been working on, using Spring and Hibernate running over a legacy database. It's been hard work, but I think we are over the hump now.

The other thing that's been keeping be busy is doing the Python URL. It's hard work - but it seems to be going down well. If there's one next week, though, it probably won't come from me...

Anyway, have a good week all. I might possibly post next week if there's a 'net café around, but don't count on it. I should have loads of photos by the 20th.

Two or three years ago, I put together a little Python app that provided shortcuts to useful places in our gruesome internal timesheet system. (The URLs aren't static, so I needed a script.) I used PyWin32 to enable you to launch these from the Windows systems tray, based on demo code by Mark Hammond. My script was a bit of a bodge, though; the UI code and the logic code were all mixed up. I always intended to go back and separate everything out, but I never got around to it...

Until now. I had a couple of changes that needed making, so I grasped the nettle and put together a general purpose module; SysTrayIcon.py. It still needs documentation, but you can see basically how to use it from the demo code at the bottom.

There is one problem with it, though - on XP, if you set up a menu item with an icon, the menu item text overlaps the icon image by a few pixels, and I can't work out how to fix it. Any pointers?

Oh, and yes, yes, I know it's not called the system tray really. But that's what everyone calls it anyway.

Home at two this time. Too hungover to blog. Gallery here.

Update: More pictures, courtesy of Edward.

Tonight's London Python Meetup at The Stage Door will be the last one organised via meetup.com, due to a rather unfortunate decision of theirs. If you want to be informed of future London Python meetings, you'll need to join the Python UK mailing list - or just subscribe to this blog.

It occurs to me that a moin page would do everything that we need - new events could be entered there, and people could subscribe to the page to be kept up to date. Perhaps I'll install a moin instance here on SVoC - or perhaps I could just add a page to the Python Wiki?

Oh, and don't forget the London Java meetup on April the 20th. I'll be there!

Don't forget the London Python meetup tomorrow night at The Stage Door near Waterloo. If it's anything like as much fun as the last one, it'll be great.

Tonight, Samba. Wish me luck.

My first stab at the Dr. Dobb's Python-URL! is out. What do you think? Did I miss anything?

Oh, and Freja is famous, too!

While Freja and I were playing with some Python last week, it occurred to me that the biggest problem that she (and presumably other kids) was having was how unforgiving computer languages are. For instance, after changing the name in the 'you_smell.py' script, she decided that she wanted to have it react to "daddy" or "ella" in certain ways. Without prompting, she tried:

if your_name.lower() == "daddy or ella":

Now, that doesn't actually work, but it's not a bad stab, I reckon. But you get no positive feedback for getting close - stuff just doesn't work. And many of the changes she made were plain syntax errors.

This wasn't a problem - I was with her, and I didn't allow her to become frustrated. But I doubt that you could leave a child of her age on her own to play with this stuff, really.

I recieved my copy of the 2nd edition of The Python Cookbook yesterday, too. Nice to see my name in print, however miniscule my contribution. Nice to get a free copy, too. ;-)

I've not had much chace to look through it yet, but it looks interesting. The new chapters on generators, descriptors and metaclasses look particularly useful. More later.

My post to the Python for Kids thread is something I've been meaning to get around to with Freja for a while. She's only eight, but the's confident with and facinated by computers, so I thought that giving her a go wouldn't do any harm if I didn't push her.

So far, she's having a blast. At first, she just drove the turtle around, changing its colour on the way. But towards the end, I suggested that she make a square, and she managed that easily enough. I'll have a bash at loops with her soon, but I really don't want to overload her.

There was another script that I knocked up for her that she had fun with, first just running it, then making simple modifications:

your_name = raw_input("What's your name? ")
if your_name.lower() == "freja":
    print "You're very stinky,", your_name
else:
    print "You smell lovely, ", your_name

She worked out for herself how to replace 'freja' with 'daddy'. Ah, simple pleasures. ;-)

Note to self: Check out PyLogo.

Michael Spencer's code limerick is, well, I don't think that superb is too strong a word. It's not just that it's a limerick itself, it's what it outputs that makes it so cool.

Yes, I know; I really need to get out more.

Four days the lists have been up, and already people are helping one another out on them. Nice.

There's me saying that I'd be happy to see the back of lambda, and the effbot goes and posts ElementPlist, a beautiful use for them. Nothing there you couldn't do without lambda, but not so elegantly.

He only does it to make me look stupid, you know. ;-)

Tim and I are getting a steadily increasing number of emails regarding WATSUP and winGuiAuto.py, so we thought that now would be the time to set up a mailing list or two.

Now if someone can just help Tim to put the files up on Sourceforge, we'll be in motion. ;-)

Congratulations to the Jolt 2005 award winners (PDF). I've read, used or will use a number of the winners:

• Better, Faster, Lighter Java by Bruce A. Tate and Justin Gehtland, Technical Books Jolt Winner
• Hibernate: A Developer's Notebook by James Elliott, Technical Books Productivity Winner
• Subversion, Change and Configuration Management Tools Jolt Winner
• Eclipse 3.0, Languages and Development Environments Jolt Winner
• Python 2.4, Languages and Development Environments Productivity Winner
• Hibernate 2.1, Libraries, Frameworks and Components Jolt Winner

Check out Google Code. Google is open sourcing some cool stuff. First featured project: PyGoogle, a Python wrapper for the Google Web APIs. There are RSS feeds, too, so you can keep up to date with what they are up to.

As EP said over on c.l.py, this is great PR for Python.

The last meetup went very well, from what little I can remember. I'm organising another for April the 13th, again at The Stage Door. Pythonistas (or is that Pythoneers?) and the Python curious all welcome.

Guido's thinking about ditching lambda, reduce(), filter() and map(). I don't use any of them, so I'd be quite happy to see them go, and the proposed product(), any() and all() functions look really good.

I'm with Peter, though - they shoudn't be builtins. We have enough of those already. I seem to remember the martellibot proposing a reductions module at some point. That would be a much better place for them - and for sum(), for that matter. Not only would that avoid bloating the builtins, it would also help to avoid the problem of people already having objects named 'any' or 'all'.

Bizarre. Wherever I go, I seem to develop the same undeserved reputation. This is a wholly unfair accusation; in many of the photos, I'm drinking a Mojito.

Last night's Python meetup was a blast. I still haven't sobered up. We had nine people at one point, which was pretty good. It ended up with Edward, Chris and I drinking Mojitos until stupid o'clock at Cubana, watching a couple of rather cute girls playing some rather good salsa.

Update: Thanks to Edward for this photo gallery.

I didn't get home until half past two. :-)

If you were there, please leave a comment so I know who you all were...

Right, I'm off for a slimming beer or two at The Stage Door with the London Pythonistas.

I might give the traditional kebab on the way home a miss, though...

While I'm posting silly little Python scripts, here's one I prepared earlier.

def inc_string(string, allowGrowth=True):
    '''Increment a string.'''
    CHAR_RANGES = [
                   Bunch(from_char=48, to_char=57),  # digits
                   Bunch(from_char=65, to_char=90),  # upper case