I was just called over by Ken and Grant to help them with a problem that they were having with the unittest/Mox test suite that they were putting together, written ahead of the functional code. I feel so proud.

Given that I'm speaking on Python mocking shortly, I thought I'd better dig through my accumulated bookmarks and see what I'm missing.

PyMock is what I've been using up till now. It's heavily inspired by EasyMock, a mocking style I'm used to. It's pretty good, really. The docs really could be better, and the failure messages are not very helpful, but it does the job.

But then I found Mox. Superficially, it's very similar to PyMock. It's also based on EasyMock. But wherever Mox does differ from PyMock, it's better.

It's much better documented than PyDoc, and has more meaningful, helpful failure messages. I think it's has more functionality, too, with some nice comparators, side effects and callbacks. StubOutWithMock() is nice too. But it's possible that PyMock has all of this - given the documentation, it's hard to tell.

Mox will warn you if you call a method on a mock that the mocked class doesn't have, which is handy. It can get confused if your mocked class uses delegation, but you can always fall back to MockAnything(). The mailing list is small but helpful.

On the downside, Mox isn't in PyPI, which is a shame. And they missed a trick not calling the documentation MoxDox. ;-)

There are many other Python mocking packages out there, too - notably pMock, a jMock style mocking library, Mock and minimock, a mocking library for the doctest unit test library.

In early development there's Mockito For Python, a port of Mockito, which will be worth keeping an eye on if Szczepan ever gets the hang of Python. ;-)

As an aside - I think we need Spamcrest!

Update: Seems there's already a Hamcrest for Python - source here.

Also - I forgot one other problem with Mox - the naming standards. One of my biggest early stumbling blocks was that my brain refused to see the the initial capital on the AndReturn() method, and I couldn't get anything working. It's Google's internal naming standard, I gather. Yuck. I feel a patch with PEP 8 style synonyms coming on.

I seem to find myself signed up to talk at PyCon UK on Mock Testing with Python. Eeeek!

Luckily, this evening I'll be at London Geek Night: Mocking with Java, so I should be able to pick up some tips.

Fuzzyman has a new mocking library for Python, which he presents in Mocking, Patching, Stubbing: all that Stuff.

Michael takes issue with Martin Fowler's Mocks Aren't Stubs; specifically where he defines mocks as objects pre-programmed with expectations which form a specification of the calls they are expected to receive. Michael's mocks are not pre-programmed with expectations - his expectations are defined after the test execution.

Now, to me, this is a trivial distinction - the important difference between a stub and a mock is in the existence of expectations. Whether the expectations are defined before or after the text execution is not crucial - it's still a mock.

It does matter in terms or usability, of course. It feels more natural to Michael to define his expectations after the test object is exercised, along with his assertions. For me, I have to say that defining the mock object's behaviors all in one place makes sense, so both the expected method calls and any return values should defined together - and the return values have to be pre-defined. We are using EasyMock here at GU (along with the Hamcrest constraints library) and I like it just fine. But that's more a matter of taste than anything else.

Julian and I have been asked to put together some unit and functional test coverage figures. We've not actually been given any time in which to do it, though, so I wouldn't hold your breath.

Looks like some work was put into using Emma to provide these figures, but it's not finished.

Over in the Python world, there's been some discussion over on c.l.py about Ned Batchelder's coverage.py, which looks like a fairly nifty module.

OTOH, I think you have to be a bit careful with metrics such as those provided by these coverage tools. Ned himself points out many of the problems with taking your coverage figures at face value (though Emma's block-level approach fixes some of them).

I'd add one more issue - gaming. People can start to see good coverage figures as an end in and of themselves. If the coverage isn't at, oh, say 80%, the tests must be crap. If they are at 100%, they must be great. It's not just that the figures can be meaningless, they can actually lead people astray.

Now, for an approach to coverage testing that actually makes sure you are testing that your code does the right thing, check out Jester & Pester.

I've been trying to upgrade the version of Selenium core that we are using from the rather old 0.6 to the shiny new 0.8.2. It's turning into a bit of a slog.

Firstly the assertNoFailureOnNextAndGoto user extension that we are using to ensure that each test starts out logged out needed an upgrade, but Margo sorted that one out.

Next, we had a problem with our assertLocations. Selenium 0.6 would pass the assertion if the supplied values matched any part of the actual URL. 0.8.2 supports patterns - much superior, but the change required us to re-visit all our hundreds of tests and whack asterisks around all our asserted locations. I took over at this point to exercise a spot of regex-fu.

But now I've got a bit of a show stopper, and I'm stuck. 0.8.2 strips whitespace from type action values - which buggers things up royally if the values in question consist of Python code! I tried working around the problem by whipping up a quick user extension, but that didn't help, 'cos the whitespace has already been stripped by the time the value is passed to the function. I'm not getting any love from the Selenium forum, either.

Any suggestions from the gallery?

Update: I tried IRC, too, but there's no one there. :-( And look, see, I'm being helpful!