Thursday, April 26, 2007

Test speedups

The MIFOS tests (ApplicationTestSuite, or pretty much all the junit tests) run in 2600 seconds on my laptop. When you are checking in several times a day, that adds up to a lot of time spent waiting for tests. It also discourages good habits like running the tests frequently to find problems early, checking in frequently, certain kinds of experiments (for example "if I clean up this ugly code, will anything break?") and the like.

There are a few reasons for this. First of all, I'll agree with something that Eric Du or Li Mo (I forget which) said a month or so ago, which was that too many of our tests are integration tests (test many classes) as opposed to unit tests (test a small number of classes). This is very much a case-by-case thing, so I guess I'll just mention the case which came up today: I was testing generateId in AccountBO. And it turns out there is no need to access the database to test this method (see AccountBOTest).

I've also known for a long time that many of the tests get caught in a familiar trap of writing many objects to the database (I need a client, and a client needs a group, and a group needs a center....). Or of creating objects via the database when creating them in-memory would work just fine, be faster, and avoid problems with getting rid of them at the end of the test.

But I was surprised at the speedups around getUserContext(). Someone (sorry, I tried finding out who in the archives but I didn't find it) posted some numbers to the MIFOS mailing list saying that
replacing TestObjectFactory#getUserContext (which involves several database calls) with something faster (TestUtils#makeUser() is the usual choice) cut the run-time of a certain test in half (or something - the number varies from one test to the next and once I convinced myself that the speedup is significant I haven't really been measuring things further).

Unfortunately, globally changing getUserContext to makeUser doesn't quite work - some tests fail that way. But one of my projects lately has been going through tests and changing all those that can be changed.

Getting tests to run fast can take work (especially if you don't have the luxury of writing fast tests from the start of a project). But tests that run slowly don't tend to get run.

Tuesday, April 03, 2007

Mayfly and Hibernate

On MIFOS, we are successfully using Mayfly and Hibernate together, but there are some catches (and some future work - for me and/or other volunteers - in terms of making this work better).

First of all, MIFOS is currently on Hibernate 3.0beta4. I suspect later versions of Hibernate work too, but it would be nice to download Mayfly and Hibernate and try it out on some kind of "hello world" situation.

Next, there's the Hibernate dialect. The one we are using in MIFOS is checked in to MIFOS as MayflyDialect. It would be nice to submit this to Hibernate as a patch. I have been meaning to do this, but just haven't gotten around to it. For a while, I thought it might be changing frequently, but that hasn't been true lately.

Anyway, enough of the boring stuff. The interesting part is whether we can hook up the features of Mayfly which distinguish it from Hypersonic and the rest. For example, let's take the feature of wanting to give each test a fresh database. Suppose we have a static DataStore which contains all the tables and perhaps some data which all tests should start with (in MIFOS, getStandardStore() in DatabaseSetup ). Now, in Hibernate one typically creates a SessionFactory once (not on every test, as it is expensive to make one), and the SessionFactory has the JDBC URL built in. So how do we give a new Database for each test while being able to re-use the SessionFactory? Well, what I've done so far is open my Session with the SessionFactory#openSession(Connection connection) method. I'm probably best off just pointing to an example: testGetAllParameters in ReportsPersistenceTest.

So, anyone found better techniques? This is a good subject for collaboration, not just because it is a way to share the work, but also because everyone's way of setting these things up tends to be slightly different.