Profiling with gprof (success on a short test program)

When last we discussed profiling Mayfly, I was profiling with JIP. Brian Slesinsky, in a comment to that article, told me that he has found that JIP has a per-method cost which shows up in the profiling data (so that method calls appear to be more expensive than they really are). He suggested the NetBeans profiler.

Well, I was looking into how to install and use NetBeans (NetBeans, unlike Eclipse, does not ship with Fedora), and hadn't gotten much of anywhere until I had a lot of time (on a flight), and was left with seeng whether I could get anywhere with the tools which I have already installed. That means gcj and gprof. I got gprof working fine on a short test program (where it correctly identified the bottleneck), but didn't (yet) succeed in running it on mayfly.

I suppose if people would find them helpful, I could upload my test programs and build scripts, but basically they boiled down to:

gcj --main=Profilee -pg Profilee.java
./a.out
gprof >profiler

Getting something like this invoked from ant is largely a boring but more or less straightforward matter (although it wasn't clear to me how the classpath relates to the class and java files specified on the command line). But at least one of my invocations led to a runaway linker which made my machine swap for quite a while before I finally gave up. So although it is premature to declare victory on this just yet, I did want to report on my success with the test program.

Avoid needless generality

I've been asking people what Mayfly features would be most useful to them. I suppose I shouldn't be surprised that one of the answers I got was roughly "we use the syntax CONCAT(a, b, c) for string concatenation in MySQL and it would be nice if Mayfly had it instead of making us write it as a || b || c". This fits right into the theory that often the most important features are the silliest (as I already discussed regarding case sensitive table names).

Anyway, having decided to work on CONCAT, my first instinct was to provide hypersonic-compatible stored procedures, which allow the user to define their own CONCAT (well, the two argument version anyway; I'm not sure hypersonic has stored procedures with variable numbers of arguments). I started implementing stored procedures, and it became clear that there were plenty of corner cases (picking which overloaded method to call, error handling, type coercion, and of course the variable number of arguments issue).

So not only did it seem easier to just add a CONCAT built-in, it is closer to what the user really wants. Although defining a concatenate method and registering it with a CREATE ALIAS command isn't hard, there's no particularly compelling reason to make people do this. Figuring out where to put the CREATE ALIAS invocation may also be more of a pain than it sounds, in a system like Mayfly with no static state shared between objects.

There's a good chance I (or some other Mayfly contributor) will eventually get around to implementing the general stored procedures (this feature can be a handy way to make SQL code more portable between Mayfly and non-Mayfly databases, in cases where Mayfly doesn't yet have all the built-in functions that the other database has). But jumping right to the general feature runs a risk of either (a) designing a feature which is supposed to be general, but which isn't really suited for any use case other than the original specific one, which could have been implemented in a way which is much more straightforward for the programmer and the user, and/or (b) designing a general feature which is too complex, too slow, takes too much time to implement, etc, relative to what is really needed. So there's a lot to be said for special-purpose solutions.

Simple Design and Testing Conference

Had the chance to talk about Mayfly at the Simple Design and Testing Conference 2007 conference last weekend. As with other open space conferences I've been to, you don't know quite how a session is going to turn out, but my notes include the sample code which I showed people.