Tuning Hibernate-created SQL queries via XML mapping file

As much as I detest the entire coding in XML paradigm, which still seems to be de-rigeur on most Spring/Hibernate projects, something about the annotations-style of meta-magic for both APIs has always bothered me slightly, even if I couldn't actually put my finger on what exactly it is.

Regardless, we came across an undocumented** feature of Hibernate in which you can control the structure of the SQL queries that it generates via the XML mapping file when dealing with foreign-key relationships. Take a simple example, where a KFCMeal contains a single SideDish and a single Drink:

Now, if you try and query to find all KFCMeals which came with Fries and a Dr Pepper, the query generated would be something like:

SELECT * FROM KFCMeal, SideDish, Drink
WHERE KFCMeal.sideId = SideDish.id
AND KPCMeal.drinkId = Drink.id
AND SideDish.name = "Fries"
AND Drink.name = "Dr Pepper;

The structure of this query is directly generated by the ordering of the mapping, which places the SideDish before the Drink. You could reverse the join structure of the WHERE clause by swapping the SideDish and Drink mappings. Seems obvious, doesn't it?!? - but I'd always (naively) assumed that Hibernate would sprinkle some magic fairy dust on the query as it constructed it. In projects with complex querying requirements, you can use this effect to tune the queries for performance.

Now, to return to my original point of XML vs annotations - I'm wondering how this effect would be translated into an annotation-based mapping. When Hibernate loops over your classes to load the mappings, I'm assuming that it simply loads a list of the fields from the Class definition and that the ordering of this list could not simply determined by the ordering of the fields in the Java source file. And even if it was, you'd probably end up pepper-spraying the source with "whatever you do - don't reorder these bloody fields or it'll screw performance" comments. In the absence of a proper performance testing framework, the alternative would be to have a test which asserts the construction of the query by Hibernate, which probably wouldn't be particularly pretty without getting deep down and dirty with some JDBC code.

Suggestions welcomed here in the comments!

** I say undocumented, but to be honest I didn't go to too much investigative work beyond using the Googlebrain. Your mileage may vary.

FrAgile Development

It's not often that a post comes along that sums up the general Zeitgeist of my world view, but James Shore has managed it with the decline and fall of agile. It seems that everyone, and I mean *everyone*, is now agile.

Or not.

In light of this bright new wave of software evangelists, each and every one foraging for continuous improvement in their tools and techniques, the term "agile" seems hopelessly out of touch. Thus, I'd like to be bold enough that I've identified the existence of (and therefore copyrighted as of now) a bold new software methodology... FrAgile.

FrAgile teams have seen the success/fashionable agile techniques and have decided that it's what they want to do. A good start, until the hard reality kicks in.

Excellent examples of FrAgile teams include those that:

- have short iterations but only release every 6 months (long customer feedback cycle)
- defer risk until late in the delivery cycle (ie. the hard stuff)
- mandate high test coverage without driving development through tests
- collect technical debt, but are "too busy" to pay it back (this applies to all aspects of the dev cycle: production and test code, build and deployment)

Of course, they're all bad ideas and maybe they just need a group hug and a cattleprod in the right direction. Let me just say this, however...

Duh.

Expect to be buying my book on this exciting new development methodology in due course.

LCD Part 2: The Nous Factor

As any seasoned semi-pro will tell you:

On the whole developers are mostly well-meaning, but mostly they are still monkeys.

Not that that's their fault, of course... there's a whole ream of obstacles out there awaiting the unwary traveller of the choppy waters of IT.

For instance, from my background of Konsultancy, it always seemed that anyone who aspired to be a Developer got to be one, with (literally) no questions asked as to their suitability (past passing the generalised multi-skill argy-bargy that made up the interview process). Just suit up and ship out - direct to join the hoards of (slightly) more experienced consultants already on client site - but as you're learning the job we'll only charge the client 50% of your rate until project numero deux. All important decisions will be made by the Technical Architekts from their underground lair, so relax and just go with the flow. All the business requirements will be documented, their clarity almost dazzling in a Word document prepared by an equally unskilled group of bored analyst consultants in 2003.

As for more traditional IT departments, I understand it's a bit more of a mish-mash. When the department was small, one of the outfit's entrepreneurs (now CTO) interviewed a couple of guys for some data-entry work. They wrote an Excel spreadsheet, converted it into Access-backed VBA application, got promoted to hire some monkeys to glue the bits back on that fell off... and so forth until it got spat out as the latest Enterprise-grade-language-du-jour abortion. You get the idea.

Whatever the case, just visualise a bunch of guys who are kind of semi-trained-as-they-read-it-on-a-blog-somewhere mentoring a bunch of other guys who are not-so-up-on-the-blogs-but-they-work-with-a-guy-who-is. At the end of the day, it's all about getting (a) job done. This is not to say that there's anything wrong with ambition though, after all - just take the self-taught bedroom coders of the 80's as a cue.

But.

Believe me when I say - I've seen the future and it's got no sodding clue. At my current workplace, we've got a pair coding test which is not very difficult. However, having failed approximately 95% of candidates who pass through our doors it seems very apt to categories them into the following groups:

1. Those that get it. Rarer than a blue giraffe and mostly quite humble.
2. Uninspired bog-standard dev. Having claimed to be an expert in a particular API, cannot explain what they would change about it given the chance.
3. Clueless central. An all round embarassing experience for everyone involved.

In a roundabout way, I suppose that it's the norm - as more and more people get involved with a particular (hot) industry such as ours, the demand for skilled staff increases at a faster rate than anyone could (or intends to) train them. There are a dire lack of formal qualifications available - those that are range from the "Only One Way To Skin A Cat" (JEE Certified Architect) to the "Certificate in a cereal box" (Certified SCRUMM master). In fact, one of the best quotes regarding certifications I've ever heard was that you should ignore the positive effects of any certification unless the bearer has a certificate for a diametrically-opposed subject.

No, where can I sign up for the Certified Waterfall Master course?!?.

Cult of the Lowest Common Denominator

One of the things that amuses me most (on almost a daily basis it seems) is the general lack of focus in the world-o-sphere of exactly what the major issues are in the software development game. For a field that is already so key in how our increasingly tech-dependent culture goes about it's everyday existence, it seems to me that everyone seems to be blissfully ignoring the most obvious obstacle which prevent software projects from entering a higher level of being - that of the Lowest Common Denominator (oh, and having already typed that term twice, from here on in I'll be using the rather natty acronym LCD).

This (incredibly sporadic) series of posts will highlight the various ways I've noticed LCD manifesting itself around myself and some of my more highly esteemed colleagues.

Are you sitting comfortably? Then we'll begin...

PART 1 - Agile
My most recent thoughts on LCD were sparked by David Peterson's blogging about the chasm facing Agile practitioners in getting general acceptance of the process by the mainstream corporate development communities. I completely agree with his main viewpoints, but feel the need to add a slightly cynical slant of my own...

Agile itself (and I'm using the term as a catchall for Agile/Lean/Scrumm/XP - you know all the good stuff that you're already doing, right?) is a duplicitous being. On the face of it, the techniques and processes that surround it seem simple enough that anyone and everyone feels confident in slapping the badge onto their CV and parading it before them like the Ark of the Covenant, feeling generally invincible. For example, TDD - what's so hard about that? It just means writing a test, and that refactoring malarkey is just about renaming stuff, right? And those Agile books, they're *really* thin - there can't be too much to it if you can sum it up in so few pages.

Unfortunately, this inherent simplicity is coupled with an incredible need for discipline and boldness in all aspects of the development cycle and it's this which will present the biggest obstacle to the Agile adoption process from my perspective for the following two reasons, only the second of which will be discussed here:

1. having the development team do it right
2. having the management team support it right

For those of you that are lucky enough to be working on a corporate software project and aren't yet familiar with the process, here's this week's LCD:

Most bigwigs don't trust their development teams.

In my (admittedly somewhat jaded) experience, project sponsors don't really like thinking about the situation on the ground - just make them look good by deadline X and they're happy. And if it doesn't work, then just remember that 90% of all IT projects fail and be sure to have someone to blame it on - (handy tip: Consultancies are perfect for this and adjust their rates and slavegrinding machines accordingly (once again, another post for another time)).

When Agile is sold into a corporate structure from the top down, it's usually done by a smartly-besuited evangelist reminding the bigwigs how all their IT projects seem to fail miserably and that there's a better way. What I imagine isn't conveyed so well is that in Agile there are no absolutes in terms of delivery. That halfway through the project, the development team may decide to drop velocity to fix something that doesn't, on the face of it, appear to be broken. That they will be right to insist on doing so because they are the most qualified to make that decision. That working evenings/weekends is not the way to deliver anything. And most of all, that even in the very short term, doing it right is cheaper than doing it fast.

Successful Agile projects require acceptance from the bigwigs that projects are as driven by the people on the ground as the business and it's the level of trust and maturity to "support" rather than "run" their development teams that make or break them.