Well, I just wanted to speak about an issue for which I didn’t find any proper answer yet.
The matter is gui testing. Currently, at work, we do proper unit test for "framework level code". By "framework code", I mean code being in some utilities or services projects which are put there to be reused. These tests are usually fairly easy to do, since the context is well known. Furthermore, as once as we change this code, these tests prove their value. As changes are made quite often, be it only when adding new stuff and thus refactoring, we feel like these tests are good bang for bucks.
For example, recently, we saw a bug in some on "framework code". We wrote a failing test for it (well, I’m not 100% sure of that, but we should have had anyway ;)), fixed it and, on the go, spotted some possible improvements. We did them as well, then ran again all tests and we were pretty confident about it.
But, then, what about "project specific code" ?
Hum, what’s in it anyway ? Some "framework level code", for sure, which is already tested, and then some JPA, wicket and business related code. Testing the JPA code would basically mean testing the JPA layer, which is pointless. What about the GUI then ? Well, wicket isn’t exactly the easiest framework for it… The way its generates the ids make it hard(er) for most of the html test tools, like for example selenium. And even if, what to test actually ? Should we do some "clicks through" tests for all use cases ? All possible navigation paths ? And what about the data needed by these tests ? Using WicketTester, then we would be at pain with jquery heavy pages (well, I guess most of the selenium like frameworks are at pain in this case…). Overall, the efforts to put there are likely to be important.
So, what about the gain ? The odds that we’ll touch again these classes are quite low, way lower than framework level code. Basically, we don’t reuse these stuff. And if some functionalities/pages are fine enough, then we may even not touch them again before very long.
In the end, we have some layers which are, at the same time, harder to test and with less "return on investment". It looks like not testing these "project specific/gui layers" is a good idea…
That’s where my issue with ui test lies… This ui layer is a pain to test, and would be tested mainly for just a development, without much reuse. But, it’s an error prone layer. And it’s the most visible one… So, if, like me, you’re kind of fond of tests and bullet proof code, then you might well share my dilemma…
What to do then ? Well, for the time, I’ve no proper answer. I guess I should try to spend more time testing my ui before giving it to the outer world, even it feels awkward to do it manually. For sure, exploratory tests might help, but again they’re manual and in no way comprehensive… No clear solution here for me, unfortunately…
Any hint deeply welcome ! In fact, I guess that’s the (hidden) aim of this post : maybe someone’ll come with some magic bullet(s). After all, Christmas is coming soon, so it’s dreaming time anyway
2 points stroke me when reading this "Pragmatic Unit Testing" book :
- in order to easily test some class, they suggest to open the required methods by making them protected. I had first to check that protected included package readability (it does) but still I think it’s going a bit too far : if the method was supposed to be private, package private is the nearest visibility which would fit. I would only make it protected if I intent to override it. Do I miss something ?
- the last paragraph is called "Testing invalid parameters", and speaks of where to test the validity of the parameters. Their answer is "we find the easiest rule to adopt is the ‘keep the barbarians out at the gate" approach’, meaning internal components can trust the data they get. At work, we do it differently : every non private method has to check for the validity of its input (for which we have a dedicated assertion lib.). Would you suggest otherwise ?
During my holidays, I notably read Pragmatic Unit Testing In Java with JUnit.
Basically, this book might be nice for beginners and new comers to unit test. If you’ve never done any and are fearful of starting on your own, then consider it. Otherwise, it doesn’t bring much. It’s an easy book to read but the noise/info ratio is too much in favour of noise : I sometime had the feeling the authors were trying to fill the space.
In fact, I do like the Summary Card offered, it sums up nicely what is to be known, and what I could say is of interest in the book.
Furthermore, the book shows its age : late 2003 is like prehistory ;). So the code samples are a bit outdated and, worst, maven wasn’t popular at the time : lost of lines are lost explaining out to setup the tests’ folders, which is pointless with maven.
In case of questions/comments, do not hesitate.