Why pick JTiger Java Unit Testing Framework over good old JUnit?

When it comes to unit testing in java, one framework prominently comes in mind i.e. JUnit. But there’s a new kid on the block that offers more. It’s the JTiger Unit Testing Framework for Java 2 Standard Edition 1.5. In this interview, Tony Morris, the creator of JTiger talks about unit testing, JTiger and how it is different from other unit testing frameworks – especially JUnit.

IndicThreads >> Hi Tony. It’s a pleasure to have you on IndicThreads. A few words about yourself?

Tony Morris >> G’day. I’m 27 years old and work for IBM Australia on the J2SE implementation. I am a keen proponent of software design techniques, and at times cause conflict in debates since I challenge orthodoxical beliefs ? (I can hear my colleagues chuckling in the background) primarily because I believe that what exists today is far from optimal. I spend a lot of my time discussing such topics of interest, when I should be getting serious and writing a book or something instead. I also spend spare time playing squash about 3 times a week and chess. When it comes to those all important pieces of paper, I have attained:

• Sun Certified Programmer for the Java 2 Platform (1.4)
• Sun Certified Programmer for the Java 2 Platform (5.0)
• Sun Certified Developer for the Java 2 Platform
• Bachelor of Information Technology

IndicThreads >> Could you tell us something about the JTiger framework you have created and why you felt the need for such a framework?

Tony Morris >> The JTiger framework arose as a result of the many shortcomings of existing frameworks, the most obvious of which is JUnit. I felt the need for a few fundamental features of a test framework that were missing in existing frameworks, and a few more that would be nice to have  for example, code coverage reporting would be an excellent integration.

“The JTiger framework arose as a result of the many shortcomings of existing framework….”

I see JUnit as not much more than a glorified if/else construct i.e. assertTrue and assertFalse, or to steal Martin Fowler’s terminology, an anaemic domain model. JTiger also proves itself as a valid framework by providing its own self-tests, with high coverage metrics. The expression syntax of JTiger also fits well with the preferred perspective of unit tests as a requirements specification for the software that is under test. For example, the following JTiger unit test can be interpreted in English as a test case that when 2 and 2 are applied as operands to the addition operator, 4 is the result.

“JUnit is not much more than a glorified if/else construct….”

@Test(addition)
   void addition() {
   assertEqual(2 + 2, 4);
   }

There is a distinct one to one mapping between the English statement and the expression of the requirement as a JTiger unit test. I find this more natural, or as some of the users of JTiger put it, it just glides into place. It is the most accurate expression within the confines of the Java programming language, or at least, approaches it.

IndicThreads >> JUnit is currently the most widely used framework, so could you give us a comparison of JUnit and JTiger and how does one decide which unit testing framework to use?

Tony Morris >> The primary reason you would use JUnit over JTiger is when your test cases must execute under a Java Runtime Environment with a version less than 1.5 for some unforeseen reason. It is important to note that this does mean that the code under test must target a JRE version 1.5; it may target any runtime version 1.2 or higher. I can’t think of any other legitimate reason.

“Only reason to use JUnit is if your code must run in a version prior to 1.5…”

JTiger expresses metadata as language annotations, which are new in version 1.5, where JUnit expresses metadata in some ad hoc fashion. For example, a JUnit test case is one that is prefixed with the name test. Using language annotations provides compile-time enforcement and runtime discovery of the metadata should the need arise, for example, in tools (hint: IDE plugins ?). I believe that annotations are one of the most useful features that were added to the Java Language Specification Third Edition. I have written a comparison between JUnit and JTiger at http://www.jtiger.org/articles/comparison-to-junit.html

“Using language annotations provides compile-time enforcement and runtime discovery of the metadata….”

JTiger is able to execute JUnit tests, which provides a progressive migration path from JUnit or simply an alternative unit test execution environment with additional features if that is preferred.

JTiger arose simply to suit my own needs, and given the encouragement of friends and colleagues, it was agreed that others may benefit from it too, so my employer permitted me to release it under an open source license (CPL v1.0) as my own work.

IndicThreads >> Can you give us a simple example of a test case using JUnit and one using JTiger?

Tony Morris >> I’ll try to be unbias 🙂

Consider the following typical requirement A Person type that has a first and last name, is Serializable since we need to send instances over the wire, and adheres to the Object equals/hashCode method contracts.

package persons;

import java.io.Serializable;

interface Person extends Serializable {
   String getFirstName();
   String getLastName();
  }

public final class PersonImpl {
   private PersonImpl() throws UnsupportedOperationException {
    throw new UnsupportedOperationException();
 }

public static Person getPerson(final String firstName
, final String lastName)
throws NullPointerException {
   if(firstName == null || lastName == null) {
   throw new NullPointerException();
   }
   else {
   return new _Person(firstName, lastName);
   }
   }

private static final class _Person implements Person {
   private final String firstName;
   private final String lastName;

_Person(final String firstName, final String lastName) {
   this.firstName = firstName;
   this.lastName = lastName;
  }

public String getFirstName() {
   return firstName;
  }

public String getLastName() {
   return lastName;
  }

@Override
  public boolean equals(final Object o) {
  if(this == o) {
  return true;
 }
 else if(!(o instanceof Person)) {
   return false;
 }
 else {
   final Person p = (Person)o;
   return firstName.equals(p.getFirstName()) && lastName.equals(p.getLastName());
 }
}

@Override
  public int hashCode() {
  // let's make it simple... don't know about you, but I'm out of breath :)
  return firstName.hashCode() ^ lastName.hashCode();
   }
 }
}

Here is the JTiger unit test fixture. You’ll find that this fixture achieves 100% code coverage. That is to say, every single code instruction in PersonImpl.java is executed, even the NullPointerException! In fact, even both the first and the second part of the expression that must evaluate to true for the NullPointerException to occur is executed.

package persons;

import static persons.PersonImpl.getPerson;
import static org.jtiger.assertion.Reflection.assertHasDeclaredConstructor;
import static org.jtiger.assertion.Basic.assertFalse;
import static org.jtiger.assertion.Basic.assertEqual;
import static org.jtiger.assertion.Modifier.assertPrivate;
import static org.jtiger.assertion.Serialization.assertSerializes;
import static org.jtiger.assertion.Serialization.assertSerializesEqual;
import static
org.jtiger.assertion.ObjectFactoryContract.assertObjectFactoryFillsContract;
import static
org.jtiger.assertion.EqualsMethodContract.assertEqualsMethodFillsContract;
import static
org.jtiger.assertion.HashCodeMethodContract.assertHashCodeMethodFillsContract;
import org.jtiger.assertion.ObjectFactory;
import org.jtiger.framework.Fixture;
import org.jtiger.framework.Test;
import org.jtiger.framework.Category;
import org.jtiger.framework.ExpectException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Constructor;

// this annotation is optional
   @Fixture("Test PersonImpl")
   public final class TestPersonImpl {
   private TestPersonImpl() {

 }

 @Test
   @ExpectException(UnsupportedOperationException.class)
   @Category("factory")
   void factory() throws Throwable {
   final Constructor c = assertHasDeclaredConstructor(PersonImpl.class,null);
   assertPrivate(c);
   assertFalse(c.isAccessible());
   c.setAccessible(true);

 try {
   c.newInstance();
   }
   catch(InvocationTargetException e) {
   throw e.getCause();
   }
   }

 @Test
   @Category("names")
   void names() {
   assertEqual(getPerson("first", "last").getFirstName()
     , "first","First name didn't match");
   assertEqual(getPerson("first", "last").getLastName()
     , "last","Last name didn't match");
   }

 @Test("null names 1")
   @ExpectException(NullPointerException.class)
   @Category("names")
   void nullNames1() {
   getPerson(null, "last");
   }

 @Test("null names 2")
   @ExpectException(NullPointerException.class)
   @Category("names")
   void nullNames2() {
   getPerson("first", null);
   }

 @Test
   @Category("serialization")
   void serialization() {
   assertSerializes(getPerson("first", "last"));
   assertSerializesEqual(getPerson("first", "last"));
   }

 @Test
   @Category({"equals", "hashCode"})
   void equalsHashCode() {
   final ObjectFactory
 factory1 = new ObjectFactory
(){
   public Person newInstanceX() {
   return getPerson("John", "Doe");
   }

 public Person newInstanceY() {
   return getPerson("Mary", "Magdalene");
   }
   };

 final ObjectFactory
 factory2 = new ObjectFactory
(){
   public Person newInstanceX() {
   return getPerson("Jesus", "Magdalene");
   }

 public Person newInstanceY() {
   return getPerson("Mary", "Magdalene");
   }
   };

 final ObjectFactory
 factory3 = new ObjectFactory
(){
   public Person newInstanceX() {
   return getPerson("Bob", "Down");
   }

 public Person newInstanceY() {
   return getPerson("Bob", "Up");
   }
   };

 assertObjectFactoryFillsContract(factory1);
   assertEqualsMethodFillsContract(factory1);
   assertHashCodeMethodFillsContract(factory1);

 assertObjectFactoryFillsContract(factory2);
   assertEqualsMethodFillsContract(factory2);
   assertHashCodeMethodFillsContract(factory2);

 assertObjectFactoryFillsContract(factory3);
   assertEqualsMethodFillsContract(factory3);
   assertHashCodeMethodFillsContract(factory3);
   }
   }

Tony Morris >> The equivalent JUnit test would have the test case methods all beginning with test’, and for the case of an expected exception, it would explicitly fail the test case if the exception didn’t occur. Also, a JUnit test for the equals/hashCode implementations and serialization contract would be extremely painful to write without third party support. The important point here is that 100% coverage has been achieved over several passes with relatively minimal effort. The equivalent JUnit test case to achieve the same amount of coverage is extremely verbose and laborsome.

“JTiger includes built-in reporting, in HTML, XML, plain text….”

JTiger includes built-in reporting, in HTML, XML, plain text, or your own implementation of the org.jtiger.framework.FixtureResultsHandler interface. Attached is the HTML report for the given example.

IndicThreads >> JUnit has now been integrated with most Java IDEs and there are many other tools meant to simplify writing JUnit test cases. Is something similar happening with JTiger?

There has been some mention by JTiger users to get together and write a JTiger plugin for both Eclipse and Intellij IDEA, but nothing has substantiated from it yet. Until then, I simply use the command line execution environment from within my preferred IDE so that I can take advantage of such features as debugging, etc. and I don’t feel like I’m missing out on anything by doing so.

I have published instructions for doing this for Intellij IDEA here: http://www.jtiger.org/articles/jtiger-idea.html
The same instructions could easily be applied to Eclipse, NetBeans or an environment of your choice.

IndicThreads >> Because many Java users found JUnit inadequate to test web based / enterprise applications, we have seen the emergence of many frameworks that build on top of JUnit and simplify testing web/J2EE applications. What is JTiger’s status as regards testing J2EE applications?

Tony Morris >> I was working on implementing the J2EE specification at the end of last year (2004), where I used JTiger as a unit test framework for the work I was doing. At that time, JTiger was a prototype without a public release, and it met all my expectations in a J2EE environment. I also have a Java Questions and Answers database with a J2EE front-end application at http://qa.jtiger.org/. I wrote the application using the WebWork Model-2 web framework and Hibernate 3.x for data store and all components are tested entirely using JTiger. I know of quite a few users who work exclusively with J2EE and use JTiger for unit testing without any complaints.

To answer your question, whether or not I’m in a J2EE development environment, or J2SE, I choose JTiger as my unit test framework. I certainly have no compelling reason to do otherwise. JTiger is neutral in this respect.

I have thought about extending some of the assertions for a J2EE specific domain. For example, wouldn’t it be nice to write a JDBC 2.1 driver, then in your unit test, simply say: assertJdbc21Compliant(java.sql.Driver)? I’ve also thought about doing something for the emerging EJB 3.0 specification.

IndicThreads >> JTiger makes extensive use of the new J2SE 5.0 annotations. What are the advantages of going with annotations as compared to earlier ways of writing test cases?

Tony Morris >> Annotations provide the ability to specify metadata on Java language constructs and be able to retain that metadata information at runtime and look it up using reflection. This is in contrast to what existed prior to annotations, which was typically the specification of metadata as mere javadoc comments, which of course, is lost at compile-time.

It will also hopefully prevent the common misuse of interfaces as metadata; specifically, declaring interfaces without methods (aka tagging or marker interfaces) since interfaces are best used for interfacing to a system, not for specification of metadata. Declaration of an interface without methods and without inheriting from two or more interfaces is a horrible practice.

“JTiger takes advantage of the annotation language construct….”

JTiger takes advantage of the annotation language construct by permitting the developer to specify metadata on their unit tests and that metadata will appear in test reports or allow the developer/tester to alter the way in which the unit test is executed.

IndicThreads >> What are your thoughts on test driven development (TDD)? Do you recommend using the TDD approach to anyone planning to start using JTiger?

Tony Morris >> I think Test Driven Development is an excellent basis for discovering correct software development processes. I think that by completely understanding the reasoning behind TDD allows a developer to make variations to suit their specific needs in a given context. As a part-time university lecturer, I wish these types of topics were taught (properly) in tertiary institutions so that the next batch of new developers will be able to bring some refreshing perspectives to the industry.

“Test Driven Development is an excellent basis for discovering correct software development processes….”

I prefer to describe the fundamental basis of correctness by using requirement analysis techniques, which TDD inadvertently encourages. A trivial example is my preference to view unit tests as actual software requirement specifications, since the developer is not actually testing anything, more so, they are specifying the requirements of their software. It is only at validation time where any testing takes place. I think it’s often severely under-estimated and misunderstood just how important it is to specify software requirements correctly and concisely.

“It’s often severely under-estimated and misunderstood just how important
it is to specify software requirements correctly and concisely….”

I also find that code coverage tools provide an excellent learning curve for refining techniques to specify requirements. For example, most developers who have used a code coverage tool for their unit tests have found a segment of code that they were unable to execute with their unit tests. This begs the question, just what is the purpose of that piece of code if it near impossible to execute it? This kind of reasoning leads to the discovery of many common software design practices that turn out to be extremely flawed. Though admittedly, it does require some serious reflection, and mind bending objectivity and reasoning to really obtain or approach a level of clarity.

IndicThreads >> Finally, with so many unit testing frameworks now available, do you think developer reluctance to unit test code has been suitably tackled or does more need to be done?

Tony Morris >> I believe that there is a huge amount to do. I also believe that there is a horribly ill-informed critical mass of software developers that are reluctant to change their ways to reflect the major advantages that can be gained from integrating unit testing into the software development process. In my teaching, learning and communication with developers, I tend to target the naïve mind and encourage objective thinking and analysis on their behalf so as not to provide them with any of my biases and preconceptions.

“There’s a huge amount to do to tackle developer reluctance to unit test code….”

 “A horribly ill-informed critical mass of software developers that are reluctant to change their ways….”

I find it very satisfying when a discovery is made by a fresh graduate that not even a claimed experienced developer could possibly understand. I think the key to discovering what needs to be done and to encourage sound development techniques, is objectivity, with less emphasis on years in industry and the ever dominant egotism. After all, the industry is relatively immature. Who knows which direction we go next? Importantly, never cease the pursuit of knowledge and learning and emerging development and unit testing techniques have the potential to rapidly increase the rate of learning. Even the most knowledgeable software developer is a mere drop of water in the ocean of knowledge. The rest will fall into place.

“Even the most knowledgeable software developer is a mere drop of water in the ocean of knowledge….”

IndicThreads >> Thanks Tony. Any final notes for our readers? Also could you share your blog url / email?

Tony Morris >> Remember that learning from the mistakes of others is cheap, and to always think for yourself. Unit testing is a fundamental process of software development since it can directly digress to requirements analysis. A software solution cannot exist without a well-stated problem, or from another perspective, a software solution cannot possibly exceed the quality of the problem statement. I don’t have a blog, but my personal website is http://www.tmorris.net/ or please feel free to drop me a line at tmorris AT tmorris DOT net